Commit Graph

41 Commits

Author SHA1 Message Date
Michael Pratt aab7d75bd8 Fix minor typos
PiperOrigin-RevId: 351606635
2021-01-13 09:51:38 -08:00
Adin Scannell 4e03e87547 Fix simple mistakes identified by goreportcard.
These are primarily simplification and lint mistakes. However, minor
fixes are also included and tests added where appropriate.

PiperOrigin-RevId: 351425971
2021-01-12 12:38:22 -08:00
Adin Scannell 776016ac64 Fix native benchmarks.
PiperOrigin-RevId: 350509137
2021-01-07 01:17:10 -08:00
Adin Scannell b06e5bc5b0 Add benchmarks targets to BuildKite.
This includes minor fix-ups:

* Handle SIGTERM in runsc debug, to exit gracefully.
* Fix cmd.debug.go opening all profiles as RDONLY.
* Fix the test name in fio_test.go, and encode the block size in the test.

PiperOrigin-RevId: 350205718
2021-01-05 13:21:54 -08:00
Adin Scannell ffa9a715aa Simplify profiling and benchmarks.
- Tweak the benchmarks to work with b.N where appropriate. In many cases,
  b.N was simply being ignored. This creates an implicit dependency in the
  user passing a reasonable benchtime (less than or equal to the actual
  runtime of the test, or using the X syntax) otherwise the test runs
  forever.
- In cases where the above is impossible, explicitly set benchtime from
  the test wrapper, to prevent the above behavior (tensorflow).
- Drop the *Reverse variants, which are simply hey benchmarks. We should
  just add a hey benchmark. The platforms benchmarks already include a
  native platform, and thus these benchmarks are incredibly confusing.
  (In other words, BenchmarkNginxReverse has nothing to do with an nginx
  benchmark for runsc.)
- Remove the redunant Harness object, which contains no state, in order
  to slightly simplify the code.
- Make Block and Heap profiling actually work, but setting appropriate
  runtime parameters (and plumbing them through the config).
- Split the profiling into two phases: start and stop, since some will
  need to be started early, and others will need to happen at the end.

PiperOrigin-RevId: 349495377
2020-12-29 18:29:12 -08:00
Zach Koopmans 4b697aae55 Fix long running bazel build jobs.
- Skip the bazel clean command on the last run of the benchmark.
- Use --test.benchtime=1ns to force running the benchmark once
(https://github.com/golang/go/issues/32051)

PiperOrigin-RevId: 347124606
2020-12-11 20:56:09 -08:00
Zach Koopmans 305a456551 Adjust requests to be constant equal to b.N in network tests.
For "hey", requests >= concurrency. b.N can be set by the
--test.benchtime={b.N}x. The previous setting of b.N * c
can be surprisingly slow for larger c.

Set the requests to max(b.N, c) and log to the user if it is c.

PiperOrigin-RevId: 347053675
2020-12-11 12:37:28 -08:00
Zach Koopmans b4af9d4572 Add network benchmarks jobs
Add httpd, nginx, node, and ruby benchmarks to continuous jobs.

PiperOrigin-RevId: 346629115
2020-12-09 15:57:58 -08:00
Zach Koopmans 992769c774 Add tensorflow, ffmpeg, and redis jobs.
PiperOrigin-RevId: 346603153
2020-12-09 15:56:08 -08:00
Zach Koopmans 6d30688bd7 Fix tags on benchmark targets.
PiperOrigin-RevId: 346203209
2020-12-07 16:20:12 -08:00
Jamie Liu 786f32c2b1 Do not os.Exit() from test/benchmarks/harness.Harness.Init with no args.
PiperOrigin-RevId: 344896991
2020-11-30 15:09:28 -08:00
Ghanan Gowripalan cc5cfce4c6 Remove ARP address workaround
- Make AddressableEndpoint optional for NetworkEndpoint.
Not all NetworkEndpoints need to support addressing (e.g. ARP), so
AddressableEndpoint should only be implemented for protocols that
support addressing such as IPv4 and IPv6.

With this change, tcpip.ErrNotSupported will be returned by the stack
when attempting to modify addresses on a network endpoint that does
not support addressing.

Now that packets are fully handled at the network layer, and (with this
change) addresses are optional for network endpoints, we no longer need
the workaround for ARP where a fake ARP address was added to each NIC
that performs ARP so that packets would be delivered to the ARP layer.

PiperOrigin-RevId: 342722547
2020-11-16 14:36:10 -08:00
Zach Koopmans 792cbc06de Add debug logs to startup benchmark.
PiperOrigin-RevId: 341757694
2020-11-10 21:20:52 -08:00
Zach Koopmans 267d184084 Add all base and fs tests to Continuous Tests.
PiperOrigin-RevId: 341660511
2020-11-10 11:27:50 -08:00
Zach Koopmans 9035422ea4 Change size of startup benchmark and lower number of startups.
PiperOrigin-RevId: 341155693
2020-11-06 19:00:15 -08:00
Zach Koopmans b576de907c Add parsers golang benchmarks.
Add parser and formatting for golang benchmarks for docker benchmarks.
Change adds a library for printing and parsing Test parameters and metrics.
Benchmarks use the library to print parameters in the Benchmark title
(e.g. the name field in b.Run()), and to report CustomMetrics. Parser
uses the library to parse printed data from benchmark output and
put it into BigQuery structs.

PiperOrigin-RevId: 336365628
2020-10-09 14:29:21 -07:00
Ghanan Gowripalan a5acc0616c Support creating protocol instances with Stack ref
Network or transport protocols may want to reach the stack. Support this
by letting the stack create the protocol instances so it can pass a
reference to itself at protocol creation time.

Note, protocols do not yet use the stack in this CL but later CLs will
make use of the stack from protocols.

PiperOrigin-RevId: 334260210
2020-09-28 16:24:04 -07:00
Zach Koopmans c3fc69022a Fix Nginx Startup and Size Benchmarks.
Changes in Nginx Benchmarks in network_tests also affect Startup/Size
Nginx Benchmarks. Make sure the commands line up.

PiperOrigin-RevId: 333543697
2020-09-24 10:32:01 -07:00
Ghanan Gowripalan d35f07b36a Improve type safety for transport protocol options
The existing implementation for TransportProtocol.{Set}Option take
arguments of an empty interface type which all types (implicitly)
implement; any type may be passed to the functions.

This change introduces marker interfaces for transport protocol options
that may be set or queried which transport protocol option types
implement to ensure that invalid types are caught at compile time.
Different interfaces are used to allow the compiler to enforce read-only
or set-only socket options.

RELNOTES: n/a
PiperOrigin-RevId: 330559811
2020-09-08 12:17:39 -07:00
Zach Koopmans b9b6660dc4 Add Docs to nginx benchmark.
Adds docs to nginx and refactors both Httpd and Nginx benchmarks.

Key changes:
- Add docs and make nginx tests the same as httpd (reverse, all docs, etc.).
- Make requests scale on c * b.N -> a request per thread. This works well
with both --test.benchtime=10m (do a run that lasts at least 10m) and
--test.benchtime=10x (do b.N = 10).
-- Remove a doc from both tests (1000Kb) as 1024Kb exists.

PiperOrigin-RevId: 329751091
2020-09-02 11:22:17 -07:00
Zach Koopmans 6748438493 Fix bug in bazel build benchmark.
PiperOrigin-RevId: 329409802
2020-08-31 17:17:09 -07:00
Adin Scannell b0c53f8475 Add nogo support to go_binary and go_test targets.
Updates #3374

PiperOrigin-RevId: 328378700
2020-08-25 12:18:25 -07:00
Zach Koopmans a88cf5a2e1 Add benchmarks to continuous build.
PiperOrigin-RevId: 325892974
2020-08-10 14:52:36 -07:00
Zach Koopmans 80c80a1410 Remove old benchmark tools.
Remove the old benchmark-tools directory, including
imports in the WORKSPACE file and associated bazel rules.

The new Golang benchmark-tools can be found at //test/benchmarks
and it is functionally equivalent, excepting syscall_test
which can be found in //test/perf/linux.

PiperOrigin-RevId: 325529075
2020-08-07 16:18:51 -07:00
Zach Koopmans 7b9bfc0ce0 Port Ruby benchmark.
PiperOrigin-RevId: 325500772
2020-08-07 13:49:07 -07:00
Zach Koopmans a7bd0a7012 Port Startup and Density Benchmarks.
PiperOrigin-RevId: 325497346
2020-08-07 13:30:39 -07:00
Zach Koopmans be7079578e Port sysbench benchmark.
PiperOrigin-RevId: 324918229
2020-08-04 16:51:04 -07:00
Bhasker Hariharan 1bdadbc4f8 Fix broken httpd_test.
PiperOrigin-RevId: 324822613
2020-08-04 09:07:09 -07:00
Bhasker Hariharan 1a93a78d10 Add support for a reverse HTTPD test.
This change adds a new reverse HTTP test where the HTTPD server runs
in a native container but the client runs inside gVisor. It allows
us to test download performance under varying levels of concurrency.

Also tweaks the concurrent request numbers to test for high levels
of concurrency.

PiperOrigin-RevId: 324651203
2020-08-03 12:00:01 -07:00
Zach Koopmans 98f9527c04 Port nginx and move parsers to own package.
This change:
- Ports the nginx benchmark.
- Switches the Httpd benchmark to use 'hey' as a client.
- Moves all parsers to their own package 'tools'.

Parsers are moved to their own package because 1) parsing output of a command
is often dependent on the format of the command (e.g. 'fio --json'), 2) to
enable easier reuse, and 3) clean up and simplify actual running benchmarks
(no TestParser functions and ugly sample output in benchmark files).

PiperOrigin-RevId: 324144165
2020-07-30 21:17:45 -07:00
Zach Koopmans 78f1a18ab3 Add runsc build benchmark.
PiperOrigin-RevId: 324071377
2020-07-30 13:37:21 -07:00
Zach Koopmans 2775ecd931 Update call in Node benchmark.
PiperOrigin-RevId: 324028183
2020-07-30 10:19:14 -07:00
Zach Koopmans 1715896fc8 Port fio benchmark
PiperOrigin-RevId: 323810654
2020-07-29 10:15:53 -07:00
Zach Koopmans 6b4e11ab50 Port node benchmark.
PiperOrigin-RevId: 323810235
2020-07-29 10:06:38 -07:00
Zach Koopmans 5873b0f43f Port tensorflow benchmark.
PiperOrigin-RevId: 323633737
2020-07-28 12:56:06 -07:00
Zach Koopmans 77552f1c77 Port ffmpeg benchmark
PiperOrigin-RevId: 323383320
2020-07-27 10:10:14 -07:00
Zach Koopmans 29e5609b22 Port redis benchmark
PiperOrigin-RevId: 323381964
2020-07-27 10:01:45 -07:00
Zach Koopmans 2ecf66903e Add profiling to dockerutil
Adds profiling with `runsc debug` or pprof to dockerutil. All
targets using dockerutil should now be able to use profiling.

In addition, modifies existing benchmarks to use profiling.

PiperOrigin-RevId: 323298634
2020-07-26 22:02:51 -07:00
Zach Koopmans e3c2bd51a1 Move main methods for benchmark packages main package file.
PiperOrigin-RevId: 321875119
2020-07-17 16:23:10 -07:00
Zach Koopmans 5c8c0d65b9 Port httpd benchmark
PiperOrigin-RevId: 321478001
2020-07-15 18:21:51 -07:00
Zach Koopmans b8d3d09bd1 Initial golang Benchmarks
PiperOrigin-RevId: 321021071
2020-07-13 13:29:30 -07:00