Commit Graph

28 Commits

Author SHA1 Message Date
Zach Koopmans ce32c06843 Import benchmark-tools to main repository.
This has adapted for use with bazel from the original commit
a26e93769ebefd82593a43e22fb13a09717cfa6d.

In particular, the style has been made consistent with internal python style
guidelines, and the packages (including the main entrypoint) have been
refactored in order to allow bazel testing targets.

PiperOrigin-RevId: 283484433
2019-12-02 22:53:26 -08:00
Adin Scannell b6a00aa375 Use a GitHub credential for tagging a release.
PiperOrigin-RevId: 281617882
2019-11-20 15:39:34 -08:00
Andrei Vagin 548d65b2b6 kokoro: correct a path to outputs.zip
PiperOrigin-RevId: 280021914
2019-11-12 12:01:37 -08:00
Andrei Vagin b91ad8fa09 test: merge log files of all shards for each test suite
This significantly speeds up a process of uploading this files
to sponge and resultstore by kokoro.

PiperOrigin-RevId: 279416349
2019-11-08 16:40:51 -08:00
Adin Scannell e904823833 Fix repository build scripts.
This fixes a number of issues with the repository build process:

 * Fix the overall structure of the repository.
 * Fix the debian package description.
 * Fix the broken version number for packages.
 * Update the digest algorithm used for signing the release.

I've validated that installation works from a separate staging bucket.

Updates #852

PiperOrigin-RevId: 278716914
2019-11-05 15:16:04 -08:00
Andrei Vagin 493334f8b5 kokoro: run KVM syscall tests
We don't know how stable they are, so let's start with warning.

PiperOrigin-RevId: 278484186
2019-11-04 16:00:34 -08:00
Brad Burlage 7dcfcd53e4 Fix overloaded use of $RUNTIME.
Turns out we use $RUNTIME in scripts/common.sh to give a name to the runsc
runtime used by the tests.

PiperOrigin-RevId: 277764383
2019-10-31 11:28:18 -07:00
Brad Burlage df125c9869 Add Kokoro config for new runtime tests
PiperOrigin-RevId: 277607217
2019-10-30 16:16:15 -07:00
Andrei Vagin 8720bd643e netstack/tcp: software segmentation offload
Right now, we send each tcp packet separately, we call one system
call per-packet. This patch allows to generate multiple tcp packets
and send them by sendmmsg.

The arguable part of this CL is a way how to handle multiple headers.
This CL adds the next field to the Prepandable buffer.

Nginx test results:

Server Software:        nginx/1.15.9
Server Hostname:        10.138.0.2
Server Port:            8080

Document Path:          /10m.txt
Document Length:        10485760 bytes

w/o gso:
Concurrency Level:      5
Time taken for tests:   5.491 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      1048600200 bytes
HTML transferred:       1048576000 bytes
Requests per second:    18.21 [#/sec] (mean)
Time per request:       274.525 [ms] (mean)
Time per request:       54.905 [ms] (mean, across all concurrent requests)
Transfer rate:          186508.03 [Kbytes/sec] received

sw-gso:

Concurrency Level:      5
Time taken for tests:   3.852 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      1048600200 bytes
HTML transferred:       1048576000 bytes
Requests per second:    25.96 [#/sec] (mean)
Time per request:       192.576 [ms] (mean)
Time per request:       38.515 [ms] (mean, across all concurrent requests)
Transfer rate:          265874.92 [Kbytes/sec] received

w/o gso:
$ ./tcp_benchmark --client --duration 15  --ideal
[SUM]  0.0-15.1 sec  2.20 GBytes  1.25 Gbits/sec

software gso:
$ tcp_benchmark --client --duration 15  --ideal --gso $((1<<16)) --swgso
[SUM]  0.0-15.1 sec  3.99 GBytes  2.26 Gbits/sec

PiperOrigin-RevId: 276112677
2019-10-22 11:55:56 -07:00
Michael Pratt 03ce4dd86c Remove extra --rm
PiperOrigin-RevId: 272324038
2019-10-01 16:45:46 -07:00
Fabricio Voznika 739f53fc17 Add runsc logs to kokoro artifacts
PiperOrigin-RevId: 272286122
2019-10-01 13:51:10 -07:00
Andrei Vagin 3221e8372c kokoro: don't force to use python2
https://github.com/bazelbuild/bazel/issues/7899 was fixed
and we don't need this hack anymore.

PiperOrigin-RevId: 271434565
2019-09-26 14:37:19 -07:00
Fabricio Voznika 129c67d68e Fix runsc log collection in kokoro
PiperOrigin-RevId: 271207152
2019-09-25 14:33:11 -07:00
Andrei Vagin 2fb34c8d5c test: don't use designated initializers
This change fixes compile errors:
pty.cc:1460:7: error: expected primary-expression before '.' token
...

PiperOrigin-RevId: 271033729
2019-09-24 19:05:12 -07:00
Fabricio Voznika a1f8446921 Fix dev.sh --refresh to create target dir
PiperOrigin-RevId: 269921234
2019-09-18 16:56:01 -07:00
Adin Scannell 461123ea35 Move the component into the repository structure.
The RELEASE file must be at the top-level for the signed
repository to work correctly.

PiperOrigin-RevId: 269897109
2019-09-18 14:57:12 -07:00
Andrei Vagin b63d56b0b4 scripts/build.sh: fix kokoro failure "KOKORO_BUILD_NIGHTLY: unbound variable"
PiperOrigin-RevId: 269690988
2019-09-17 17:26:01 -07:00
Fabricio Voznika 010b093258 Bring back to life features lost in recent refactor
- Sandbox logs are generated when running tests
- Kokoro uploads the sandbox logs
- Supports multiple parallel runs
- Revive script to install locally built runsc with docker

PiperOrigin-RevId: 269337274
2019-09-16 08:17:00 -07:00
Adin Scannell 574eda8880 Update repository directory structure.
Currently it will not work with apt out of the box, as we
require the dists/ prefix, along with a distribution name.

This tweaks the overall structure to allow for the same URL
prefix to be used for all repositories, and enables multiple
architectures.

Fixes #852

PiperOrigin-RevId: 268756104
2019-09-12 13:44:18 -07:00
Adin Scannell 69f2c41b7a Drop unavailable package.
PiperOrigin-RevId: 268614014
2019-09-11 22:30:28 -07:00
Adin Scannell 96a25e080c Ensure appropriate tools are installed on image.
PiperOrigin-RevId: 268608466
2019-09-11 22:13:26 -07:00
Adin Scannell 1e6bdd5855 Update key environment variables.
PiperOrigin-RevId: 268604220
2019-09-11 21:04:42 -07:00
Adin Scannell c06ef5131f Fix authorization for continuous integration.
The credentials must be explicitly refreshed for pushing to
the repository on the Go branch.

PiperOrigin-RevId: 268589817
2019-09-11 18:49:53 -07:00
Adin Scannell 849c57314f Fix minor Kokoro issues.
A recent Kokoro change pointed to go_tests.cfg (in line with the
other configurations), which unfortunately broke the presubmits.

This change also enabled the KVM tests, which were still using a
remote execution strategy.

This fixes both of these issues and allows presubmits to pass.

One additional test was caught with this case, which seems to
have been broken. It's unclear why this was not being caught.

PiperOrigin-RevId: 268166291
2019-09-10 00:38:52 -07:00
Adin Scannell 1a0a940587 Fix repository build scripts
This has the following fixes:

* Packages are passed to the tools/make_repository.sh command.
* All matching tags are built, for commits with multiple.
* The binary path is generated by the build command.
* Output from signing the repository is supressed.
* Allow a release author.

Change-Id: I2d08954ba76e35612f352be99d5bb99080f80892
2019-09-06 14:28:12 -07:00
Adin Scannell 91518fd553
Fix build when no tags are present
This should correct the continuous build.
2019-09-04 22:24:42 -07:00
Adin Scannell bcddd0a477 Fix continuous build breakage
PiperOrigin-RevId: 267277711
2019-09-04 18:49:19 -07:00
Adin Scannell 67a2ab1438 Impose order on test scripts.
The simple test script has gotten out of control. Shard this script into
different pieces and attempt to impose order on overall test structure. This
change helps lay some of the foundations for future improvements.

 * The runsc/test directories are moved into just test/.
 * The runsc/test/testutil package is split into logical pieces.
 * The scripts/ directory contains new top-level targets.
 * Each test is now responsible for building targets it requires.
 * The install functionality is moved into `runsc` itself for simplicity.
 * The existing kokoro run_tests.sh file now just calls all (can be split).

After this change is merged,  I will create multiple distinct workflows for
Kokoro, one for each of the scripts currently targeted by `run_tests.sh` today,
which should dramatically reduce the time-to-run for the Kokoro tests, and
provides a better foundation for further improvements to the infrastructure.

PiperOrigin-RevId: 267081397
2019-09-03 22:02:43 -07:00