Commit Graph

99 Commits

Author SHA1 Message Date
Kevin Krakauer 8c35614760 iptables: enable iptables tests as nonblocking
PiperOrigin-RevId: 302506064
2020-03-23 13:36:26 -07:00
Michael Pratt 238e80fe38 Automated rollback of changelist 300409401
PiperOrigin-RevId: 301891849
2020-03-19 14:10:37 -07:00
Eyal Soha e0fbcdcb7f Add packetimpact tests to presubmit and CI testing
PiperOrigin-RevId: 301872161
2020-03-19 12:33:29 -07:00
Kevin Krakauer fc16e64396 Automated rollback of changelist 301476456
PiperOrigin-RevId: 301650898
2020-03-18 13:08:11 -07:00
Kevin Krakauer 1cc5a71a0e iptables: fix type in script name, mark some new tests as skipped
PiperOrigin-RevId: 301476456
2020-03-17 16:23:00 -07:00
Zach Koopmans d8cf54f20f Internal change
PiperOrigin-RevId: 300836270
2020-03-13 15:18:14 -07:00
Zach Koopmans fd84cddab0 Internal change
PiperOrigin-RevId: 300410856
2020-03-11 14:46:17 -07:00
Michael Pratt 5ee9bbb15d Upgrade Kythe
PiperOrigin-RevId: 300409401
2020-03-11 14:39:21 -07:00
Brad Burlage ededa90d07 Internal change.
PiperOrigin-RevId: 296972565
2020-02-24 14:22:32 -08:00
gVisor bot 3a20eccf8b Internal change.
PiperOrigin-RevId: 296055452
2020-02-19 14:18:11 -08:00
gVisor bot ddc02e489c Internal change.
PiperOrigin-RevId: 296039442
2020-02-19 13:12:31 -08:00
gVisor bot 90d0851c0e Internal change.
PiperOrigin-RevId: 296016675
2020-02-19 11:36:20 -08:00
gVisor bot 8dae8a10f0 Internal change.
PiperOrigin-RevId: 295857142
2020-02-18 17:04:58 -08:00
gVisor bot cf1e50a809 Minor runtime test fixes.
* Allow scripts/common.sh to be sourced from outside the scripts/ directory
* Fix passing empty args to Bazel, which causes the tool to exit with a failure
  even if the command succeeds.

PiperOrigin-RevId: 294785456
2020-02-12 16:27:07 -08:00
Brad Burlage bc504d5202 Fix build_file in runtimes_tests.
PiperOrigin-RevId: 294286242
2020-02-10 13:10:06 -08:00
Eyal Soha 37abbbc547 Add packetdrill tests to presubmit and CI testing
PiperOrigin-RevId: 293409718
2020-02-05 11:18:02 -08:00
Brad Burlage 6cd7901d7d Add 1 Kokoro job per runtime test.
PiperOrigin-RevId: 293019326
2020-02-03 15:56:57 -08:00
gVisor bot bc3a24d627 Internal change.
PiperOrigin-RevId: 292587459
2020-01-31 13:19:42 -08:00
Brad Burlage 24cfbf4b98 Fix corpus_name to match our ingestion config[1].
PiperOrigin-RevId: 291412676
2020-01-24 11:45:44 -08:00
Brad Burlage 14d2ed1ad7 Fix kythe build.
* Pass --auth_credentials now that we're using RBE
* Fix kzips not being uploaded to the root of the GCS bucket

PiperOrigin-RevId: 291241757
2020-01-23 14:14:24 -08:00
Brad Burlage 98e83c444f Try running kythe build on RBE.
Also add our RBE project/instance to the --config=remote defaults.

PiperOrigin-RevId: 291201426
2020-01-23 11:01:38 -08:00
Adin Scannell cb3906ae00 Add tools for generating images.
This formalizes the adhoc scripts previously in kokoro. The image targets can
be used by e.g. benchmarks in order to automated image prepation.

PiperOrigin-RevId: 290982744
2020-01-22 10:39:45 -08:00
Nicolas Lacasse a944fcd946 Install Bazel 2.0.0 on kokoro images.
PiperOrigin-RevId: 290850738
2020-01-21 18:23:59 -08:00
Fabricio Voznika 290908fa8a Configure issue reviver to run with Kokoro
PiperOrigin-RevId: 288921032
2020-01-09 10:17:54 -08:00
Zach Koopmans e548ce1805 Add python3-pip as dependency for Kokoro VM images.
bm-tools requires python3 and pip3 in order to run
tests. Add pip3 so that dependencies correctly install
for Kokoro runs with bazel.

PiperOrigin-RevId: 286923840
2019-12-23 11:49:29 -08:00
Kevin Krakauer be2754a4b9 Add iptables testing framework.
It would be preferrable to test iptables via syscall tests, but there are some
problems with that approach:

* We're limited to loopback-only, as syscall tests involve only a single
  container. Other link interfaces (e.g. fdbased) should be tested.
* We'd have to shell out to call iptables anyways, as the iptables syscall
  interface itself is too large and complex to work with alone.
* Running the Linux/native version of the syscall test will require root, which
  is a pain to configure, is inherently unsafe, and could leave host iptables
  misconfigured.

Using the go_test target allows there to be no new test runner.

PiperOrigin-RevId: 285274275
2019-12-12 14:42:11 -08:00
Brad Burlage e690651c67 Run kythe build with -std=c++17
We seem to be getting some compiler errors when using the Kythe
extractors.bazelrc.

I'm not able to reproduce this on my local machine, so I'm hoping copying the
cxxopt from gvisor.dev/pr/1350 will fix the build.

PiperOrigin-RevId: 285054258
2019-12-11 14:48:11 -08:00
Brad Burlage a0aa784ecf Remove pyenv calls but log the python 3 version in use.
Apparently our Kokoro VM images don't have pyenv -- I previously tested this on
the Kokoro QA shared pool.

PiperOrigin-RevId: 284855160
2019-12-10 14:55:35 -08:00
Brad Burlage 87337e92e3 Add Kokoro configs for publishing Kythe xrefs.
PiperOrigin-RevId: 284835614
2019-12-10 13:25:27 -08:00
Fabricio Voznika 3c2e2f7d12 Update Kokoro image to install Golang 1.13
PiperOrigin-RevId: 284308422
2019-12-06 20:13:09 -08:00
Adin Scannell b6a00aa375 Use a GitHub credential for tagging a release.
PiperOrigin-RevId: 281617882
2019-11-20 15:39:34 -08:00
Andrei Vagin 14f4461f93 kokoro: update images to install zip
PiperOrigin-RevId: 279406266
2019-11-08 15:45:16 -08:00
Andrei Vagin 50d6236111 Update kokoro images to install junitparser
junitparser will be used to merge junit xml files.

PiperOrigin-RevId: 279387305
2019-11-08 14:08:54 -08:00
Adin Scannell e904823833 Fix repository build scripts.
This fixes a number of issues with the repository build process:

 * Fix the overall structure of the repository.
 * Fix the debian package description.
 * Fix the broken version number for packages.
 * Update the digest algorithm used for signing the release.

I've validated that installation works from a separate staging bucket.

Updates #852

PiperOrigin-RevId: 278716914
2019-11-05 15:16:04 -08:00
Andrei Vagin 493334f8b5 kokoro: run KVM syscall tests
We don't know how stable they are, so let's start with warning.

PiperOrigin-RevId: 278484186
2019-11-04 16:00:34 -08:00
Brad Burlage df125c9869 Add Kokoro config for new runtime tests
PiperOrigin-RevId: 277607217
2019-10-30 16:16:15 -07:00
Andrei Vagin 8720bd643e netstack/tcp: software segmentation offload
Right now, we send each tcp packet separately, we call one system
call per-packet. This patch allows to generate multiple tcp packets
and send them by sendmmsg.

The arguable part of this CL is a way how to handle multiple headers.
This CL adds the next field to the Prepandable buffer.

Nginx test results:

Server Software:        nginx/1.15.9
Server Hostname:        10.138.0.2
Server Port:            8080

Document Path:          /10m.txt
Document Length:        10485760 bytes

w/o gso:
Concurrency Level:      5
Time taken for tests:   5.491 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      1048600200 bytes
HTML transferred:       1048576000 bytes
Requests per second:    18.21 [#/sec] (mean)
Time per request:       274.525 [ms] (mean)
Time per request:       54.905 [ms] (mean, across all concurrent requests)
Transfer rate:          186508.03 [Kbytes/sec] received

sw-gso:

Concurrency Level:      5
Time taken for tests:   3.852 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      1048600200 bytes
HTML transferred:       1048576000 bytes
Requests per second:    25.96 [#/sec] (mean)
Time per request:       192.576 [ms] (mean)
Time per request:       38.515 [ms] (mean, across all concurrent requests)
Transfer rate:          265874.92 [Kbytes/sec] received

w/o gso:
$ ./tcp_benchmark --client --duration 15  --ideal
[SUM]  0.0-15.1 sec  2.20 GBytes  1.25 Gbits/sec

software gso:
$ tcp_benchmark --client --duration 15  --ideal --gso $((1<<16)) --swgso
[SUM]  0.0-15.1 sec  3.99 GBytes  2.26 Gbits/sec

PiperOrigin-RevId: 276112677
2019-10-22 11:55:56 -07:00
Fabricio Voznika 739f53fc17 Add runsc logs to kokoro artifacts
PiperOrigin-RevId: 272286122
2019-10-01 13:51:10 -07:00
Adin Scannell 0c4d080631 Ensure runsc is uploaded.
One would reasonably assume that a field named "regex" would expect
a regular expression. However, in this case, one would be wrong.

The "regex" field actually requires "FileSet" [1] syntax.

?\_(?)_/?

[1] http://ant.apache.org/manual/Types/fileset.html

PiperOrigin-RevId: 271917356
2019-09-29 23:49:34 -07:00
Andrei Vagin 7582385f05 kokoro: don't pass KOKORO_REPO_KEY in presubmit jobs
We don't want to upload packages from the presubmit jobs.

This will fix the error:
[11:01:34][ERROR] Cannot inject environment variables into
                  the build without allowed_env_vars regexes.

PiperOrigin-RevId: 271622996
2019-09-27 12:23:51 -07:00
Brad Burlage e6cf60bc0e Fix runsc checksum artifact name.
scripts/build.sh uses sha512sum, not sha256sum.

PiperOrigin-RevId: 270713604
2019-09-23 10:58:18 -07:00
Adin Scannell 191297ecbe Fix artifact pattern for repositories.
PiperOrigin-RevId: 269468986
2019-09-16 18:41:13 -07:00
Adin Scannell 2bbf73d9ed Remove stale configurations.
PiperOrigin-RevId: 268947847
2019-09-13 11:39:55 -07:00
Adin Scannell 1e6bdd5855 Update key environment variables.
PiperOrigin-RevId: 268604220
2019-09-11 21:04:42 -07:00
Adin Scannell c06ef5131f Fix authorization for continuous integration.
The credentials must be explicitly refreshed for pushing to
the repository on the Go branch.

PiperOrigin-RevId: 268589817
2019-09-11 18:49:53 -07:00
Nicolas Lacasse f52dd451ed Bump bazel, rules_go, gazelle, and go toolchain to latest versions.
PiperOrigin-RevId: 268486127
2019-09-11 10:19:50 -07:00
Adin Scannell 849c57314f Fix minor Kokoro issues.
A recent Kokoro change pointed to go_tests.cfg (in line with the
other configurations), which unfortunately broke the presubmits.

This change also enabled the KVM tests, which were still using a
remote execution strategy.

This fixes both of these issues and allows presubmits to pass.

One additional test was caught with this case, which seems to
have been broken. It's unclear why this was not being caught.

PiperOrigin-RevId: 268166291
2019-09-10 00:38:52 -07:00
Adin Scannell 67a2ab1438 Impose order on test scripts.
The simple test script has gotten out of control. Shard this script into
different pieces and attempt to impose order on overall test structure. This
change helps lay some of the foundations for future improvements.

 * The runsc/test directories are moved into just test/.
 * The runsc/test/testutil package is split into logical pieces.
 * The scripts/ directory contains new top-level targets.
 * Each test is now responsible for building targets it requires.
 * The install functionality is moved into `runsc` itself for simplicity.
 * The existing kokoro run_tests.sh file now just calls all (can be split).

After this change is merged,  I will create multiple distinct workflows for
Kokoro, one for each of the scripts currently targeted by `run_tests.sh` today,
which should dramatically reduce the time-to-run for the Kokoro tests, and
provides a better foundation for further improvements to the infrastructure.

PiperOrigin-RevId: 267081397
2019-09-03 22:02:43 -07:00
Adin Scannell 3ec0b64d8c Update image to install docker and fix permissions
PiperOrigin-RevId: 266467361
2019-08-30 15:27:47 -07:00
Adin Scannell 599136b25e Internal change.
PiperOrigin-RevId: 266073190
2019-08-29 13:59:33 -07:00