Commit Graph

73 Commits

Author SHA1 Message Date
gVisor bot c81ac8ec3b Merge pull request #2672 from amscanne:shim-integrated
PiperOrigin-RevId: 321053634
2020-07-13 16:10:58 -07:00
Ayush Ranjan 28c635e5d4 Fix runtime test target
PiperOrigin-RevId: 321029113
2020-07-13 14:07:41 -07:00
Zach Koopmans b8d3d09bd1 Initial golang Benchmarks
PiperOrigin-RevId: 321021071
2020-07-13 13:29:30 -07:00
Kevin Krakauer f1aa919181 Ensure ip6tables module installed
This module isn't always loaded automatically.

PiperOrigin-RevId: 317164471
2020-06-18 13:06:50 -07:00
Kevin Krakauer 20afd66e01 Invoke bazel query via bash function.
PiperOrigin-RevId: 315514034
2020-06-09 10:48:24 -07:00
Adin Scannell 2c6c4365ea Move to make for tag release workflow.
This will make tag & release workflows idempotent.

PiperOrigin-RevId: 314154888
2020-06-01 10:31:26 -07:00
Adin Scannell 04a1f431e3 Fix the APT repository structure.
This change fixes the apt repository structure to avoid emiting warnings on
Ubuntu 18.04 (and potentially other versions). This requires a slight refactor
of the repository generation scripts, since we can no longer copy the same
release files for different "suites".

This should avoid the warning by setting the suite to the distribution:
  https://github.com/Debian/apt/blob/master/apt-pkg/metaindex.cc#L75

This change also moves over to the standardized Makefile entrypoint, which
makes settings clearer and enables local testing.

PiperOrigin-RevId: 313817017
2020-05-29 11:30:02 -07:00
Andrei Vagin c01e103256 Allow to run kvm syscall tests on the RBE cluster
PiperOrigin-RevId: 309265978
2020-04-30 11:33:07 -07:00
Zach Koopmans 15a822a193 VFS2: Get HelloWorld image tests to pass with VFS2
This change includes:
- Modifications to loader_test.go to get TestCreateMountNamespace to
pass with VFS2.
- Changes necessary to get TestHelloWorld in image tests to pass with
VFS2. This means runsc can run the hello-world container with docker
on VSF2.

Note: Containers that use sockets will not run with these changes.
See "//test/image/...". Any tests here with sockets currently fail
(which is all of them but HelloWorld).
PiperOrigin-RevId: 308363072
2020-04-24 18:23:37 -07:00
Adin Scannell c60613475c Standardize all Docker images.
This change moves all Docker images to a standard location, and abstracts the
build process so that they can be maintained in an automated fashion. This also
allows the images to be architecture-independent.

All images will now be referred to by the test framework via the canonical
`gvisor.dev/images/<name>`, where `<name>` is a function of the path within the
source tree.

In a subsequent change, continuous integration will be added so that the images
will always be correct and available locally.

In the end, using `bazel` for Docker containers is simply not possible. Given
that we already have the need to use `make` with the base container (for
Docker), we extend this approach to get more flexibility.

This change also adds a self-documenting and powerful Makefile that is intended
to replace the collection of scripts in scripts. Canonical (self-documenting)
targets can be added here for targets that understand which images need to be
loaded and/or built.

PiperOrigin-RevId: 308322438
2020-04-24 14:11:42 -07:00
Andrei Vagin f87964e829 kokoro: save all files from test.outputs/
If a test fails by timeout, bazel doesn't generate outputs.zip.

PiperOrigin-RevId: 308307815
2020-04-24 12:52:49 -07:00
Adin Scannell 1481499fe2 Simplify Docker test infrastructure.
This change adds a layer of abstraction around the internal Docker APIs,
and eliminates all direct dependencies on Dockerfiles in the infrastructure.

A subsequent change will automated the generation of local images (with
efficient caching). Note that this change drops the use of bazel container
rules, as that experiment does not seem to be viable.

PiperOrigin-RevId: 308095430
2020-04-23 11:33:30 -07:00
Adin Scannell a27d6329df Remove unnecessary kokoro configurations.
PiperOrigin-RevId: 307841689
2020-04-22 10:15:10 -07:00
Ting-Yu Wang 470633d7e9 Fix release.sh. git commands need to be run in git repo.
PiperOrigin-RevId: 307458938
2020-04-20 12:58:32 -07:00
Adin Scannell 9ba3086d9d Move runtime_tests.sh to align with other scripts.
PiperOrigin-RevId: 307435879
2020-04-20 11:08:59 -07:00
Andrei Vagin 1ebfdcc86c kokoro: fix handling of apt-get errors
When a command is called as if expression, its error
code can be get only in this if block.

For example, the next script prints 0:

if ( false ); then
  true
fi
echo $?

PiperOrigin-RevId: 305638629
2020-04-09 01:12:27 -07:00
Zach Koopmans 035836193e Fix typo in benchmarks.sh
PiperOrigin-RevId: 304440599
2020-04-02 11:25:26 -07:00
Zach Koopmans 37025990d6 Add "/snap/bin" to PATH.
"gcloud" may be installed as a snap, under "/snap/bin". Make
sure this is in our PATH so that we can use gcloud.

PiperOrigin-RevId: 304297180
2020-04-01 17:40:19 -07:00
Zach Koopmans 57e67e32b5 Debug script issues on Kokoro.
PiperOrigin-RevId: 304068950
2020-03-31 16:17:56 -07:00
Zach Koopmans 4aee370640 Internal change.
PiperOrigin-RevId: 303773475
2020-03-30 10:44:55 -07:00
Kevin Krakauer 4f374da60c iptable: fix tests timeouts
Tests were run assuming a runtime of "runsc" was present, and did not
have --net-raw enabled.
2020-03-25 20:37:48 -07:00
Eyal Soha e0fbcdcb7f Add packetimpact tests to presubmit and CI testing
PiperOrigin-RevId: 301872161
2020-03-19 12:33:29 -07:00
Zach Koopmans d8cf54f20f Internal change
PiperOrigin-RevId: 300836270
2020-03-13 15:18:14 -07:00
Zach Koopmans b8fda7f34f Run "startup" benchmark in Kokoro benchmark job.
PiperOrigin-RevId: 300798423
2020-03-13 11:51:50 -07:00
Kevin Krakauer 4054b021f0 iptables: ready tests to be enabled in kokoro
Fixed flakes (tested via --runs_per_test=100) and added skips for
not-yet-implemented features. Once submitted, the iptables tests will be
ready to enable in kokoro.
2020-03-11 15:13:58 -07:00
Zach Koopmans fd84cddab0 Internal change
PiperOrigin-RevId: 300410856
2020-03-11 14:46:17 -07:00
Fabricio Voznika b23999f3e4 Fix runsc permissions for "dev.sh --refresh"
PiperOrigin-RevId: 299466906
2020-03-06 17:01:51 -08:00
Adin Scannell c96bb4d2eb Fix apt-get reliability issues.
This is frequently causing the core build scripts to fail. The core ubuntu
distribution will perform an auto-update at first start, which may cause the
lock file to be held. All apt-get commands may be done in a loop in order to
retry to avoid this issue. We may want to consider retrying other pieces, but
for now this should avoid the most frequent cause of build flakes.

PiperOrigin-RevId: 297704789
2020-02-27 15:36:13 -08:00
Adin Scannell 8e2b14fecf Use automated release notes, if available.
PiperOrigin-RevId: 297628615
2020-02-27 10:22:41 -08:00
Brad Burlage ededa90d07 Internal change.
PiperOrigin-RevId: 296972565
2020-02-24 14:22:32 -08:00
gVisor bot 3a20eccf8b Internal change.
PiperOrigin-RevId: 296055452
2020-02-19 14:18:11 -08:00
gVisor bot cf1e50a809 Minor runtime test fixes.
* Allow scripts/common.sh to be sourced from outside the scripts/ directory
* Fix passing empty args to Bazel, which causes the tool to exit with a failure
  even if the command succeeds.

PiperOrigin-RevId: 294785456
2020-02-12 16:27:07 -08:00
Brad Burlage f2d3efca1d Fix undeclared variable error in common_build.sh.
PiperOrigin-RevId: 293413711
2020-02-05 11:34:46 -08:00
Eyal Soha 37abbbc547 Add packetdrill tests to presubmit and CI testing
PiperOrigin-RevId: 293409718
2020-02-05 11:18:02 -08:00
Brad Burlage 6cd7901d7d Add 1 Kokoro job per runtime test.
PiperOrigin-RevId: 293019326
2020-02-03 15:56:57 -08:00
Kevin Krakauer 0ade523f06 Fix iptables tests that were broken by rename.
The name of the runner binary target changed from "runner" to "runner-image",
causing iptables tests to fail.

PiperOrigin-RevId: 292242263
2020-01-29 16:27:12 -08:00
Adin Scannell d29e59af9f Standardize on tools directory.
PiperOrigin-RevId: 291745021
2020-01-27 12:21:00 -08:00
Brad Burlage 98e83c444f Try running kythe build on RBE.
Also add our RBE project/instance to the --config=remote defaults.

PiperOrigin-RevId: 291201426
2020-01-23 11:01:38 -08:00
Adin Scannell 04e3d56db1 Fix master build case.
Otherwise, this will be built when building a specific
release tag (typically in the past), causing the master
binary to be overwritten with something older.

We can generally assume that tags will be applied after
the commit has been integrated, and therefore that any
builds pointing to tags will use only the tags.

Another way to fix this would be to introduce something
akin to the KOKORO_BUILD_NIGHTLY environment variable,
but it doesn't seem strictly necessary.

PiperOrigin-RevId: 291198171
2020-01-23 10:45:48 -08:00
Tamir Duberstein debd213da6 Allow dual stack sockets to operate on AF_INET
Fixes #1490
Fixes #1495

PiperOrigin-RevId: 289523250
2020-01-13 14:47:22 -08:00
Fabricio Voznika 290908fa8a Configure issue reviver to run with Kokoro
PiperOrigin-RevId: 288921032
2020-01-09 10:17:54 -08:00
Kevin Krakauer be2754a4b9 Add iptables testing framework.
It would be preferrable to test iptables via syscall tests, but there are some
problems with that approach:

* We're limited to loopback-only, as syscall tests involve only a single
  container. Other link interfaces (e.g. fdbased) should be tested.
* We'd have to shell out to call iptables anyways, as the iptables syscall
  interface itself is too large and complex to work with alone.
* Running the Linux/native version of the syscall test will require root, which
  is a pain to configure, is inherently unsafe, and could leave host iptables
  misconfigured.

Using the go_test target allows there to be no new test runner.

PiperOrigin-RevId: 285274275
2019-12-12 14:42:11 -08:00
Adin Scannell cf477c86ca Mark runner_test as manual.
Because it is local-only, it should also be marked manual.

PiperOrigin-RevId: 284596186
2019-12-09 11:28:41 -08:00
Fabricio Voznika 3c2e2f7d12 Update Kokoro image to install Golang 1.13
PiperOrigin-RevId: 284308422
2019-12-06 20:13:09 -08:00
Adin Scannell 371e210b83 Add runtime tracing.
This adds meaningful annotations to the trace generated by the runtime/trace
package.

PiperOrigin-RevId: 284290115
2019-12-06 17:00:07 -08:00
Zach Koopmans ce32c06843 Import benchmark-tools to main repository.
This has adapted for use with bazel from the original commit
a26e93769ebefd82593a43e22fb13a09717cfa6d.

In particular, the style has been made consistent with internal python style
guidelines, and the packages (including the main entrypoint) have been
refactored in order to allow bazel testing targets.

PiperOrigin-RevId: 283484433
2019-12-02 22:53:26 -08:00
Adin Scannell b6a00aa375 Use a GitHub credential for tagging a release.
PiperOrigin-RevId: 281617882
2019-11-20 15:39:34 -08:00
Andrei Vagin 548d65b2b6 kokoro: correct a path to outputs.zip
PiperOrigin-RevId: 280021914
2019-11-12 12:01:37 -08:00
Andrei Vagin b91ad8fa09 test: merge log files of all shards for each test suite
This significantly speeds up a process of uploading this files
to sponge and resultstore by kokoro.

PiperOrigin-RevId: 279416349
2019-11-08 16:40:51 -08:00
Adin Scannell e904823833 Fix repository build scripts.
This fixes a number of issues with the repository build process:

 * Fix the overall structure of the repository.
 * Fix the debian package description.
 * Fix the broken version number for packages.
 * Update the digest algorithm used for signing the release.

I've validated that installation works from a separate staging bucket.

Updates #852

PiperOrigin-RevId: 278716914
2019-11-05 15:16:04 -08:00