Commit Graph

79 Commits

Author SHA1 Message Date
Craig Chi 21d0334e7f Add FUSE integration test
This commit adds an integration test framework for FUSE support. Please
refer to the test example and test/fuse/README.md for further details.

Fixes #3098
2020-08-04 12:27:55 -07:00
Adin Scannell c1c643abe7 Delete unused script.
PiperOrigin-RevId: 323676189
2020-07-28 16:23:40 -07:00
Adin Scannell 7881610a3f Use the appropriate remote configuration.
PiperOrigin-RevId: 323646156
2020-07-28 13:53:13 -07:00
Zach Koopmans 2ecf66903e Add profiling to dockerutil
Adds profiling with `runsc debug` or pprof to dockerutil. All
targets using dockerutil should now be able to use profiling.

In addition, modifies existing benchmarks to use profiling.

PiperOrigin-RevId: 323298634
2020-07-26 22:02:51 -07:00
Zach Koopmans b38bae0088 Mark Passing Integration Tests for VFS2.
Mark the currently passing integration/image tests for
VFS2. Bugs will be filed for remaining failing tests.

Updates #1487

PiperOrigin-RevId: 323297260
2020-07-26 21:43:46 -07:00
Ayush Ranjan 03c30ec634 Runtime tests: Download language runtime image
PiperOrigin-RevId: 321839398
2020-07-17 13:07:55 -07:00
gVisor bot c81ac8ec3b Merge pull request #2672 from amscanne:shim-integrated
PiperOrigin-RevId: 321053634
2020-07-13 16:10:58 -07:00
Ayush Ranjan 28c635e5d4 Fix runtime test target
PiperOrigin-RevId: 321029113
2020-07-13 14:07:41 -07:00
Zach Koopmans b8d3d09bd1 Initial golang Benchmarks
PiperOrigin-RevId: 321021071
2020-07-13 13:29:30 -07:00
Kevin Krakauer f1aa919181 Ensure ip6tables module installed
This module isn't always loaded automatically.

PiperOrigin-RevId: 317164471
2020-06-18 13:06:50 -07:00
Kevin Krakauer 20afd66e01 Invoke bazel query via bash function.
PiperOrigin-RevId: 315514034
2020-06-09 10:48:24 -07:00
Adin Scannell 2c6c4365ea Move to make for tag release workflow.
This will make tag & release workflows idempotent.

PiperOrigin-RevId: 314154888
2020-06-01 10:31:26 -07:00
Adin Scannell 04a1f431e3 Fix the APT repository structure.
This change fixes the apt repository structure to avoid emiting warnings on
Ubuntu 18.04 (and potentially other versions). This requires a slight refactor
of the repository generation scripts, since we can no longer copy the same
release files for different "suites".

This should avoid the warning by setting the suite to the distribution:
  https://github.com/Debian/apt/blob/master/apt-pkg/metaindex.cc#L75

This change also moves over to the standardized Makefile entrypoint, which
makes settings clearer and enables local testing.

PiperOrigin-RevId: 313817017
2020-05-29 11:30:02 -07:00
Andrei Vagin c01e103256 Allow to run kvm syscall tests on the RBE cluster
PiperOrigin-RevId: 309265978
2020-04-30 11:33:07 -07:00
Zach Koopmans 15a822a193 VFS2: Get HelloWorld image tests to pass with VFS2
This change includes:
- Modifications to loader_test.go to get TestCreateMountNamespace to
pass with VFS2.
- Changes necessary to get TestHelloWorld in image tests to pass with
VFS2. This means runsc can run the hello-world container with docker
on VSF2.

Note: Containers that use sockets will not run with these changes.
See "//test/image/...". Any tests here with sockets currently fail
(which is all of them but HelloWorld).
PiperOrigin-RevId: 308363072
2020-04-24 18:23:37 -07:00
Adin Scannell c60613475c Standardize all Docker images.
This change moves all Docker images to a standard location, and abstracts the
build process so that they can be maintained in an automated fashion. This also
allows the images to be architecture-independent.

All images will now be referred to by the test framework via the canonical
`gvisor.dev/images/<name>`, where `<name>` is a function of the path within the
source tree.

In a subsequent change, continuous integration will be added so that the images
will always be correct and available locally.

In the end, using `bazel` for Docker containers is simply not possible. Given
that we already have the need to use `make` with the base container (for
Docker), we extend this approach to get more flexibility.

This change also adds a self-documenting and powerful Makefile that is intended
to replace the collection of scripts in scripts. Canonical (self-documenting)
targets can be added here for targets that understand which images need to be
loaded and/or built.

PiperOrigin-RevId: 308322438
2020-04-24 14:11:42 -07:00
Andrei Vagin f87964e829 kokoro: save all files from test.outputs/
If a test fails by timeout, bazel doesn't generate outputs.zip.

PiperOrigin-RevId: 308307815
2020-04-24 12:52:49 -07:00
Adin Scannell 1481499fe2 Simplify Docker test infrastructure.
This change adds a layer of abstraction around the internal Docker APIs,
and eliminates all direct dependencies on Dockerfiles in the infrastructure.

A subsequent change will automated the generation of local images (with
efficient caching). Note that this change drops the use of bazel container
rules, as that experiment does not seem to be viable.

PiperOrigin-RevId: 308095430
2020-04-23 11:33:30 -07:00
Adin Scannell a27d6329df Remove unnecessary kokoro configurations.
PiperOrigin-RevId: 307841689
2020-04-22 10:15:10 -07:00
Ting-Yu Wang 470633d7e9 Fix release.sh. git commands need to be run in git repo.
PiperOrigin-RevId: 307458938
2020-04-20 12:58:32 -07:00
Adin Scannell 9ba3086d9d Move runtime_tests.sh to align with other scripts.
PiperOrigin-RevId: 307435879
2020-04-20 11:08:59 -07:00
Andrei Vagin 1ebfdcc86c kokoro: fix handling of apt-get errors
When a command is called as if expression, its error
code can be get only in this if block.

For example, the next script prints 0:

if ( false ); then
  true
fi
echo $?

PiperOrigin-RevId: 305638629
2020-04-09 01:12:27 -07:00
Zach Koopmans 035836193e Fix typo in benchmarks.sh
PiperOrigin-RevId: 304440599
2020-04-02 11:25:26 -07:00
Zach Koopmans 37025990d6 Add "/snap/bin" to PATH.
"gcloud" may be installed as a snap, under "/snap/bin". Make
sure this is in our PATH so that we can use gcloud.

PiperOrigin-RevId: 304297180
2020-04-01 17:40:19 -07:00
Zach Koopmans 57e67e32b5 Debug script issues on Kokoro.
PiperOrigin-RevId: 304068950
2020-03-31 16:17:56 -07:00
Zach Koopmans 4aee370640 Internal change.
PiperOrigin-RevId: 303773475
2020-03-30 10:44:55 -07:00
Kevin Krakauer 4f374da60c iptable: fix tests timeouts
Tests were run assuming a runtime of "runsc" was present, and did not
have --net-raw enabled.
2020-03-25 20:37:48 -07:00
Eyal Soha e0fbcdcb7f Add packetimpact tests to presubmit and CI testing
PiperOrigin-RevId: 301872161
2020-03-19 12:33:29 -07:00
Zach Koopmans d8cf54f20f Internal change
PiperOrigin-RevId: 300836270
2020-03-13 15:18:14 -07:00
Zach Koopmans b8fda7f34f Run "startup" benchmark in Kokoro benchmark job.
PiperOrigin-RevId: 300798423
2020-03-13 11:51:50 -07:00
Kevin Krakauer 4054b021f0 iptables: ready tests to be enabled in kokoro
Fixed flakes (tested via --runs_per_test=100) and added skips for
not-yet-implemented features. Once submitted, the iptables tests will be
ready to enable in kokoro.
2020-03-11 15:13:58 -07:00
Zach Koopmans fd84cddab0 Internal change
PiperOrigin-RevId: 300410856
2020-03-11 14:46:17 -07:00
Fabricio Voznika b23999f3e4 Fix runsc permissions for "dev.sh --refresh"
PiperOrigin-RevId: 299466906
2020-03-06 17:01:51 -08:00
Adin Scannell c96bb4d2eb Fix apt-get reliability issues.
This is frequently causing the core build scripts to fail. The core ubuntu
distribution will perform an auto-update at first start, which may cause the
lock file to be held. All apt-get commands may be done in a loop in order to
retry to avoid this issue. We may want to consider retrying other pieces, but
for now this should avoid the most frequent cause of build flakes.

PiperOrigin-RevId: 297704789
2020-02-27 15:36:13 -08:00
Adin Scannell 8e2b14fecf Use automated release notes, if available.
PiperOrigin-RevId: 297628615
2020-02-27 10:22:41 -08:00
Brad Burlage ededa90d07 Internal change.
PiperOrigin-RevId: 296972565
2020-02-24 14:22:32 -08:00
gVisor bot 3a20eccf8b Internal change.
PiperOrigin-RevId: 296055452
2020-02-19 14:18:11 -08:00
gVisor bot cf1e50a809 Minor runtime test fixes.
* Allow scripts/common.sh to be sourced from outside the scripts/ directory
* Fix passing empty args to Bazel, which causes the tool to exit with a failure
  even if the command succeeds.

PiperOrigin-RevId: 294785456
2020-02-12 16:27:07 -08:00
Brad Burlage f2d3efca1d Fix undeclared variable error in common_build.sh.
PiperOrigin-RevId: 293413711
2020-02-05 11:34:46 -08:00
Eyal Soha 37abbbc547 Add packetdrill tests to presubmit and CI testing
PiperOrigin-RevId: 293409718
2020-02-05 11:18:02 -08:00
Brad Burlage 6cd7901d7d Add 1 Kokoro job per runtime test.
PiperOrigin-RevId: 293019326
2020-02-03 15:56:57 -08:00
Kevin Krakauer 0ade523f06 Fix iptables tests that were broken by rename.
The name of the runner binary target changed from "runner" to "runner-image",
causing iptables tests to fail.

PiperOrigin-RevId: 292242263
2020-01-29 16:27:12 -08:00
Adin Scannell d29e59af9f Standardize on tools directory.
PiperOrigin-RevId: 291745021
2020-01-27 12:21:00 -08:00
Brad Burlage 98e83c444f Try running kythe build on RBE.
Also add our RBE project/instance to the --config=remote defaults.

PiperOrigin-RevId: 291201426
2020-01-23 11:01:38 -08:00
Adin Scannell 04e3d56db1 Fix master build case.
Otherwise, this will be built when building a specific
release tag (typically in the past), causing the master
binary to be overwritten with something older.

We can generally assume that tags will be applied after
the commit has been integrated, and therefore that any
builds pointing to tags will use only the tags.

Another way to fix this would be to introduce something
akin to the KOKORO_BUILD_NIGHTLY environment variable,
but it doesn't seem strictly necessary.

PiperOrigin-RevId: 291198171
2020-01-23 10:45:48 -08:00
Tamir Duberstein debd213da6 Allow dual stack sockets to operate on AF_INET
Fixes #1490
Fixes #1495

PiperOrigin-RevId: 289523250
2020-01-13 14:47:22 -08:00
Fabricio Voznika 290908fa8a Configure issue reviver to run with Kokoro
PiperOrigin-RevId: 288921032
2020-01-09 10:17:54 -08:00
Kevin Krakauer be2754a4b9 Add iptables testing framework.
It would be preferrable to test iptables via syscall tests, but there are some
problems with that approach:

* We're limited to loopback-only, as syscall tests involve only a single
  container. Other link interfaces (e.g. fdbased) should be tested.
* We'd have to shell out to call iptables anyways, as the iptables syscall
  interface itself is too large and complex to work with alone.
* Running the Linux/native version of the syscall test will require root, which
  is a pain to configure, is inherently unsafe, and could leave host iptables
  misconfigured.

Using the go_test target allows there to be no new test runner.

PiperOrigin-RevId: 285274275
2019-12-12 14:42:11 -08:00
Adin Scannell cf477c86ca Mark runner_test as manual.
Because it is local-only, it should also be marked manual.

PiperOrigin-RevId: 284596186
2019-12-09 11:28:41 -08:00
Fabricio Voznika 3c2e2f7d12 Update Kokoro image to install Golang 1.13
PiperOrigin-RevId: 284308422
2019-12-06 20:13:09 -08:00