Commit Graph

62 Commits

Author SHA1 Message Date
Fabricio Voznika 739f53fc17 Add runsc logs to kokoro artifacts
PiperOrigin-RevId: 272286122
2019-10-01 13:51:10 -07:00
Adin Scannell 0c4d080631 Ensure runsc is uploaded.
One would reasonably assume that a field named "regex" would expect
a regular expression. However, in this case, one would be wrong.

The "regex" field actually requires "FileSet" [1] syntax.

?\_(?)_/?

[1] http://ant.apache.org/manual/Types/fileset.html

PiperOrigin-RevId: 271917356
2019-09-29 23:49:34 -07:00
Andrei Vagin 7582385f05 kokoro: don't pass KOKORO_REPO_KEY in presubmit jobs
We don't want to upload packages from the presubmit jobs.

This will fix the error:
[11:01:34][ERROR] Cannot inject environment variables into
                  the build without allowed_env_vars regexes.

PiperOrigin-RevId: 271622996
2019-09-27 12:23:51 -07:00
Brad Burlage e6cf60bc0e Fix runsc checksum artifact name.
scripts/build.sh uses sha512sum, not sha256sum.

PiperOrigin-RevId: 270713604
2019-09-23 10:58:18 -07:00
Adin Scannell 191297ecbe Fix artifact pattern for repositories.
PiperOrigin-RevId: 269468986
2019-09-16 18:41:13 -07:00
Adin Scannell 2bbf73d9ed Remove stale configurations.
PiperOrigin-RevId: 268947847
2019-09-13 11:39:55 -07:00
Adin Scannell 1e6bdd5855 Update key environment variables.
PiperOrigin-RevId: 268604220
2019-09-11 21:04:42 -07:00
Adin Scannell c06ef5131f Fix authorization for continuous integration.
The credentials must be explicitly refreshed for pushing to
the repository on the Go branch.

PiperOrigin-RevId: 268589817
2019-09-11 18:49:53 -07:00
Nicolas Lacasse f52dd451ed Bump bazel, rules_go, gazelle, and go toolchain to latest versions.
PiperOrigin-RevId: 268486127
2019-09-11 10:19:50 -07:00
Adin Scannell 849c57314f Fix minor Kokoro issues.
A recent Kokoro change pointed to go_tests.cfg (in line with the
other configurations), which unfortunately broke the presubmits.

This change also enabled the KVM tests, which were still using a
remote execution strategy.

This fixes both of these issues and allows presubmits to pass.

One additional test was caught with this case, which seems to
have been broken. It's unclear why this was not being caught.

PiperOrigin-RevId: 268166291
2019-09-10 00:38:52 -07:00
Adin Scannell 67a2ab1438 Impose order on test scripts.
The simple test script has gotten out of control. Shard this script into
different pieces and attempt to impose order on overall test structure. This
change helps lay some of the foundations for future improvements.

 * The runsc/test directories are moved into just test/.
 * The runsc/test/testutil package is split into logical pieces.
 * The scripts/ directory contains new top-level targets.
 * Each test is now responsible for building targets it requires.
 * The install functionality is moved into `runsc` itself for simplicity.
 * The existing kokoro run_tests.sh file now just calls all (can be split).

After this change is merged,  I will create multiple distinct workflows for
Kokoro, one for each of the scripts currently targeted by `run_tests.sh` today,
which should dramatically reduce the time-to-run for the Kokoro tests, and
provides a better foundation for further improvements to the infrastructure.

PiperOrigin-RevId: 267081397
2019-09-03 22:02:43 -07:00
Adin Scannell 3ec0b64d8c Update image to install docker and fix permissions
PiperOrigin-RevId: 266467361
2019-08-30 15:27:47 -07:00
Adin Scannell 599136b25e Internal change.
PiperOrigin-RevId: 266073190
2019-08-29 13:59:33 -07:00
Adin Scannell f048c8be1e Fix permissions on .ssh files
PiperOrigin-RevId: 266066839
2019-08-28 22:36:16 -07:00
Adin Scannell 06ae36185e Fix kokoro image build scripts
The /tmpfs directory needs to be writable, and kokoro needs passwordless sudo
access to execute the test scripts.

PiperOrigin-RevId: 266063723
2019-08-28 22:07:12 -07:00
Adin Scannell 784f48a78d kokoro: Add scripts to rebuild images.
These scripts generated the following images:

gvisor-kokoro-testing/image-a53bac71541a209e (ubuntu 18.04)
gvisor-kokoro-testing/image-f5b20c5fbd23f448 (ubuntu 16.04)

Any modifications to these scripts should generate new images.

PiperOrigin-RevId: 265843929
2019-08-27 23:29:02 -07:00
Adin Scannell 7436ea247b Fix Kokoro revision and 'go get usage'
As a convenience for debugging, also factor the scripts such that
can be run without Kokoro. In the future, this may be used to add
additional presubmit hooks that run without Kokoro.

PiperOrigin-RevId: 251474868
2019-06-04 11:07:27 -07:00
Adin Scannell 216da0b733 Add tooling for Go-compatible branch.
The WORKSPACE go_repositories can be generated from a standard go.mod file. Add
the necessary gazelle hooks to do so, and include a test that sanity checks
there are no changes. This go.mod file will be used in a subsequent commit to
generate a go gettable branch of the repository.

This commit also adds a tools/go_branch.sh script, which given an existing go
branch in the repository, will add an additional synthetic change to the branch
bringing it up-to-date with HEAD.

As a final step, a cloudbuild script is included, which can be used to automate
the process for every change pushed to the repository. This may be used after
an initial go branch is pushed, but this is manual process.

PiperOrigin-RevId: 251095016
2019-06-01 23:10:43 -07:00
Adin Scannell 132bf68de4 Switch to new dedicated RBE project.
PiperOrigin-RevId: 250970783
2019-05-31 15:43:30 -07:00
Andrei Vagin 673358c0d9 runsc/do: allow to run commands in a host network namespace
PiperOrigin-RevId: 250329795
2019-05-30 12:05:16 -07:00
Andrei Vagin 85380ff03d gvisor/runsc: use a veth link address instead of generating a new one
PiperOrigin-RevId: 248367340
Change-Id: Id792afcfff9c9d2cfd62cae21048316267b4a924
2019-05-15 11:11:58 -07:00
Andrei Vagin fff21b99e4 kokoro: run tests with a default docker container runtime
We want to know that our environment set up properly
and docker tests pass with a native runtime.

PiperOrigin-RevId: 248229294
Change-Id: I06c221e5eeed6e01bdd1aa935333c57e8eadc498
2019-05-14 16:01:50 -07:00
Andrei Vagin 9f2b12c624 gvisor/runsc/tests: set timeout for http.Get()
WaitForHTTP tries GET requests on a port until the call succeeds or timeout.

But we want to be sure that one of our attempts will not stuck for
the whole timeout.

All timeouts are increased to 30 seconds, because test cases with smaller
timeouts fail sometimes even for the native container runtime (runc).

PiperOrigin-RevId: 247888467
Change-Id: I03cfd3275286bc686a78fd26da43231d20667851
2019-05-13 00:50:44 -07:00
Andrei Vagin 9e1c253fe8 gvisor: run bazel in a docker container
bazel has a lot of dependencies and users don't want to install them
just to build gvisor.

These changes allows to run bazel in a docker container.
A bazel cache is on the local file system (~/.cache/bazel), so
incremental builds should be fast event after recreating a bazel
container.

Here is an example how to build runsc:
make BAZEL_OPTIONS="build runsc:runsc" bazel

Change-Id: I8c0a6d0c30e835892377fb6dd5f4af7a0052d12a
PiperOrigin-RevId: 246570877
2019-05-03 14:13:08 -07:00
Andrei Vagin 3f3e3a6303 gvisor/kokoro: save runsc logs
PiperOrigin-RevId: 246542315
Change-Id: Ia9ba2bc104e0af3277d3b6102122c13d320ea802
2019-05-03 11:21:22 -07:00
Michael Pratt 4d52a55201 Change copyright notice to "The gVisor Authors"
Based on the guidelines at
https://opensource.google.com/docs/releasing/authors/.

1. $ rg -l "Google LLC" | xargs sed -i 's/Google LLC.*/The gVisor Authors./'
2. Manual fixup of "Google Inc" references.
3. Add AUTHORS file. Authors may request to be added to this file.
4. Point netstack AUTHORS to gVisor AUTHORS. Drop CONTRIBUTORS.

Fixes #209

PiperOrigin-RevId: 245823212
Change-Id: I64530b24ad021a7d683137459cafc510f5ee1de9
2019-04-29 14:26:23 -07:00
Lantao Liu e815666717 Fix gvisor-containerd-shim download in the test.
The file layout in the bucket is changed a little bit recently to support both v1 shim and v2 shim.

PiperOrigin-RevId: 243682904
Change-Id: Ic1373c6dc088ef41f829e7ce3ea3762e1e2b0292
2019-04-15 13:56:52 -07:00
Nicolas Lacasse f44f2f73b0 Make it easier for humans to use RBE, and maintain our bazelrc.
This CL merges all RBE-specific configuration from .bazelrc_rbe into .bazelrc
so that it will be picked up by default by users running bazel.

It also checks in a bazelrc from the upstream bazel-toolchains repository, and
imports that into our repo-specific .bazelrc. This makes it easier to maintain
and update the bazelrc going forward.

Documentation was added to the README.

PiperOrigin-RevId: 242208733
Change-Id: Iea32de9be85b024bd74f88909b56b2a8ab34851a
2019-04-05 15:48:32 -07:00
Andrei Vagin 4a8062990f gvisor/kokoro: don't downgrade libseccomp
It isn't required with a new vm image.

PiperOrigin-RevId: 237915203
Change-Id: I813e7f1801d0766cec598413718e5ddd4c3d8cdf
2019-03-11 17:01:49 -07:00
Andrei Vagin 832589cb07 Fix tests which fail in kokoro
* open_create_test_runsc_ptrace_shared doesn't expect the write access to /
* exec_test_runsc_ptrace_shared could not find /usr/share/zoneinfo/
* clock_gettime_test_runsc_ptrace_shared didn't expect that
  a thread cpu time can be zero.
* affinity_test_runsc_ptrace_shared expected minimum 3 cpus

PiperOrigin-RevId: 237509429
Change-Id: I477937e5d2cdf3f8720836bfa972abd35d8220a3
2019-03-08 13:33:32 -08:00
Nicolas Lacasse 4bb1d5efb1 Remove TODO that we won't do.
There is no need to run the simple tests with RBE.

PiperOrigin-RevId: 236682573
Change-Id: I6c7fa25a33ef5e4d0b2cc2badc12ff9e1a70f015
2019-03-04 10:47:24 -08:00
Fabricio Voznika 3b44377eda Fix "-c dbg" build break
Remove allocation from vCPU.die() to save stack space.

Closes #131

PiperOrigin-RevId: 236238102
Change-Id: Iafca27a1a3a472d4cb11dcda9a2060e585139d11
2019-02-28 18:38:34 -08:00
Andrei Vagin cbd6b35c56 gvisor/kokoro: run apt-get update before apt-get install
We need to update the package lists, otherwise apt-get
install can request an old package which has been
removed from repositories.

PiperOrigin-RevId: 233879031
Change-Id: I2e1b3afd9d01008f774f10efd8852fd3f5e1c882
2019-02-13 19:38:44 -08:00
Andrei Vagin ecce96bab5 gvisor: Run syscall tests in kokoro on the rbe cluster
PiperOrigin-RevId: 233458853
Change-Id: I92c734b8075aa31e040fe7b4770bcf608e271e7a
2019-02-11 12:43:38 -08:00
Nicolas Lacasse f89085219d Pin containerd to v1.2.2. in integration tests.
We no longer need to depend on master, since v1.2.2 contains a commit to work
around "malformed input path" bug:
27c6449c2c

PiperOrigin-RevId: 229790405
Change-Id: Ia93b66959e6433bf096d6742be6064c1d7ed8b18
2019-01-17 11:51:22 -08:00
Nicolas Lacasse d2f8586a1f Kokoro: Build all packages as part of presubmit.
We now build all packages (including //test/...) with RBE as part of the Kokoro
presubmit.

The tests do not yet use RBE, because there are some failures. The Golang unit,
integration, and image tests still run locally.

The syscall test suite needs even more work to make it pass on RBE. Those will
be enabled in follow-up CLs. They currently are not enabled at all on Kokoro.

PiperOrigin-RevId: 226562208
Change-Id: Idd2b81b3e8f07bf300c77e68990493ba97d16e23
2018-12-21 17:13:34 -08:00
Nicolas Lacasse 7a3b4b7bdd Configure Kokoro build with RBE.
PiperOrigin-RevId: 226552586
Change-Id: I078502d46e2e4a8252ef0477309af8d3ffbbf88d
2018-12-21 15:42:59 -08:00
Nicolas Lacasse faabaad5a2 Turn off kokoro key-fetching until the key is available.
PiperOrigin-RevId: 225883292
Change-Id: I7e7f139d5865914c51866ea1d23220fcf221748c
2018-12-17 13:52:45 -08:00
Nicolas Lacasse f7e8dc57c5 Refactor kokoro/run_tests.sh
This will make it easier to add RBE to bazel.

PiperOrigin-RevId: 225865250
Change-Id: I530b5e09875267c18dc6e7e16590fe9e128253ac
2018-12-17 12:00:30 -08:00
Nicolas Lacasse 54694086df Use more descriptive service account key name.
PiperOrigin-RevId: 225854218
Change-Id: I7843e9e04c58a87d3a7e20d43e64c677ad9c961b
2018-12-17 10:59:28 -08:00
Nicolas Lacasse 13e4930948 Use containerd at HEAD until better tagged version is available.
Go 1.11.3 has a bug:
https://github.com/golang/go/issues/29241

This is fixed/workarounded in containerd:
52de371700

Until that commit has made it into a tagged version, we will use containerd at
head.

PiperOrigin-RevId: 225636987
Change-Id: I7e32beb7751f566f5b41682a29a14442c1aa56c2
2018-12-14 19:15:01 -08:00
Nicolas Lacasse ed930354ef Give Kokoro access to RBE service.
PiperOrigin-RevId: 225599728
Change-Id: I70cd9f9d7375e56ae8d0a531ad4efb41418e7402
2018-12-14 16:14:02 -08:00
Brian Geffon d3bc79bc84 Open source system call tests.
PiperOrigin-RevId: 224886231
Change-Id: I0fccb4d994601739d8b16b1d4e6b31f40297fb22
2018-12-10 14:42:34 -08:00
Kevin Krakauer 7b86d36a63 Fix crictl tests.
gvisor-containerd-shim moved. It now has a stable URL that run_tests.sh always
uses.

PiperOrigin-RevId: 223188822
Change-Id: I5687c78289404da27becd8d5949371e580fdb360
2018-11-28 10:10:13 -08:00
Michael Pratt 071aeea9d3 Disable crictl tests
gvisor-containerd-shim installation is currently broken.

PiperOrigin-RevId: 223002877
Change-Id: I2b890c5bf602a96c475c3805f24852ead8593a35
2018-11-27 09:25:20 -08:00
Kevin Krakauer 704b56a40d First crictl integration tests.
More tests will come, but it's worth getting what's done so far reviewed.

PiperOrigin-RevId: 219734531
Change-Id: If15ca6e6855e3d1cc28c83b5f9c3a72cb65b2e59
2018-11-01 18:29:07 -07:00
Kevin Krakauer 805a27c441 Install containerd and crictl when running tests in Kokoro.
PiperOrigin-RevId: 219166563
Change-Id: If4922eb5dd119b00f2c4bc7a5e016016ce9b9e45
2018-10-29 11:56:26 -07:00
Ian Gudger 8fce67af24 Use correct company name in copyright header
PiperOrigin-RevId: 217951017
Change-Id: Ie08bf6987f98467d07457bcf35b5f1ff6e43c035
2018-10-19 16:35:11 -07:00
Fabricio Voznika 3f46f2e501 Fix sandbox chroot
Sandbox was setting chroot, but was not chaging the working
dir. Added test to ensure this doesn't happen in the future.

PiperOrigin-RevId: 215676270
Change-Id: I14352d3de64a4dcb90e50948119dc8328c9c15e1
2018-10-03 20:44:20 -07:00
Fabricio Voznika cb23232c37 Fix build break in test
integration_test runs manually and breakage wasn't detected. Added test to
kokoro to ensure breakages are detected in the future.

PiperOrigin-RevId: 207772835
Change-Id: Iada81b579b558477d4db3516b38366ef6a2e933d
2018-08-07 13:48:35 -07:00