Commit Graph

52 Commits

Author SHA1 Message Date
Jamie Liu 1ad3822200 Add go:build directives as required by Go 1.17's gofmt.
PiperOrigin-RevId: 385894869
2021-07-20 16:28:45 -07:00
Fabricio Voznika 394c6089c3 Fix test_app task-tree
Executing `select {}` to wait forever triggers Go runtime deadlock
detection and kills the child, causing the number actual processes
be less than expected.

PiperOrigin-RevId: 376298799
2021-05-27 19:55:03 -07:00
Ayush Ranjan d924515b09 [perf] Fix profiling in benchmarking jobs.
Due to https://github.com/moby/moby/issues/42345, the docker daemon is passing
the incorrect `--root` flag to runsc. So our profiler is not able to find the
container stat files where it expects them to be.

PiperOrigin-RevId: 372067954
2021-05-05 00:02:37 -07:00
Adin Scannell 8192cccda6 Clean test tags.
PiperOrigin-RevId: 369505182
2021-04-20 13:11:25 -07:00
Fabricio Voznika 3007ae647d Fail tests when container returns non-zero status
PiperOrigin-RevId: 366839955
2021-04-05 11:39:53 -07:00
Zeling Feng 9e86dfc9c5 Fix logs for packetimpact tests cleanup
- Don't cleanup containers in Network.Cleanup, otherwise containers will
  be killed and removed several times.
- Don't set AutoRemove for containers. This will prevent the confusing
  'removal already in progress' messages.

Fixes #3795

PiperOrigin-RevId: 364404414
2021-03-22 14:10:00 -07:00
Ayush Ranjan a9441aea27 [op] Replace syscall package usage with golang.org/x/sys/unix in pkg/.
The syscall package has been deprecated in favor of golang.org/x/sys.

Note that syscall is still used in the following places:
- pkg/sentry/socket/hostinet/stack.go: some netlink related functionalities
  are not yet available in golang.org/x/sys.
- syscall.Stat_t is still used in some places because os.FileInfo.Sys() still
  returns it and not unix.Stat_t.

Updates #214

PiperOrigin-RevId: 360701387
2021-03-03 10:25:58 -08:00
Zach Koopmans ba51999fa6 Fix bug with iperf and don't profile runc.
Fix issue with iperf where b.N wasn't changing across runs.
Also, if the given runtime is runc/not given, don't run a profile against it.

PiperOrigin-RevId: 357231450
2021-02-12 11:28:16 -08:00
Dean Deng 1efe0ebc59 Switch uses of os.Getenv that check for empty string to os.LookupEnv.
Whether the variable was found is already returned by syscall.Getenv.
os.Getenv drops this value while os.Lookupenv passes it along.

PiperOrigin-RevId: 351674032
2021-01-13 15:15:20 -08:00
Etienne Perot f34aaf7ef1 testutil: Create a `multiLogger` that logs to multiple `Loggers`.
This is useful when using the shell library in order to log to both the test
log and the standard logs.

PiperOrigin-RevId: 351673465
2021-01-13 15:09:50 -08:00
Fabricio Voznika 8b0f0b4d11 Delete shim v1
gvisor-containerd-shim is not compatible with containerd 1.1 or earlier.
Starting from containerd 1.2, shim v2 is the preferred interface.

PiperOrigin-RevId: 351485556
2021-01-12 17:54:10 -08:00
Etienne Perot 11787a601e Create console test library.
This creates a TTY pair and runs `/bin/sh` in interactive mode within it.
It provides useful helper functions to interact with the shell and read the
output of commands run within it.

This is meant to be used for testing upcoming changes allowing `runsc exec` to
work in `-detach=false -tty=true` mode.

PiperOrigin-RevId: 350841006
2021-01-08 14:51:50 -08:00
Adin Scannell b06e5bc5b0 Add benchmarks targets to BuildKite.
This includes minor fix-ups:

* Handle SIGTERM in runsc debug, to exit gracefully.
* Fix cmd.debug.go opening all profiles as RDONLY.
* Fix the test name in fio_test.go, and encode the block size in the test.

PiperOrigin-RevId: 350205718
2021-01-05 13:21:54 -08:00
Adin Scannell ffa9a715aa Simplify profiling and benchmarks.
- Tweak the benchmarks to work with b.N where appropriate. In many cases,
  b.N was simply being ignored. This creates an implicit dependency in the
  user passing a reasonable benchtime (less than or equal to the actual
  runtime of the test, or using the X syntax) otherwise the test runs
  forever.
- In cases where the above is impossible, explicitly set benchtime from
  the test wrapper, to prevent the above behavior (tensorflow).
- Drop the *Reverse variants, which are simply hey benchmarks. We should
  just add a hey benchmark. The platforms benchmarks already include a
  native platform, and thus these benchmarks are incredibly confusing.
  (In other words, BenchmarkNginxReverse has nothing to do with an nginx
  benchmark for runsc.)
- Remove the redunant Harness object, which contains no state, in order
  to slightly simplify the code.
- Make Block and Heap profiling actually work, but setting appropriate
  runtime parameters (and plumbing them through the config).
- Split the profiling into two phases: start and stop, since some will
  need to be started early, and others will need to happen at the end.

PiperOrigin-RevId: 349495377
2020-12-29 18:29:12 -08:00
Peter Johnston eeb23531eb Support icmpv6 transport protocol
PiperOrigin-RevId: 346101076
2020-12-07 08:44:44 -08:00
Adin Scannell 80552b936d Support partitions for other tests.
PiperOrigin-RevId: 345399936
2020-12-03 01:00:21 -08:00
Zeling Feng 8b692f5931 Make testutil.RandomID safe for concurrent uses
testutil.RandomID was using Rand.Read which is not safe for concurrent use.
It caused name conflicts in packetimpact tests when they are run in parallel.
Adding a mutex to protect the Rand.Read operation.

PiperOrigin-RevId: 345360062
2020-12-02 19:14:51 -08:00
Zeling Feng 59a2c785bf Do not start a ContainerExec twice
ContainerExecStart and ContainerExecAttach both call the /exec/id/start API
endpoint.

PiperOrigin-RevId: 344946627
2020-11-30 21:22:55 -08:00
Adin Scannell 3a60bc47a0 Ensure containerd is used from installed location.
Currently, if containerd is installed locally via tools/installers/containerd,
then it will not necessarily be used if containerd is installed in the system
path. This means that the existing containerd tests are all likely broken.

Also, use libbtrfs-dev instead of btrfs-tools, which is not actually required.

PiperOrigin-RevId: 344879109
2020-11-30 13:40:23 -08:00
Adin Scannell 2320ce5b7d Fail gracefully if Docker is not configured with ipv6.
PiperOrigin-RevId: 343927315
2020-11-23 14:24:27 -08:00
Fabricio Voznika e2d9a68eef Add support for TTY in multi-container
Fixes #2714

PiperOrigin-RevId: 342950412
2020-11-17 14:51:24 -08:00
Fabricio Voznika 9e9fec3a09 Enable more VFS2 tests
Updates #1487

PiperOrigin-RevId: 335516732
2020-10-05 15:54:36 -07:00
Ayush Ranjan 23fcbd8722 [testing] Use container address to talk to server running inside container.
Docker does not have IPv6 port forwarding as tracked by the following issue:
https://github.com/moby/moby/issues/11518

So when running bazel itself inside a docker container, we can not use the host
port bindings to communicate with sockets inside the container. This was causing
integration tests and image tests to fail when run through our Makefile targets.

PiperOrigin-RevId: 332355051
2020-09-17 17:58:08 -07:00
Zeling Feng 86c1ae095a Add support to run packetimpact tests against Fuchsia
blaze test <test_name>_fuchsia_test will run the corresponding packetimpact
test against fuchsia.

PiperOrigin-RevId: 329835290
2020-09-02 19:19:40 -07:00
Fabricio Voznika 32e7a54f7f Make flag propagation automatic
Use reflection and tags to provide automatic conversion from
Config to flags. This makes adding new flags less error-prone,
skips flags using default values (easier to read), and makes
tests correctly use default flag values for test Configs.

Updates #3494

PiperOrigin-RevId: 328662070
2020-08-26 20:24:41 -07:00
Zach Koopmans 2b0b5e2521 Remove go profiling flag from dockerutil.
Go profiling was removed from runsc debug in a previous change.

PiperOrigin-RevId: 328203826
2020-08-24 13:53:10 -07:00
Bhasker Hariharan 710adf23cd Use a explicit random src for RandomID.
PiperOrigin-RevId: 327659759
2020-08-20 11:07:29 -07:00
Fabricio Voznika be76c7ce6e Move boot.Config to its own package
Updates #3494

PiperOrigin-RevId: 327548511
2020-08-19 18:37:42 -07:00
gVisor bot 3be8b49c70 Merge pull request #3375 from kevinGC:ipt-test-early-return
PiperOrigin-RevId: 326693922
2020-08-14 11:24:56 -07:00
Zach Koopmans 89f3197fc3 Mark integration tests as passing in VFS2 except CheckpointRestore.
Mark all tests passing for VFS2 in:
image_test
integration_test

There's no way to do negative look ahead/behind in golang test regex,
so check if the tests uses VFS2 and skip CheckPointRestore if it does.

PiperOrigin-RevId: 326050915
2020-08-11 10:37:32 -07:00
Kevin Krakauer 805a96d7ba Speed up iptables tests
//test/iptables:iptables_test runs 30 seconds faster on my machine.

* Using contexts instead of many smaller timeouts makes the tests less
  likely to flake and removes unnecessary complexity.
* We also use context to properly shut down concurrent goroutines and
  the test container.
* Container logs are always logged.
2020-08-10 17:50:01 -07:00
Zach Koopmans 7b9bfc0ce0 Port Ruby benchmark.
PiperOrigin-RevId: 325500772
2020-08-07 13:49:07 -07:00
Fabricio Voznika 190b1e6bd4 Stop profiling when the sentry exits
Also removes `--profile-goroutine` because it's equivalent
to `debug --stacks`.

PiperOrigin-RevId: 325061502
2020-08-05 11:30:11 -07:00
Kevin Krakauer bc8201d01b Have dockerutil.Wait* respect the context deadline
PiperOrigin-RevId: 324044634
2020-07-30 11:29:24 -07:00
Kevin Krakauer d9c9420335 ip6tables testing
We skip gVisor tests for now, as ip6tables aren't yet implemented.
2020-07-28 10:51:14 -07:00
Zach Koopmans 2ecf66903e Add profiling to dockerutil
Adds profiling with `runsc debug` or pprof to dockerutil. All
targets using dockerutil should now be able to use profiling.

In addition, modifies existing benchmarks to use profiling.

PiperOrigin-RevId: 323298634
2020-07-26 22:02:51 -07:00
Kevin Krakauer fb8be7e627 make connect(2) fail when dest is unreachable
Previously, ICMP destination unreachable datagrams were ignored by TCP
endpoints. This caused connect to hang when an intermediate router
couldn't find a route to the host.

This manifested as a Kokoro error when Docker IPv6 was enabled. The Ruby
image test would try to install the sinatra gem and hang indefinitely
attempting to use an IPv6 address.

Fixes #3079.
2020-07-22 16:51:42 -07:00
Adin Scannell e1a04f84e8 Add standard entrypoints for test targets.
PiperOrigin-RevId: 322265513
2020-07-20 18:05:05 -07:00
gVisor bot c81ac8ec3b Merge pull request #2672 from amscanne:shim-integrated
PiperOrigin-RevId: 321053634
2020-07-13 16:10:58 -07:00
Zach Koopmans b8d3d09bd1 Initial golang Benchmarks
PiperOrigin-RevId: 321021071
2020-07-13 13:29:30 -07:00
Adin Scannell 2afff44403 Update shim to build using bazel.
The go.mod dependency tree for the shim was somehow contradictory. After
resolving these issues (e.g. explicitly imported k8s 1.14, pulling a
specific dbus version), and adding all dependencies, the shim can now be
build as part of the regular bazel tree.

As part of this process, minor cleanup was done in all the source files:
headers were standardized (and include "The gVisor Authors" in addition
to the "The containerd Authors" if originally derived from containerd
sources), and comments were cleaned up to meet coding standards.

This change makes the containerd installation dynamic, so that multiple
versions can be tested, and drops the static installer for the VM image
itself.

This change also updates test/root/crictl_test.go and related utilities,
so that the containerd tests can be run on any version (and in cases
where it applies, they can be run on both v1 and v2 as parameterized
tests).
2020-07-09 17:39:47 -07:00
Zach Koopmans f3fa43cf23 Move all tests to new docker API.
Moves following to new dockerutil API:
- //test/e2e:integration_test
- //test/image:image_test
- //test/iptables:iptables_test
- //test/root:root_test
- //test/packetimpact:packetimpact_test

PiperOrigin-RevId: 320253118
2020-07-08 13:26:23 -07:00
Ayush Ranjan e3db9bda60 Enable shards in runtime test runner.
Fixed an issue with the runtime test runner which enables us to run tests in
shards. We had to touch the status file as indicated by an env var.

PiperOrigin-RevId: 320236205
2020-07-08 12:05:04 -07:00
Fabricio Voznika 97f6b20e89 Move mount configutation to RunOpts
Separate mount configuration from links and move it to
RunOpts, like the other options.

PiperOrigin-RevId: 317010158
2020-06-17 18:43:26 -07:00
Ian Lewis 8ea99d58ff Set the HOME environment variable for sub-containers.
Fixes #701

PiperOrigin-RevId: 316025635
2020-06-11 19:31:24 -07:00
gVisor bot 92bafd7929 Automated rollback of changelist 311424257
PiperOrigin-RevId: 313300554
2020-05-26 17:40:57 -07:00
Fabricio Voznika cdf48e8516 Fix TestTmpFile
Split check for file in /tmp from working directory test.
Fix readonly case which should not fail to create working
dir.

PiperOrigin-RevId: 312702930
2020-05-21 11:08:10 -07:00
Bhasker Hariharan 8605c97136 Automated rollback of changelist 311285868
PiperOrigin-RevId: 311424257
2020-05-13 16:13:37 -07:00
Ian Gudger e4058c0355 Replace test_runner.sh bash script with Go.
PiperOrigin-RevId: 311285868
2020-05-13 01:22:42 -07:00
Eyal Soha c9199bab92 More descriptive error message for missing docker image.
Tested:
  Ran a packetimpact test after `docker image rm` and examined the message.
PiperOrigin-RevId: 308370603
2020-04-24 20:00:24 -07:00