63 lines
3.6 KiB
Markdown
63 lines
3.6 KiB
Markdown
# gVisor Runtime Tests
|
|
|
|
App Engine uses gvisor to sandbox application containers. The runtime tests aim
|
|
to test `runsc` compatibility with these
|
|
[standard runtimes](https://cloud.google.com/appengine/docs/standard/runtimes).
|
|
The test itself runs the language-defined tests inside the sandboxed standard
|
|
runtime container.
|
|
|
|
Note: [Ruby runtime](https://cloud.google.com/appengine/docs/standard/ruby) is
|
|
currently in beta mode and so we do not run tests for it yet.
|
|
|
|
### Testing Locally
|
|
|
|
To run runtime tests individually from a given runtime, use the following table.
|
|
|
|
Language | Version | Download Image | Run Test(s)
|
|
-------- | ------- | ------------------------------------------- | -----------
|
|
Go | 1.12 | `make -C images load-runtimes_go1.12` | If the test name ends with `.go`, it is an on-disk test: <br> `docker run --runtime=runsc -it gvisor.dev/images/runtimes/go1.12 ( cd /usr/local/go/test ; go run run.go -v -- <TEST_NAME>... )` <br> Otherwise it is a tool test: <br> `docker run --runtime=runsc -it gvisor.dev/images/runtimes/go1.12 go tool dist test -v -no-rebuild ^TEST1$\|^TEST2$...`
|
|
Java | 11 | `make -C images load-runtimes_java11` | `docker run --runtime=runsc -it gvisor.dev/images/runtimes/java11 jtreg -agentvm -dir:/root/test/jdk -noreport -timeoutFactor:20 -verbose:summary <TEST_NAME>...`
|
|
NodeJS | 12.4.0 | `make -C images load-runtimes_nodejs12.4.0` | `docker run --runtime=runsc -it gvisor.dev/images/runtimes/nodejs12.4.0 python tools/test.py --timeout=180 <TEST_NAME>...`
|
|
Php | 7.3.6 | `make -C images load-runtimes_php7.3.6` | `docker run --runtime=runsc -it gvisor.dev/images/runtimes/php7.3.6 make test "TESTS=<TEST_NAME>..."`
|
|
Python | 3.7.3 | `make -C images load-runtimes_python3.7.3` | `docker run --runtime=runsc -it gvisor.dev/images/runtimes/python3.7.3 ./python -m test <TEST_NAME>...`
|
|
|
|
To run an entire runtime test locally, use the following table.
|
|
|
|
Note: java runtime test take 1+ hours with 16 cores.
|
|
|
|
Language | Version | Running the test suite
|
|
-------- | ------- | ----------------------------------------
|
|
Go | 1.12 | `make go1.12-runtime-tests{_vfs2}`
|
|
Java | 11 | `make java11-runtime-tests{_vfs2}`
|
|
NodeJS | 12.4.0 | `make nodejs12.4.0-runtime-tests{_vfs2}`
|
|
Php | 7.3.6 | `make php7.3.6-runtime-tests{_vfs2}`
|
|
Python | 3.7.3 | `make python3.7.3-runtime-tests{_vfs2}`
|
|
|
|
#### Clean Up
|
|
|
|
Sometimes when runtime tests fail or when the testing container itself crashes
|
|
unexpectedly, the containers are not removed or sometimes do not even exit. This
|
|
can cause some docker commands like `docker system prune` to hang forever.
|
|
|
|
Here are some helpful commands (should be executed in order):
|
|
|
|
```bash
|
|
docker ps -a # Lists all docker processes; useful when investigating hanging containers.
|
|
docker kill $(docker ps -a -q) # Kills all running containers.
|
|
docker rm $(docker ps -a -q) # Removes all exited containers.
|
|
docker system prune # Remove unused data.
|
|
```
|
|
|
|
### Testing Infrastructure
|
|
|
|
There are 3 components to this tests infrastructure:
|
|
|
|
- [`runner`](runner) - This is the test entrypoint. This is the binary is
|
|
invoked by `bazel test`. The runner spawns the target runtime container
|
|
using `runsc` and then copies over the `proctor` binary into the container.
|
|
- [`proctor`](proctor) - This binary acts as our agent inside the container
|
|
which communicates with the runner and actually executes tests.
|
|
- [`exclude`](exclude) - Holds a CSV file for each language runtime containing
|
|
the full path of tests that should be excluded from running along with a
|
|
reason for exclusion.
|