Commit Graph

89 Commits

Author SHA1 Message Date
Zeling Feng 164a2fe386 Remove unused docker image
- Remove packetimpact docker image
- Update Makefile to remove unnecessary steps
- Remove unused runner flags

PiperOrigin-RevId: 416621865
2021-12-15 12:34:24 -08:00
Zach Koopmans 74536aba2c [benchmarks] Don't run vfs1 benchmarks anymore.
PiperOrigin-RevId: 413860206
2021-12-02 23:58:36 -08:00
Daniel Dao 51f39a1204
cgroupv2: skip tests on containerd < 1.4
containerd < 1.4 does not support cgroupv2, so we adjust the Make
targets and installer scripts to skip test run on those versions.

Signed-off-by: Daniel Dao <dqminh89@gmail.com>
2021-11-25 11:55:19 +00:00
Adin Scannell f1a46c928f Support STAGED_BINARIES to run prebuilt binaries with the test pipeline.
In some cases, it may be desirable to prebuild binaries and run all tests,
for example to run benchmarks with various experiments. Allow the top-level
Makefile to support this by checking for a STAGED_BINARIES variable.

PiperOrigin-RevId: 410673120
2021-11-17 17:49:35 -08:00
Fabricio Voznika 9262ea47a5 Add support for containerd 1.5
"cri.runtimeoptions.v1" moved to "runtimeoptions.v1" and containerd
configuration format version 2 is required.

Updates #6449

PiperOrigin-RevId: 405474653
2021-10-25 13:17:25 -07:00
Adin Scannell 8dc3be7a61 Use --output=starlark trick for all build path parsing.
There is no reason to limit this to debian paths.

PiperOrigin-RevId: 392469415
2021-08-23 10:53:38 -07:00
Ayush Ranjan c2353e4055 [op] Fix //debian:debian.
Co-authored-by: Andrei Vagin <avagin@google.com>
PiperOrigin-RevId: 390232925
2021-08-11 15:28:51 -07:00
Zach Koopmans c3326c440a [benchmarks] Fix redis benchmark.
Intermittenly, the connection between the client redis_benchmark container can be
flaky with the server, even if the server has been up for a long time. If this
happens, just re-run a client until we get a result.

Also, don't start a new server for each operation.

Also also, modify Makefile run-benchmark call to accept RUNTIME=runc correctly.

PiperOrigin-RevId: 378918886
2021-06-11 11:56:15 -07:00
Ayush Ranjan 915b813798 [op] Remove Build job from GitHub actions.
We already have blocking nogo tests which show all findings. This job was
building all nogo targets, and posting all the findings to GitHub as a check
run. Building nogo takes a while so we actually end up wasting a lot of time
doing redundant work.

This is aligned with our goal of moving away from GitHub actions to BuildKite
only.

PiperOrigin-RevId: 370134875
2021-04-23 12:33:48 -07:00
Adin Scannell 8192cccda6 Clean test tags.
PiperOrigin-RevId: 369505182
2021-04-20 13:11:25 -07:00
Ayush Ranjan 064a849f36 [op] Split nogo target out of unit tests.
Building nogo targets takes a very long time. This change extracts it into its
own BuildKite job.

This change also additionally speeds up other targets that were using the bazel
flag --test_tag_filters. Without --build_tag_filters, the filter is not
applied while building the specified targets and so we might end up building
targets that are not actually tested.

PiperOrigin-RevId: 368918211
2021-04-16 14:20:46 -07:00
Rahat Mahmood 932c8abd0f Implement cgroupfs.
A skeleton implementation of cgroupfs. It supports trivial cpu and
memory controllers with no support for hierarchies.

PiperOrigin-RevId: 366561126
2021-04-02 21:10:44 -07:00
Zach Koopmans 1b53550e55 Add vfs1 to go/runsc-benchmarks
PiperOrigin-RevId: 366470480
2021-04-02 10:41:23 -07:00
Ayush Ranjan ee6b22ca63 [perf] Run benchmarks with VFS2.
The run-benchmark target would run the benchmark with VFS1.

PiperOrigin-RevId: 362754188
2021-03-13 20:29:56 -08:00
Rahat Mahmood 192318a231 fusefs: Implement default_permissions and allow_other mount options.
By default, fusefs defers node permission checks to the server. The
default_permissions mount option enables the usual unix permission
checks based on the node owner and mode bits. Previously fusefs was
incorrectly checking permissions unconditionally.

Additionally, fusefs should restrict filesystem access to processes
started by the mount owner to prevent the fuse daemon from gaining
priviledge over other processes. The allow_other mount option
overrides this behaviour. Previously fusefs was incorrectly skipping
this check.

Updates #3229

PiperOrigin-RevId: 362419092
2021-03-11 17:59:13 -08:00
Adin Scannell 017348af99 Drop gazelle target from the Makefile.
This is replaced with a straight call to bazel. Unfortunately,
the built gazelle target requires a bazel installation to run
anyways.

PiperOrigin-RevId: 355211990
2021-02-02 11:29:02 -08:00
Ayush Ranjan cbcebfea80 [infra] Fix gazelle target.
PiperOrigin-RevId: 354991724
2021-02-01 11:53:30 -08:00
Fabricio Voznika c7fc4a5d66 Add fsstress tests
Updates #5273

PiperOrigin-RevId: 352088736
2021-01-15 14:51:51 -08:00
Zach Koopmans 95371cff35 Don't run profiles on runc.
PiperOrigin-RevId: 351906812
2021-01-14 17:04:13 -08:00
Adin Scannell a684bfb6c0 Split container tests from unit tests.
PiperOrigin-RevId: 351632484
2021-01-13 11:48:50 -08:00
Adin Scannell fb95e13df5 Don't remove release directory.
If the release directory is a parent directory (for reasons), then this causes
a few problems (to say the least).

PiperOrigin-RevId: 351515882
2021-01-12 22:25:46 -08:00
Fabricio Voznika 8b0f0b4d11 Delete shim v1
gvisor-containerd-shim is not compatible with containerd 1.1 or earlier.
Starting from containerd 1.2, shim v2 is the preferred interface.

PiperOrigin-RevId: 351485556
2021-01-12 17:54:10 -08:00
Adin Scannell a20da70829 Fix Go branch for arm64.
This requires several changes:
* Templates must preserve relevant tags.
* Pagetables templates are split into two targets, each preserving tags.
* The binary VDSO is similarly split into two targets, with some juggling.
* The top level tools/go_branch.sh now does a crossbuild of ARM64 as well,
  and checks and merges the results of the two branches together.

Fixes #5178

PiperOrigin-RevId: 351304330
2021-01-11 22:33:36 -08:00
Adin Scannell 0c99ab7090 Support releasing aarch64 builds.
This change works around an issue in rules_pkg, described here:
  https://github.com/bazelbuild/rules_pkg/pull/263

PiperOrigin-RevId: 350869030
2021-01-08 18:03:17 -08:00
Adin Scannell df1b23c8de Fix sha256 for github-workflow.json.
This was not being tested as part of the unit test workflows, and thus was
not being hit normally. These tests are also added to the unit tests target.

PiperOrigin-RevId: 350766814
2021-01-08 08:17:41 -08:00
Andrei Vagin 8de562b799 Add ARM smoke test
make BAZEL_CONFIG=aarch64 arm-qemu-smoke-test

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2021-01-07 17:41:43 -08:00
Adin Scannell 776016ac64 Fix native benchmarks.
PiperOrigin-RevId: 350509137
2021-01-07 01:17:10 -08:00
Adin Scannell fa8682da0f Export a pprof visualization endpoint.
This allows us to link directly to profiling results from
the build results. The code uses the standard pprof http
server, exported from the Cloud Run instance.

PiperOrigin-RevId: 350440910
2021-01-06 15:44:07 -08:00
Adin Scannell b06e5bc5b0 Add benchmarks targets to BuildKite.
This includes minor fix-ups:

* Handle SIGTERM in runsc debug, to exit gracefully.
* Fix cmd.debug.go opening all profiles as RDONLY.
* Fix the test name in fio_test.go, and encode the block size in the test.

PiperOrigin-RevId: 350205718
2021-01-05 13:21:54 -08:00
Adin Scannell 899b9ba46a Add BuildKite annotations for failures and profiles.
This change cleans up some minor Makefile issues, and adds support for
BuildKite annotations on failure and on profiles being generated. These
annotations will make failures very clear and link to the artifacts.

This change is a stepping stone for aggregating coverage data from all
individual test jobs, as this will also happen in .buildkite/annotate.sh.

PiperOrigin-RevId: 349606598
2020-12-30 15:09:12 -08:00
Fabricio Voznika eeee055d60 Set --nocache_test_results to runtime tests
If not set, the cached result is used even when runtime options
are changed, because they are not visible to blaze/bazel.

PiperOrigin-RevId: 348074339
2020-12-17 12:42:09 -08:00
Adin Scannell 7da25e6dc3 Restore refresh target.
PiperOrigin-RevId: 347864621
2020-12-16 11:45:28 -08:00
Adin Scannell 5bdc167d17 Fix run and sudo targets.
These are not passing arguments properly. This breaks the current
pre-command for BuildKite.

PiperOrigin-RevId: 347062729
2020-12-11 13:25:44 -08:00
Ian Lewis 0eb4acad37 Fix website build
Fix 'run' function call so that parameters are passed properly to the function.

PiperOrigin-RevId: 346929952
2020-12-10 21:50:03 -08:00
Adin Scannell a855a814d6 Refactor the Makefile to avoid recursive Make.
Recursive make is difficult to follow and debug. Drop this by using
internal functions, which, while difficult, are easier than trying to
following recursive invokations.

Further simplify the Makefile by collapsing the image bits and removing
the tools/vm directory, which is effectively unused.

Fixes #4952

PiperOrigin-RevId: 346569133
2020-12-09 15:53:23 -08:00
Peter Johnston eeb23531eb Support icmpv6 transport protocol
PiperOrigin-RevId: 346101076
2020-12-07 08:44:44 -08:00
Adin Scannell 216abfb6df Update containerd tests for 1.4+ to 1.4.3.
PiperOrigin-RevId: 345764404
2020-12-04 15:05:46 -08:00
Zach Koopmans 4b198b18e1 Fix load-benchmarks-images call.
make load-benchmarks-images is empty with the deleted line.

PiperOrigin-RevId: 345741855
2020-12-04 13:14:09 -08:00
Fabricio Voznika 9eb77281c4 Update containerd to 1.3.9
PiperOrigin-RevId: 345564927
2020-12-03 16:55:44 -08:00
Adin Scannell 80552b936d Support partitions for other tests.
PiperOrigin-RevId: 345399936
2020-12-03 01:00:21 -08:00
Adin Scannell 756bc3e52b Clean up build output.
This change also simplifies and documents the build_cmd pipeline, and
reduces general noise for debugging Makefile issues.

It also drops the mapping for /etc/docker/daemon.json, which if it
does not exist initially will create this as a directory (causing lots
of confusion and breaks).

PiperOrigin-RevId: 343932456
2020-11-23 14:45:34 -08:00
Ayush Ranjan 2a1974b076 [infra] Do not use debug flags for testing.
This causes some networking related tests to take a very long time.
Upon failure, tests can be manually run with debug flags to debug. As is the
strace logs are not available from a test run.

PiperOrigin-RevId: 342156294
2020-11-12 16:30:05 -08:00
Zach Koopmans 792cbc06de Add debug logs to startup benchmark.
PiperOrigin-RevId: 341757694
2020-11-10 21:20:52 -08:00
Adin Scannell a0e2966df5 Show run-benchmark output.
PiperOrigin-RevId: 341667792
2020-11-10 12:01:29 -08:00
Adin Scannell b2d5b71ecd Evaluate T as a shell variable, not a Make variable.
PiperOrigin-RevId: 341531230
2020-11-09 19:13:08 -08:00
Zach Koopmans 6c93dc970f Add pull image statment to benchmarks rule.
PiperOrigin-RevId: 341132662
2020-11-06 15:46:51 -08:00
Ayush Ranjan f27edcc708 [runtime tests] Add partitions to runtime tests.
This will allow us to run massive runtime tests live java to run in parallel
across multiple jobs.

PiperOrigin-RevId: 340956246
2020-11-05 17:11:32 -08:00
Zach Koopmans 7caefd68df Internal Change
PiperOrigin-RevId: 340941898
2020-11-05 15:43:14 -08:00
Kevin Krakauer 181fea0b58 Make RedirectTarget thread safe
Fixes #4613.

PiperOrigin-RevId: 339746784
2020-10-29 14:28:56 -07:00
Ian Lewis cc772f3d54 Add a platform portability blog post
Also fixes the docker_image bazel rule, and website-server make target.

Fixes #3273

PiperOrigin-RevId: 338606668
2020-10-22 21:22:54 -07:00