Commit Graph

60 Commits

Author SHA1 Message Date
Nicolas Lacasse 337b36e570 Add escapes to newlines in syzkaller instructions.
So they can be copy-pasted.

PiperOrigin-RevId: 362605833
2021-03-12 14:52:50 -08:00
Ayush Ranjan dea894238b [infra] Update JDK11 version for java runtime tests.
PiperOrigin-RevId: 358085809
2021-02-17 19:28:43 -08:00
Andrei Vagin 63c9dd3656 images: Rework syzkaller documentation.
PiperOrigin-RevId: 355660221
2021-02-04 10:41:15 -08:00
Ayush Ranjan fa2d3698c4 [infra] Do not recompile integration test executables each time.
Instead build the executable into the image.

PiperOrigin-RevId: 355631672
2021-02-04 08:28:38 -08:00
Ayush Ranjan f2c881f684 [vfs] Make sticky bit check consistent with Linux.
Our implementation of vfs.CheckDeleteSticky was not consistent with Linux,
specifically not consistent with fs/linux.h:check_sticky().

One of the biggest differences was that the vfs implementation did not
allow the owner of the sticky directory to delete files inside it that belonged
to other users.

This change makes our implementation consistent with Linux.

Also adds an integration test to check for this. This bug is also present in
VFS1.

Updates #3027

PiperOrigin-RevId: 355557425
2021-02-03 22:44:51 -08:00
Ayush Ranjan bfad3be0b7 Make ubuntutest image x86 only.
PiperOrigin-RevId: 355315140
2021-02-02 21:06:39 -08:00
Tamir Duberstein fb422db706 Update go-tools to 2020.2.1
...and a bunch of other things as I worked through the rot. Notably:
  - Upgrade to bazel 4.0.0
  - Upgrade to Go 1.15.7

Remove go_branch stderr suppression; this made it quite difficult to see
what was failing while developing this patch.

PiperOrigin-RevId: 355257833
2021-02-02 14:53:12 -08:00
gVisor bot 5230c8e13c Merge pull request #5403 from avagin:syzkaller3
PiperOrigin-RevId: 355047815
2021-02-01 16:08:49 -08:00
Ayush Ranjan 0da3c72c9d [infra] Consolidate all ubuntu tests into one image.
This makes it easier to add more tests that run on Ubuntu. We can now just
add a bash script and call that from integration_test without having to set up
another image.

PiperOrigin-RevId: 355000410
2021-02-01 12:30:45 -08:00
Andrei Vagin 44e0a0eac9 Add syzkaller docker image 2021-01-29 19:00:57 -08:00
Adin Scannell 48dfb8db9e Add image presubmit tests and mark fsstress x86_64 only.
PiperOrigin-RevId: 353118942
2021-01-21 15:53:34 -08:00
Ayush Ranjan 1a072e298e [infra] Attempt to fix Kokoro image workflow.
#5274 broke `make load-basic_fsstress` target. It works locally but for some
inconceivable reason it fails to build on Kokoro.

Error message:
```
Step 1/12 : FROM alpine
 ---> 1fca6fe4a1ec
Step 2/12 : RUN apk add git
 ---> Running in 109a5326259a
standard_init_linux.go:207: exec user process caused "no such file or directory"
The command '/bin/sh -c apk add git' returned a non-zero code: 1
make: *** [load-basic_fsstress] Error 1
tools/images.mk:149: recipe for target 'load-basic_fsstress' failed
```

Copied the approach from the following issue to achieve the same goal:
https://github.com/nodejs/docker-node/issues/586#issue-273737852.

PiperOrigin-RevId: 352602945
2021-01-19 10:46:59 -08:00
Fabricio Voznika c7fc4a5d66 Add fsstress tests
Updates #5273

PiperOrigin-RevId: 352088736
2021-01-15 14:51:51 -08:00
Andrei Vagin 52de442ce1 Build the arm-qemu docker image only for x86_64
It isn't used on aarch64.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2021-01-08 11:39:18 -08:00
Andrei Vagin 8de562b799 Add ARM smoke test
make BAZEL_CONFIG=aarch64 arm-qemu-smoke-test

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2021-01-07 17:41:43 -08:00
Adin Scannell 65a2242db4 Tweak aarch64 support.
A few images were broken with respect to aarch64. We should now
be able to run push-all-images with ARCH=aarch64 as part of the
regular continuous integration builds, and add aarch64 smoke tests
(via user emulation for now) to the regular test suite (future).

PiperOrigin-RevId: 346685462
2020-12-09 18:51:17 -08:00
Adin Scannell a855a814d6 Refactor the Makefile to avoid recursive Make.
Recursive make is difficult to follow and debug. Drop this by using
internal functions, which, while difficult, are easier than trying to
following recursive invokations.

Further simplify the Makefile by collapsing the image bits and removing
the tools/vm directory, which is effectively unused.

Fixes #4952

PiperOrigin-RevId: 346569133
2020-12-09 15:53:23 -08:00
Peter Johnston eeb23531eb Support icmpv6 transport protocol
PiperOrigin-RevId: 346101076
2020-12-07 08:44:44 -08:00
Adin Scannell dbd4a6e3e5 Add BuildKite agent.
This has no effect on the continuous integration system, and simply
publishes a cached container image containing the agent and metrics
agent with known provenance.

PiperOrigin-RevId: 345274375
2020-12-02 11:28:26 -08:00
Adin Scannell 756bc3e52b Clean up build output.
This change also simplifies and documents the build_cmd pipeline, and
reduces general noise for debugging Makefile issues.

It also drops the mapping for /etc/docker/daemon.json, which if it
does not exist initially will create this as a directory (causing lots
of confusion and breaks).

PiperOrigin-RevId: 343932456
2020-11-23 14:45:34 -08:00
Kevin Krakauer d66aebb15a Add the gVisor admission webhook
PiperOrigin-RevId: 339913577
2020-10-30 12:02:09 -07:00
Ian Lewis cc772f3d54 Add a platform portability blog post
Also fixes the docker_image bazel rule, and website-server make target.

Fixes #3273

PiperOrigin-RevId: 338606668
2020-10-22 21:22:54 -07:00
Adin Scannell 0a7e32bd17 Add easier-to-use docker_image target.
PiperOrigin-RevId: 337415009
2020-10-15 17:21:24 -07:00
Adin Scannell 06200cb5cf Improve multi-arch support.
This change allows Dockerfiles named Dockerfile.$(ARCH) and makes list-images
list only supported architectures.

Updates #2847

PiperOrigin-RevId: 336108293
2020-10-08 10:05:40 -07:00
Zach Koopmans c3fc69022a Fix Nginx Startup and Size Benchmarks.
Changes in Nginx Benchmarks in network_tests also affect Startup/Size
Nginx Benchmarks. Make sure the commands line up.

PiperOrigin-RevId: 333543697
2020-09-24 10:32:01 -07:00
Fabricio Voznika 93fd164fa2 Add "Containing a Real Vulnerability" blog post
PiperOrigin-RevId: 332477119
2020-09-18 10:28:07 -07:00
Tamir Duberstein 964447c8ce Clean up image construction
- Skip `docker inspect`; `docker pull` is idempotent
- Remove unnecessary CMD directives in Dockerfiles
- Run bazel before building images to catch errors sooner

PiperOrigin-RevId: 331107815
2020-09-11 01:57:42 -07:00
Ian Lewis 26439f9a43 Add syntax highlighting to website
Adds a syntax highlighting theme css so that code snippets are highlighted
properly.

PiperOrigin-RevId: 330733737
2020-09-09 09:08:37 -07:00
Zeling Feng 86c1ae095a Add support to run packetimpact tests against Fuchsia
blaze test <test_name>_fuchsia_test will run the corresponding packetimpact
test against fuchsia.

PiperOrigin-RevId: 329835290
2020-09-02 19:19:40 -07:00
Zach Koopmans b9b6660dc4 Add Docs to nginx benchmark.
Adds docs to nginx and refactors both Httpd and Nginx benchmarks.

Key changes:
- Add docs and make nginx tests the same as httpd (reverse, all docs, etc.).
- Make requests scale on c * b.N -> a request per thread. This works well
with both --test.benchtime=10m (do a run that lasts at least 10m) and
--test.benchtime=10x (do b.N = 10).
-- Remove a doc from both tests (1000Kb) as 1024Kb exists.

PiperOrigin-RevId: 329751091
2020-09-02 11:22:17 -07:00
Zach Koopmans 7b9bfc0ce0 Port Ruby benchmark.
PiperOrigin-RevId: 325500772
2020-08-07 13:49:07 -07:00
Zach Koopmans a7bd0a7012 Port Startup and Density Benchmarks.
PiperOrigin-RevId: 325497346
2020-08-07 13:30:39 -07:00
Fabricio Voznika f20e63e31b Add LinkAt support to gofer
Updates #1198

PiperOrigin-RevId: 325350818
2020-08-06 18:16:21 -07:00
Zach Koopmans be7079578e Port sysbench benchmark.
PiperOrigin-RevId: 324918229
2020-08-04 16:51:04 -07:00
Kevin Krakauer 12c2c6ae30 docs: tools/images.mk does not exist.
PiperOrigin-RevId: 324833018
2020-08-04 10:02:58 -07:00
Zach Koopmans 98f9527c04 Port nginx and move parsers to own package.
This change:
- Ports the nginx benchmark.
- Switches the Httpd benchmark to use 'hey' as a client.
- Moves all parsers to their own package 'tools'.

Parsers are moved to their own package because 1) parsing output of a command
is often dependent on the format of the command (e.g. 'fio --json'), 2) to
enable easier reuse, and 3) clean up and simplify actual running benchmarks
(no TestParser functions and ugly sample output in benchmark files).

PiperOrigin-RevId: 324144165
2020-07-30 21:17:45 -07:00
Jamie Liu 1b11326ecd Call lseek(0, SEEK_CUR) unconditionally in runsc fsgofer's Readdir(offset=0).
9P2000.L is silent as to how readdir RPCs interact with directory mutation. The
most performant option is for Treaddir with offset=0 to restart iteration,
avoiding needing to walk+open+clunk a new directory fid between invocations of
getdents64(2), and the VFS2 gofer client assumes this is the case. Make this
actually true for the runsc fsgofer.

Fixes #3344, #3345, #3355

PiperOrigin-RevId: 324090384
2020-07-30 15:02:22 -07:00
Zach Koopmans 78f1a18ab3 Add runsc build benchmark.
PiperOrigin-RevId: 324071377
2020-07-30 13:37:21 -07:00
Zach Koopmans 1715896fc8 Port fio benchmark
PiperOrigin-RevId: 323810654
2020-07-29 10:15:53 -07:00
Zach Koopmans 6b4e11ab50 Port node benchmark.
PiperOrigin-RevId: 323810235
2020-07-29 10:06:38 -07:00
Zach Koopmans 5873b0f43f Port tensorflow benchmark.
PiperOrigin-RevId: 323633737
2020-07-28 12:56:06 -07:00
gVisor bot 7c1c5917da Merge pull request #2797 from Rajpratik71:optimization/pip-no-cache
PiperOrigin-RevId: 323508910
2020-07-27 22:18:37 -07:00
Zach Koopmans 77552f1c77 Port ffmpeg benchmark
PiperOrigin-RevId: 323383320
2020-07-27 10:10:14 -07:00
Zach Koopmans 29e5609b22 Port redis benchmark
PiperOrigin-RevId: 323381964
2020-07-27 10:01:45 -07:00
Pratik raj b9d9418334 optimize size and time using "--no-cache-dir"
Using "--no-cache-dir" flag in pip install ,make sure dowloaded packages
by pip don't cached on system . This is a best practise which make sure
to fetch ftom repo instead of using local cached one . Further , in case
of Docker Containers , by restricing caching , we can reduce image size.
In term of stats , it depends upon the number of python packages
multiplied by their respective size . e.g for heavy packages with a lot
of dependencies it reduce a lot by don't caching pip packages.

Further , more detail information can be found at

https://medium.com/sciforce/strategies-of-docker-images-optimization-2ca9cc5719b6
2020-07-25 13:26:52 +05:30
Adin Scannell ab0262bd94 Convert go_generics tests to starlark.
For some reason these tests were broken when run via the bazel docker
container. The mechanism used was a bit crazy (self-extracting bundle),
so convert them to use straight-forward starlark rules. This has the
added advantaged that they are now independent tests.
2020-07-23 18:00:12 -07:00
Adin Scannell e1a04f84e8 Add standard entrypoints for test targets.
PiperOrigin-RevId: 322265513
2020-07-20 18:05:05 -07:00
Ian Lewis feb1d3d5a7 Clean up html on the website.
- Fixes some html validation issues.
- Fixes links on security basics blog post.
- Adds rel=noopener to links with target=_blank and adds a check to
  htmlproofer.
- Add favicon check to htmlproofer.

Fixes #3286
Fixes #3284

PiperOrigin-RevId: 321892602
2020-07-17 18:27:41 -07:00
Ting-Yu Wang 5593320bee Update README on cross-building images.
PiperOrigin-RevId: 321887956
2020-07-17 17:44:53 -07:00
Zach Koopmans 5c8c0d65b9 Port httpd benchmark
PiperOrigin-RevId: 321478001
2020-07-15 18:21:51 -07:00