Commit Graph

128 Commits

Author SHA1 Message Date
Adin Scannell c5a939d76c Update vm scripts to handle existing kbuilder user.
PiperOrigin-RevId: 311751972
2020-05-15 10:09:54 -07:00
Adin Scannell 4502b73d00 Update Kokoro images to include newer gcloud.
PiperOrigin-RevId: 311658774
2020-05-14 20:22:45 -07:00
Adin Scannell f589a85889 Run issue_reviver via GitHub.
PiperOrigin-RevId: 311600872
2020-05-14 14:02:43 -07:00
gVisor bot a3f97a757a Merge pull request #2513 from amscanne:website-integrated
PiperOrigin-RevId: 311184385
2020-05-12 12:55:23 -07:00
Andrei Vagin 5d54ddcf03 make: exit with non-zero code if "bazel build" failed
Without this fix, make exits with zero code when bazel build failed:

$ make run TARGETS="--abra --kadabra"
ERROR: Unrecognized option: --abra
$ echo $?
0

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2020-05-07 18:25:32 -07:00
Adin Scannell 5536073969 make: bazel docker container should clean itself up.
This change two does things:

1) Name the container based on the canonical directory path.

2) Allow the container to exit after bazel itself has exited.

The first is necessary to support multiple working directories,
while the second one allows these instances to clean up properly.

PiperOrigin-RevId: 310460748
2020-05-07 16:39:37 -07:00
Adin Scannell 1f4087e7cd Fix tags used for determining file sets.
Updates #2569
Updates #2298

PiperOrigin-RevId: 310423629
2020-05-07 13:19:01 -07:00
Adin Scannell 508e25b6d6 Adapt website to use g3doc sources and bazel.
This adapts the merged website repository to use the image and bazel
build framework. It explicitly avoids the container_image rules provided
by bazel, opting instead to build with direct docker commands when
necessary.

The relevant build commands are incorporated into the top-level
Makefile.
2020-05-06 14:15:18 -07:00
gVisor bot 660a1a1028 Internal change.
PiperOrigin-RevId: 309801320
2020-05-04 12:49:29 -07:00
Andrei Vagin c01e103256 Allow to run kvm syscall tests on the RBE cluster
PiperOrigin-RevId: 309265978
2020-04-30 11:33:07 -07:00
Andrei Vagin 44a57646d8 make_repository.sh has to print only the repo path on stdout
PiperOrigin-RevId: 309176385
2020-04-30 00:34:26 -07:00
Adin Scannell 64723470a6 Use existing bazeldefs with top-level BUILD file.
PiperOrigin-RevId: 308901116
2020-04-28 14:46:19 -07:00
Adin Scannell c60613475c Standardize all Docker images.
This change moves all Docker images to a standard location, and abstracts the
build process so that they can be maintained in an automated fashion. This also
allows the images to be architecture-independent.

All images will now be referred to by the test framework via the canonical
`gvisor.dev/images/<name>`, where `<name>` is a function of the path within the
source tree.

In a subsequent change, continuous integration will be added so that the images
will always be correct and available locally.

In the end, using `bazel` for Docker containers is simply not possible. Given
that we already have the need to use `make` with the base container (for
Docker), we extend this approach to get more flexibility.

This change also adds a self-documenting and powerful Makefile that is intended
to replace the collection of scripts in scripts. Canonical (self-documenting)
targets can be added here for targets that understand which images need to be
loaded and/or built.

PiperOrigin-RevId: 308322438
2020-04-24 14:11:42 -07:00
Adin Scannell 1481499fe2 Simplify Docker test infrastructure.
This change adds a layer of abstraction around the internal Docker APIs,
and eliminates all direct dependencies on Dockerfiles in the infrastructure.

A subsequent change will automated the generation of local images (with
efficient caching). Note that this change drops the use of bazel container
rules, as that experiment does not seem to be viable.

PiperOrigin-RevId: 308095430
2020-04-23 11:33:30 -07:00
Fabricio Voznika 7d1b7daf7e Disable nogo because it breaks Go 1.13
Even though the default build option is to use 1.14, we want to be
want to keep the ability to target different Go versions for testing
and in case the new release has bugs.

PiperOrigin-RevId: 308078876
2020-04-23 10:20:52 -07:00
Andrei Vagin 0c586946ea Specify a memory file in platform.New().
PiperOrigin-RevId: 307941984
2020-04-22 17:50:10 -07:00
Adin Scannell c615aafa21 Add internal nogo analysis & checkescape tool.
See tools/nogo/README.md.

The checkescape tool is able to perform recursive escape analysis, using the
actual generated binary to confirm the results produced by the compiler itself.

As an initial use case, this replaces the manual escape analysis tests used for
go_marshal, and validates that the CopyIn and CopyOut paths will not require
any allocation or stack splits.

Updates #2243

PiperOrigin-RevId: 307532986
2020-04-20 20:58:20 -07:00
Adin Scannell 1a597e01be Add a functional vm_test for root_test.
This change renames the tools/images directory to tools/vm for clarity, and
adds a functional vm_test. Sharding is also added to the same test, and some
documentation added around key flags & variables to describe how they work.

Subsequent changes will add vm_tests for other cases, such as the runtime tests.

PiperOrigin-RevId: 307492245
2020-04-20 15:48:27 -07:00
gVisor bot e1959f58dc Merge pull request #2321 from lubinszARM:pr_nogo
PiperOrigin-RevId: 306300032
2020-04-13 13:41:45 -07:00
Adin Scannell aa75a3da51 Fix build.sh and VM targets.
PiperOrigin-RevId: 306289643
2020-04-13 12:48:38 -07:00
Nicolas Lacasse ca868e3e38 Automated rollback of changelist 305940483
PiperOrigin-RevId: 305944892
2020-04-10 14:29:01 -07:00
Nicolas Lacasse 99056b6bd6 Upgrade Kokoro and RBE to bazel 3.0.0
PiperOrigin-RevId: 305940483
2020-04-10 14:04:07 -07:00
Nicolas Lacasse 82dfc406e2 Automated rollback of changelist 305922105
PiperOrigin-RevId: 305927989
2020-04-10 12:53:17 -07:00
Nicolas Lacasse 76c4314c4f Install Bazel 3.0.0 on Kokoro image.
PiperOrigin-RevId: 305922105
2020-04-10 12:23:10 -07:00
gVisor bot 78126611e6 Merge pull request #2253 from amscanne:nogo
PiperOrigin-RevId: 305807868
2020-04-09 19:16:46 -07:00
Nicolas Lacasse 5b41f33642 Remove "no-sandbox" tag.
It seems no longer necessary.

PiperOrigin-RevId: 305758572
2020-04-09 14:18:49 -07:00
Bin Lu ab54d4f496 remove nogo exemption for machine_arm64_unsafe.go
Minimize the use of unsafe.

Signed-off-by: Bin Lu <bin.lu@arm.com>
2020-04-09 03:54:58 -04:00
Fabricio Voznika 6dd5a1f3fe Clean up TODOs
PiperOrigin-RevId: 305592245
2020-04-08 17:58:13 -07:00
Adin Scannell 867eeb18d8 Remove lostcancel warnings.
Updates #2243
2020-04-08 10:14:34 -07:00
Adin Scannell f888b9ce83 Fix unused result errors.
This fixes a bug in the proc net directory.

Updates #2243
2020-04-08 10:14:34 -07:00
Adin Scannell 928a7c60b8 Fix all printf formatting errors.
Updates #2243
2020-04-08 10:14:34 -07:00
Adin Scannell 94b793262d Fix all copy locks violations.
This required minor restructuring of how system call tables were saved
and restored, but it makes way more sense this way.

Updates #2243
2020-04-08 10:00:14 -07:00
Rahat Mahmood 1561ae3037 go-marshal: Allow array lens to be consts and simple expressions.
Previously, go-marshal only allowed literals for array
lengths. However, it's very common for ABI structs to have a fix-sized
array whose length is defined by a constant; for example PATH_MAX.
Having to convert all such arrays to have literal lengths is too
awkward.

PiperOrigin-RevId: 304289345
2020-04-01 16:51:28 -07:00
Rahat Mahmood 507f997213 go-marshal: Improve collision detection of import statments.
Previously, the import statement collision detection mechanism aborted
go-marshal whenever it detected two imports in any package that has
the same local name. Consider this trivial package, defined by the the
following two source files:

file1.go:

package example
import (
        path/a/to/foo
)
...

file2.go:

package example
import (
       another/package/with/final/component/foo
)
...

Go-marshal previously couldn't handle generating code for the the
above package, even if none of the types marked for marshalling used
either of the imported foo packages. This turns out to be too
restrictive as we run into this a lot in practice. Examples include
"encoding/binary" vs "gvisor/pkg/binary/binary", and "sync" vs
"gvisor/pkg/sync/sync".

This change allows go-marshal to proceed with marshalling, and only
abort if the code generated by go-marshal references any such
ambiguous import names.

PiperOrigin-RevId: 304131190
2020-04-01 00:43:55 -07:00
Rahat Mahmood 840980aeba Implement automated marshalling for slices of Marshallable types.
PiperOrigin-RevId: 304119255
2020-03-31 22:56:09 -07:00
Zach Koopmans e36eccc4b1 BigQuery schema for benchmark-tools dashboard.
PiperOrigin-RevId: 303805784
2020-03-30 13:05:45 -07:00
Zach Koopmans 4aee370640 Internal change.
PiperOrigin-RevId: 303773475
2020-03-30 10:44:55 -07:00
Adin Scannell a5742f177a Add nogo exemption for machine_arm64_unsafe.go 2020-03-26 12:29:14 -07:00
Adin Scannell 882ed330e6 nogo: enable sanitizers.
This enables all relevant santizers (though most analyzers will not find
much, it will prevent instances from creeping in), and codifies existing
exceptions in tools/nogo.js to be fixed.
2020-03-25 16:21:27 -07:00
Adin Scannell d440fe0613 Fix go_marshal Example name.
There is a canonical naming convention for Examples, which are checked
by analyzers. This must be fixed since adding exceptions for generated
code will be more challenging.
2020-03-25 16:18:05 -07:00
Eyal Soha fed59953aa Statically link libpthread for static c++ binaries.
The posix_server works fine when run in locally or in docker but fails in the
kokoro GCP build environment. Linking libpthread statically fixes it.

PiperOrigin-RevId: 302139082
2020-03-20 18:24:00 -07:00
Zach Koopmans b9210b2855 Fix bm-tools to run on bazel.
Fixes random stuff that is broken on bazel/kokoro.
- random output coming back as "bytes" object instead of str
- missed syntax error in bazel
- a flag is missing in the version of gcloud on kokoro

PiperOrigin-RevId: 301915289
2020-03-19 16:07:05 -07:00
Eyal Soha 3192e55ffe Packetimpact in Go with c++ stub
PiperOrigin-RevId: 301382690
2020-03-17 08:53:27 -07:00
Adin Scannell e5d9a4010b Add ability to execute go.mod in gopath context. 2020-02-28 10:21:17 -08:00
Adin Scannell c96bb4d2eb Fix apt-get reliability issues.
This is frequently causing the core build scripts to fail. The core ubuntu
distribution will perform an auto-update at first start, which may cause the
lock file to be held. All apt-get commands may be done in a loop in order to
retry to avoid this issue. We may want to consider retrying other pieces, but
for now this should avoid the most frequent cause of build flakes.

PiperOrigin-RevId: 297704789
2020-02-27 15:36:13 -08:00
Rahat Mahmood aa9f8abaef Implement automated marshalling for newtypes on arrays.
PiperOrigin-RevId: 297693838
2020-02-27 14:52:26 -08:00
Adin Scannell 8e2b14fecf Use automated release notes, if available.
PiperOrigin-RevId: 297628615
2020-02-27 10:22:41 -08:00
Adin Scannell 10aa4d3b34 Factor platform tags.
PiperOrigin-RevId: 296519566
2020-02-21 15:06:08 -08:00
Zach Koopmans 3733499952 Fix master installer.
Sometimes, when we start a new instance, the file
lock on "apt" is locked. Add a loop to the master
installer.

In addition, the "apt-get install" fails to register
runsc in docker, so run the appropriate scripts to
get that to happen.

Also, add some helpful log messages.

PiperOrigin-RevId: 296497357
2020-02-21 13:18:51 -08:00
gVisor bot f1b72752e5 Implement automated marshalling for newtypes on primitives.
PiperOrigin-RevId: 296322954
2020-02-20 16:23:56 -08:00