Commit Graph

144 Commits

Author SHA1 Message Date
Michael Pratt 6d806ee719 Remove various uses of 'blacklist'
Updates #2972

PiperOrigin-RevId: 316942245
2020-06-17 12:34:33 -07:00
Fabricio Voznika fbe41987c9 Enable nogo again
PiperOrigin-RevId: 316011323
2020-06-11 17:34:02 -07:00
Michael Pratt 0c7a5bc69c Add nogo TODO.
PiperOrigin-RevId: 315911025
2020-06-11 09:13:01 -07:00
gVisor bot ac37979ca8 Merge pull request #1875 from zhiqiangxu:fix_scope_add
PiperOrigin-RevId: 315341669
2020-06-08 13:34:58 -07:00
Andrei Vagin 5e4d7072a6 make: Allow to specify bazel startup options.
And don't parse binary paths for the build target.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2020-06-02 11:30:26 -07:00
Adin Scannell 2c6c4365ea Move to make for tag release workflow.
This will make tag & release workflows idempotent.

PiperOrigin-RevId: 314154888
2020-06-01 10:31:26 -07:00
Adin Scannell 04a1f431e3 Fix the APT repository structure.
This change fixes the apt repository structure to avoid emiting warnings on
Ubuntu 18.04 (and potentially other versions). This requires a slight refactor
of the repository generation scripts, since we can no longer copy the same
release files for different "suites".

This should avoid the warning by setting the suite to the distribution:
  https://github.com/Debian/apt/blob/master/apt-pkg/metaindex.cc#L75

This change also moves over to the standardized Makefile entrypoint, which
makes settings clearer and enables local testing.

PiperOrigin-RevId: 313817017
2020-05-29 11:30:02 -07:00
gVisor bot 226cba97fb Merge pull request #2771 from amscanne:fix_build
PiperOrigin-RevId: 313652557
2020-05-28 13:49:31 -07:00
gVisor bot c826bb3502 Merge pull request #2748 from amscanne:go_branch
PiperOrigin-RevId: 313404235
2020-05-27 09:32:50 -07:00
Adin Scannell a6325cca51 Ensure docker group exists in the container.
The --groups command will fail if the group is not defined in
/etc/groups, even though it is specified by GID.

By coincidence, the group happens to be there for many installations of
Ubuntu 18.04 (which uses 999 for the Docker group), but it is strangely
absent in others (which use 130).

Fixes #2765
2020-05-25 18:21:03 -07:00
Adin Scannell 8437ef752d Normalize permissions in the go branch.
Fixes #2722
2020-05-20 22:57:52 -07:00
Reapor-Yurnero 059879e143 Implement gap tracking in the segment set.
This change was derived from a change by:
  Reapor-Yurnero <reapor.yurnero@gmail.com>

And has been modified by:
  Adin Scannell <ascannell@google.com>

(The original change author is preserved for the commit.)

This change implements gap tracking in the segment set by adding additional
information in each node, and using that information to speed up gap finding
from a linear scan to a O(log(n)) walk of the tree.

This gap tracking is optional, and will default to off except for segment
instances that set gapTracking equal to 1 in their const lists.

PiperOrigin-RevId: 312621607
2020-05-20 22:50:07 -07:00
Nicolas Lacasse 2374660f11 Automated rollback of changelist 312522097
PiperOrigin-RevId: 312529859
2020-05-20 12:17:06 -07:00
Nicolas Lacasse 7f57a09008 Upgrade to Bazel 3.1.0.
This upgrades the Kokoro images, bazel toolchains used by RBE runners, and
rules_go, gazelle, and go toolchain versions.

PiperOrigin-RevId: 312522097
2020-05-20 11:36:24 -07:00
Adin Scannell cbfb55869e Implement Go branch updater with GitHub actions.
PiperOrigin-RevId: 312155686
2020-05-18 14:36:16 -07:00
Adin Scannell c5a939d76c Update vm scripts to handle existing kbuilder user.
PiperOrigin-RevId: 311751972
2020-05-15 10:09:54 -07:00
Adin Scannell 4502b73d00 Update Kokoro images to include newer gcloud.
PiperOrigin-RevId: 311658774
2020-05-14 20:22:45 -07:00
Adin Scannell f589a85889 Run issue_reviver via GitHub.
PiperOrigin-RevId: 311600872
2020-05-14 14:02:43 -07:00
gVisor bot a3f97a757a Merge pull request #2513 from amscanne:website-integrated
PiperOrigin-RevId: 311184385
2020-05-12 12:55:23 -07:00
Andrei Vagin 5d54ddcf03 make: exit with non-zero code if "bazel build" failed
Without this fix, make exits with zero code when bazel build failed:

$ make run TARGETS="--abra --kadabra"
ERROR: Unrecognized option: --abra
$ echo $?
0

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2020-05-07 18:25:32 -07:00
Adin Scannell 5536073969 make: bazel docker container should clean itself up.
This change two does things:

1) Name the container based on the canonical directory path.

2) Allow the container to exit after bazel itself has exited.

The first is necessary to support multiple working directories,
while the second one allows these instances to clean up properly.

PiperOrigin-RevId: 310460748
2020-05-07 16:39:37 -07:00
Adin Scannell 1f4087e7cd Fix tags used for determining file sets.
Updates #2569
Updates #2298

PiperOrigin-RevId: 310423629
2020-05-07 13:19:01 -07:00
Adin Scannell 508e25b6d6 Adapt website to use g3doc sources and bazel.
This adapts the merged website repository to use the image and bazel
build framework. It explicitly avoids the container_image rules provided
by bazel, opting instead to build with direct docker commands when
necessary.

The relevant build commands are incorporated into the top-level
Makefile.
2020-05-06 14:15:18 -07:00
gVisor bot 660a1a1028 Internal change.
PiperOrigin-RevId: 309801320
2020-05-04 12:49:29 -07:00
Andrei Vagin c01e103256 Allow to run kvm syscall tests on the RBE cluster
PiperOrigin-RevId: 309265978
2020-04-30 11:33:07 -07:00
Andrei Vagin 44a57646d8 make_repository.sh has to print only the repo path on stdout
PiperOrigin-RevId: 309176385
2020-04-30 00:34:26 -07:00
Adin Scannell 64723470a6 Use existing bazeldefs with top-level BUILD file.
PiperOrigin-RevId: 308901116
2020-04-28 14:46:19 -07:00
Adin Scannell c60613475c Standardize all Docker images.
This change moves all Docker images to a standard location, and abstracts the
build process so that they can be maintained in an automated fashion. This also
allows the images to be architecture-independent.

All images will now be referred to by the test framework via the canonical
`gvisor.dev/images/<name>`, where `<name>` is a function of the path within the
source tree.

In a subsequent change, continuous integration will be added so that the images
will always be correct and available locally.

In the end, using `bazel` for Docker containers is simply not possible. Given
that we already have the need to use `make` with the base container (for
Docker), we extend this approach to get more flexibility.

This change also adds a self-documenting and powerful Makefile that is intended
to replace the collection of scripts in scripts. Canonical (self-documenting)
targets can be added here for targets that understand which images need to be
loaded and/or built.

PiperOrigin-RevId: 308322438
2020-04-24 14:11:42 -07:00
Adin Scannell 1481499fe2 Simplify Docker test infrastructure.
This change adds a layer of abstraction around the internal Docker APIs,
and eliminates all direct dependencies on Dockerfiles in the infrastructure.

A subsequent change will automated the generation of local images (with
efficient caching). Note that this change drops the use of bazel container
rules, as that experiment does not seem to be viable.

PiperOrigin-RevId: 308095430
2020-04-23 11:33:30 -07:00
Fabricio Voznika 7d1b7daf7e Disable nogo because it breaks Go 1.13
Even though the default build option is to use 1.14, we want to be
want to keep the ability to target different Go versions for testing
and in case the new release has bugs.

PiperOrigin-RevId: 308078876
2020-04-23 10:20:52 -07:00
Andrei Vagin 0c586946ea Specify a memory file in platform.New().
PiperOrigin-RevId: 307941984
2020-04-22 17:50:10 -07:00
Adin Scannell c615aafa21 Add internal nogo analysis & checkescape tool.
See tools/nogo/README.md.

The checkescape tool is able to perform recursive escape analysis, using the
actual generated binary to confirm the results produced by the compiler itself.

As an initial use case, this replaces the manual escape analysis tests used for
go_marshal, and validates that the CopyIn and CopyOut paths will not require
any allocation or stack splits.

Updates #2243

PiperOrigin-RevId: 307532986
2020-04-20 20:58:20 -07:00
Adin Scannell 1a597e01be Add a functional vm_test for root_test.
This change renames the tools/images directory to tools/vm for clarity, and
adds a functional vm_test. Sharding is also added to the same test, and some
documentation added around key flags & variables to describe how they work.

Subsequent changes will add vm_tests for other cases, such as the runtime tests.

PiperOrigin-RevId: 307492245
2020-04-20 15:48:27 -07:00
gVisor bot e1959f58dc Merge pull request #2321 from lubinszARM:pr_nogo
PiperOrigin-RevId: 306300032
2020-04-13 13:41:45 -07:00
Adin Scannell aa75a3da51 Fix build.sh and VM targets.
PiperOrigin-RevId: 306289643
2020-04-13 12:48:38 -07:00
Nicolas Lacasse ca868e3e38 Automated rollback of changelist 305940483
PiperOrigin-RevId: 305944892
2020-04-10 14:29:01 -07:00
Nicolas Lacasse 99056b6bd6 Upgrade Kokoro and RBE to bazel 3.0.0
PiperOrigin-RevId: 305940483
2020-04-10 14:04:07 -07:00
Nicolas Lacasse 82dfc406e2 Automated rollback of changelist 305922105
PiperOrigin-RevId: 305927989
2020-04-10 12:53:17 -07:00
Nicolas Lacasse 76c4314c4f Install Bazel 3.0.0 on Kokoro image.
PiperOrigin-RevId: 305922105
2020-04-10 12:23:10 -07:00
gVisor bot 78126611e6 Merge pull request #2253 from amscanne:nogo
PiperOrigin-RevId: 305807868
2020-04-09 19:16:46 -07:00
Nicolas Lacasse 5b41f33642 Remove "no-sandbox" tag.
It seems no longer necessary.

PiperOrigin-RevId: 305758572
2020-04-09 14:18:49 -07:00
Bin Lu ab54d4f496 remove nogo exemption for machine_arm64_unsafe.go
Minimize the use of unsafe.

Signed-off-by: Bin Lu <bin.lu@arm.com>
2020-04-09 03:54:58 -04:00
Fabricio Voznika 6dd5a1f3fe Clean up TODOs
PiperOrigin-RevId: 305592245
2020-04-08 17:58:13 -07:00
Adin Scannell 867eeb18d8 Remove lostcancel warnings.
Updates #2243
2020-04-08 10:14:34 -07:00
Adin Scannell f888b9ce83 Fix unused result errors.
This fixes a bug in the proc net directory.

Updates #2243
2020-04-08 10:14:34 -07:00
Adin Scannell 928a7c60b8 Fix all printf formatting errors.
Updates #2243
2020-04-08 10:14:34 -07:00
Adin Scannell 94b793262d Fix all copy locks violations.
This required minor restructuring of how system call tables were saved
and restored, but it makes way more sense this way.

Updates #2243
2020-04-08 10:00:14 -07:00
Rahat Mahmood 1561ae3037 go-marshal: Allow array lens to be consts and simple expressions.
Previously, go-marshal only allowed literals for array
lengths. However, it's very common for ABI structs to have a fix-sized
array whose length is defined by a constant; for example PATH_MAX.
Having to convert all such arrays to have literal lengths is too
awkward.

PiperOrigin-RevId: 304289345
2020-04-01 16:51:28 -07:00
Rahat Mahmood 507f997213 go-marshal: Improve collision detection of import statments.
Previously, the import statement collision detection mechanism aborted
go-marshal whenever it detected two imports in any package that has
the same local name. Consider this trivial package, defined by the the
following two source files:

file1.go:

package example
import (
        path/a/to/foo
)
...

file2.go:

package example
import (
       another/package/with/final/component/foo
)
...

Go-marshal previously couldn't handle generating code for the the
above package, even if none of the types marked for marshalling used
either of the imported foo packages. This turns out to be too
restrictive as we run into this a lot in practice. Examples include
"encoding/binary" vs "gvisor/pkg/binary/binary", and "sync" vs
"gvisor/pkg/sync/sync".

This change allows go-marshal to proceed with marshalling, and only
abort if the code generated by go-marshal references any such
ambiguous import names.

PiperOrigin-RevId: 304131190
2020-04-01 00:43:55 -07:00
Rahat Mahmood 840980aeba Implement automated marshalling for slices of Marshallable types.
PiperOrigin-RevId: 304119255
2020-03-31 22:56:09 -07:00