Netstack is supposed to be somewhat independent of the rest of gVisor, and
others should be able to use it without pulling in excessive dependencies.
Currently, there is no way to fight dependency creep besides careful code
review.
This change introduces a test rule `netstack_deps_check` that ensures the target
only relies on gVisor targets and a short allowlist of external dependencies.
Users who add a dependency will see an error and have to manually update the
allowlist.
The set of packages to test comes from //runsc, as it uses packages we would
expect users to commonly rely on. It was generated via:
$ find ./runsc -name BUILD | xargs grep tcpip | awk '{print $2}' | sort | uniq
(Note: We considered giving //pkg/tcpip it's own go.mod, but this breaks go
tooling.)
PiperOrigin-RevId: 368456711
Split usermem package to help remove syserror dependency in go_marshal.
New hostarch package contains code not dependent on syserror.
PiperOrigin-RevId: 365651233
Stateify methods are always called without holding the appropriate
locks. The system is paused and we know there will be no mutations
when we call Save/Load, so this is perfectly safe. However, checklocks
can't know about this, and it will always complain.
Mark stateify generated methods that touch struct fields as
"checklocksignore" to avoid this.
PiperOrigin-RevId: 364610241
We were only supporting dynamic struct types. With this change, users can make
any type dynamic. The tool (correctly) blindly just generates the remaining
methods needed to implement Marshallable using the 3 methods defined by the
user on the dynamic type.
This is helpful in situations like:
type StringArray []string
Added a test for such a use case.
PiperOrigin-RevId: 364463164
We can generate more than one apt repo for the same package. If we will
sign a package again, its file will be changed and all hashes that have
been generated before will be invalid.
This binary is used to recursively enable and generate Merkle tree files
for all files and directories in a file system from inside a gVisor
sandbox.
PiperOrigin-RevId: 362418770
The dynamic type user defines the marshalling logic, so we don't need to test
for things like alignment, absence of slices, etc.
For dynamic types, the go_marshal generator just generates the missing methods
required to implement marshal.Marshallable.
PiperOrigin-RevId: 361676311
This validates that struct fields if annotated with "// checklocks:mu" where
"mu" is a mutex field in the same struct then access to the field is only
done with "mu" locked.
All types that are guarded by a mutex must be annotated with
// +checklocks:<mutex field name>
For more details please refer to README.md.
PiperOrigin-RevId: 360729328
This makes it easier to implement dynamically sized types in go-marshal. You
really only need to implement MarshalBytes, UnmarshalBytes and SizeBytes to
implement the entire interface.
By using the `dynamic` tag, the autogenerator will generate the rest of the
methods for us.
This change also simplifies how KernelIPTGetEntries implements Marshallable
using the newly added utility.
PiperOrigin-RevId: 356397114
...and a bunch of other things as I worked through the rot. Notably:
- Upgrade to bazel 4.0.0
- Upgrade to Go 1.15.7
Remove go_branch stderr suppression; this made it quite difficult to see
what was failing while developing this patch.
PiperOrigin-RevId: 355257833
This is required only for the built-in bazel nogo functionality.
Since we roll these targets manually via the wrappers, we don't need
to use go_tool_library. The inconsistent use of these targets leads
to conflicting instantiations of go_default_library and go_tool_library,
which both contain the same output files.
PiperOrigin-RevId: 355184975
This change also adds an extra sanity check to the make_apt.sh script,
in order to ensure that this simple mistake does not occur again.
PiperOrigin-RevId: 355101754
It's unclear why permissions wind up corrupted, but these can be cleared
on any failure, similar to the bazel cache itself:
https://buildkite.com/gvisor/pipeline/builds/2304#_
PiperOrigin-RevId: 355057421
The GH Build action has been failing with the error message:
```
--- BUILD -c opt //runsc
tee: /proc/self/fd/2: No such device or address
```
tee /dev/fd/2 seems to be the canonical way of copying stdin to stderr.
So use that instead.
PiperOrigin-RevId: 353259087
Files removed from the working tree were not being properly removed from
the branch, leading to symbol conflicts while building. This requires the
change to 'git add --all' in the tools/go_branch.sh script.
But why was this not caught by CI? The "git clean -f" command by default
only cleans files in the current working directory. In order to clean the
whole tree recursively, we need to specify a pathspec, which is ".".
In addition to these fixes, re-add the "go tests" command to help prevent
this from happening again, since merges on the Go branch will happen in
GitHub actions for simplicity. The Go test is retained in BuildKite.
PiperOrigin-RevId: 351503804
gvisor-containerd-shim is not compatible with containerd 1.1 or earlier.
Starting from containerd 1.2, shim v2 is the preferred interface.
PiperOrigin-RevId: 351485556
This can race and cause issues. Instead, a manual shutdown can be done
via 'bazel-shutdown' if required for specific cases. The ARM64 builds
are now done using cross-compilation, so this hack is not necessary.
PiperOrigin-RevId: 351477908
These are primarily simplification and lint mistakes. However, minor
fixes are also included and tests added where appropriate.
PiperOrigin-RevId: 351425971
This requires several changes:
* Templates must preserve relevant tags.
* Pagetables templates are split into two targets, each preserving tags.
* The binary VDSO is similarly split into two targets, with some juggling.
* The top level tools/go_branch.sh now does a crossbuild of ARM64 as well,
and checks and merges the results of the two branches together.
Fixes#5178
PiperOrigin-RevId: 351304330
We log a warning if objdump fails, but this appears in the build log, not test
log, which can make it hard to notice.
Include it with the actual escape output as context on "(possible)" to make it
more clear when something is wrong.
PiperOrigin-RevId: 350355759
For validation, the "on" key in existing YAML files is changed to a literal
string. In the YAML spec, on is a keyword which encodes a boolean value, so
without relying on a specific implementation the YAML files are technically
not encoding an object that complies with the specification.
PiperOrigin-RevId: 350172147
This change cleans up some minor Makefile issues, and adds support for
BuildKite annotations on failure and on profiles being generated. These
annotations will make failures very clear and link to the artifacts.
This change is a stepping stone for aggregating coverage data from all
individual test jobs, as this will also happen in .buildkite/annotate.sh.
PiperOrigin-RevId: 349606598
A few images were broken with respect to aarch64. We should now
be able to run push-all-images with ARCH=aarch64 as part of the
regular continuous integration builds, and add aarch64 smoke tests
(via user emulation for now) to the regular test suite (future).
PiperOrigin-RevId: 346685462
Recursive make is difficult to follow and debug. Drop this by using
internal functions, which, while difficult, are easier than trying to
following recursive invokations.
Further simplify the Makefile by collapsing the image bits and removing
the tools/vm directory, which is effectively unused.
Fixes#4952
PiperOrigin-RevId: 346569133
Currently, if containerd is installed locally via tools/installers/containerd,
then it will not necessarily be used if containerd is installed in the system
path. This means that the existing containerd tests are all likely broken.
Also, use libbtrfs-dev instead of btrfs-tools, which is not actually required.
PiperOrigin-RevId: 344879109
This change also simplifies and documents the build_cmd pipeline, and
reduces general noise for debugging Makefile issues.
It also drops the mapping for /etc/docker/daemon.json, which if it
does not exist initially will create this as a directory (causing lots
of confusion and breaks).
PiperOrigin-RevId: 343932456
Zach Koopmans
Kevin Krakauer
Adin Scannell
gVisor bot
Rahat Mahmood
Ayush Ranjan
Andrei Vagin
Chong Cai
Bhasker Hariharan
Tamir Duberstein
Fabricio Voznika
Andrei Vagin
Michael Pratt
Lai Jiangshan
Dean Deng
Jamie Liu