Build constraints are now inferred from go:build directives rather than +build
directives. +build directives are still emitted in generated files as required
in Go 1.16 and earlier.
Note that go/build/constraint was added in Go 1.16, so gVisor now requires Go
1.16.
PiperOrigin-RevId: 387240779
With debugging enabled, go-marshal can generate too much output for
bazel under default configurations, which can cause builds to
fail. The limit defaults to 1 MB.
PiperOrigin-RevId: 372030402
gohacks.Memmove() takes in the number of bytes to move. The current generated
code passes len(src) and len(dst) as the number of bytes to move.
However, the marshal.Marshallable interface allows passing in larger buffers.
The stated precondition is that the buffer should be "at least" SizeBytes()
in length but it is allowed to be larger.
This change now correctly calls Memmove with the argument for the number of
bytes to move as type.SizeBytes(). This was caught when I made lisafs use the
Unsafe marshalling API and got a lot of memory violations.
PiperOrigin-RevId: 368952642
Split usermem package to help remove syserror dependency in go_marshal.
New hostarch package contains code not dependent on syserror.
PiperOrigin-RevId: 365651233
We were only supporting dynamic struct types. With this change, users can make
any type dynamic. The tool (correctly) blindly just generates the remaining
methods needed to implement Marshallable using the 3 methods defined by the
user on the dynamic type.
This is helpful in situations like:
type StringArray []string
Added a test for such a use case.
PiperOrigin-RevId: 364463164
The dynamic type user defines the marshalling logic, so we don't need to test
for things like alignment, absence of slices, etc.
For dynamic types, the go_marshal generator just generates the missing methods
required to implement marshal.Marshallable.
PiperOrigin-RevId: 361676311
This makes it easier to implement dynamically sized types in go-marshal. You
really only need to implement MarshalBytes, UnmarshalBytes and SizeBytes to
implement the entire interface.
By using the `dynamic` tag, the autogenerator will generate the rest of the
methods for us.
This change also simplifies how KernelIPTGetEntries implements Marshallable
using the newly added utility.
PiperOrigin-RevId: 356397114
These are primarily simplification and lint mistakes. However, minor
fixes are also included and tests added where appropriate.
PiperOrigin-RevId: 351425971
This change also adds support to go_stateify for detecting an appropriate
receiver name, avoiding a large number of false positives.
PiperOrigin-RevId: 335994587
Previously, the go-marshal-generated tests did not respect build
tags. This can cause the test to unbuildable under some build
configurations, as the original types the tests refer to may not be
defined.
This CL copies the build tags from the input files to the test,
similar to the generated library; however test packages have an
additional constraint. A test package cannot be totally empty
(i.e. have no test/example/benchmark defined), otherwise the go
compiler returns an error. To ensure the generated test package always
contains a testable entity under all build configurations, we now emit
an extra test file with no build tags that contains a single no-op
example.
PiperOrigin-RevId: 334496821
CopyContext is a better name for the interface because from
go-marshal's perspective, the interface has nothing to do with a
task. A kernel.Task happens to implement the interface, but so can
other things like MemoryManager and IO sequences.
PiperOrigin-RevId: 331959678
This immediately revealed an escape analysis violation (!), where
the sync.Map was being used in a context that escapes were not
allowed. This is a relatively minor fix and is included.
PiperOrigin-RevId: 328611237
Due to how marshallable interface implementation was generated, we could not
marshal a struct whose named started with W because there was a naming
collision with parameter (w io.Writer) and type (w *StuctName).
Used "writer" as parameter name to avoid collision.
PiperOrigin-RevId: 328343930
- All Marshal* and Unmarshal* methods now require buffers to be correctly sized
- Only the Copy{In/Out} variants can handle smaller buffers (or address spaces)
PiperOrigin-RevId: 322953881
See tools/nogo/README.md.
The checkescape tool is able to perform recursive escape analysis, using the
actual generated binary to confirm the results produced by the compiler itself.
As an initial use case, this replaces the manual escape analysis tests used for
go_marshal, and validates that the CopyIn and CopyOut paths will not require
any allocation or stack splits.
Updates #2243
PiperOrigin-RevId: 307532986
Previously, go-marshal only allowed literals for array
lengths. However, it's very common for ABI structs to have a fix-sized
array whose length is defined by a constant; for example PATH_MAX.
Having to convert all such arrays to have literal lengths is too
awkward.
PiperOrigin-RevId: 304289345
Previously, the import statement collision detection mechanism aborted
go-marshal whenever it detected two imports in any package that has
the same local name. Consider this trivial package, defined by the the
following two source files:
file1.go:
package example
import (
path/a/to/foo
)
...
file2.go:
package example
import (
another/package/with/final/component/foo
)
...
Go-marshal previously couldn't handle generating code for the the
above package, even if none of the types marked for marshalling used
either of the imported foo packages. This turns out to be too
restrictive as we run into this a lot in practice. Examples include
"encoding/binary" vs "gvisor/pkg/binary/binary", and "sync" vs
"gvisor/pkg/sync/sync".
This change allows go-marshal to proceed with marshalling, and only
abort if the code generated by go-marshal references any such
ambiguous import names.
PiperOrigin-RevId: 304131190
There is a canonical naming convention for Examples, which are checked
by analyzers. This must be fixed since adding exceptions for generated
code will be more challenging.
Users of the API only care about whether the copy in/out succeeds in
their entirety, which is already signalled by the returned error.
PiperOrigin-RevId: 296297843
This can happen due to conditional compilation, where a subset of the
source files contain no marshallable types. go-marshal is still
required to write an output file in these cases, since bazel defines
the output package before calling go-marshal.
PiperOrigin-RevId: 296074321
Since we tag entire packages as marshallable, due to conditional
compiling for different architectures we can end up with sets of
source files that don't contain any marshallable types. It's safe to
silently ignore this scenario.
PiperOrigin-RevId: 295831871
- Add missing build tags to files in the abi package.
- Add the marshal package as a sentry dependency, allowed by deps_test.
- Fix an issue with our top-level go_library BUILD rule, which
incorrectly shadows the variable containing the input set of source
files. This caused the expansion for the go_marshal clause to
silently omit input files.
- Fix formatting when copying build tags to gomarshal-generated files.
- Fix a bug with import statement collision detection in go-marshal.
PiperOrigin-RevId: 295112284
Add new Marshallable interface methods CopyIn/CopyOut, which can be directly
called on the marshalled object, avoiding an interface indirection. Such
indirections are problematic because they always cause the marshalled object to
escape.
PiperOrigin-RevId: 295028010
Note that files will need to be appropriately segmented in order for the
mechanism to work, in suffixes implying special tags. This only needs to happen
for cases where marshal or state structures are defined, which should be rare
and mostly architecture specific.
PiperOrigin-RevId: 293231579
Because the abi will depend on the core types for marshalling (usermem,
context, safemem, safecopy), these need to be flattened from the sentry
directory. These packages contain no sentry-specific details.
PiperOrigin-RevId: 291811289
Jamie Liu
Zach Koopmans
Kevin Krakauer
Rahat Mahmood
Ayush Ranjan
Adin Scannell
Ting-Yu Wang
gVisor bot
Boyuan He