Commit Graph

1878 Commits

Author SHA1 Message Date
Adin Scannell c98e7f0d19 Signalfd support
Note that the exact semantics for these signalfds are slightly different from
Linux. These signalfds are bound to the process at creation time. Reads, polls,
etc. are all associated with signals directed at that task. In Linux, all
signalfd operations are associated with current, regardless of where the
signalfd originated.

In practice, this should not be an issue given how signalfds are used. In order
to fix this however, we will need to plumb the context through all the event
APIs. This gets complicated really quickly, because the waiter APIs are all
netstack-specific, and not generally exposed to the context.  Probably not
worthwhile fixing immediately.

PiperOrigin-RevId: 269901749
2019-09-18 15:16:42 -07:00
Adin Scannell 461123ea35 Move the component into the repository structure.
The RELEASE file must be at the top-level for the signed
repository to work correctly.

PiperOrigin-RevId: 269897109
2019-09-18 14:57:12 -07:00
Bin Lu 38bc0b6b6a enable syscalls/linux to support arm64
Signed-off-by: Bin Lu <bin.lu@arm.com>
Change-Id: I45af8a54304f8bb0e248ab15f4e20b173ea9e430
2019-09-18 10:13:06 +00:00
Bin Lu 8e73e2cec5 enable kvm/testutil to support arm64
enable kvm/testutil to support arm64

The Arm64 user-mode execution stat consists of:
1, X0- X30
2, PC, SP, PSTATE
3, TPIDR_EL0, used for TLS
4, V0-V31: 32 128-bit registers for floating point and simd
5, FPSR

Currently, we first try to achieve goals 1 and 2.

This patch provids basic test utils for goals 1 & 2

Signed-off-by: Bin Lu <bin.lu@arm.com>
2019-09-18 09:57:59 +00:00
Andrei Vagin b63d56b0b4 scripts/build.sh: fix kokoro failure "KOKORO_BUILD_NIGHTLY: unbound variable"
PiperOrigin-RevId: 269690988
2019-09-17 17:26:01 -07:00
Ghanan Gowripalan 60fe8719e1 Automated rollback of changelist 268047073
PiperOrigin-RevId: 269658971
2019-09-17 14:47:09 -07:00
Nicolas Lacasse 062190d983 Follow-up fixes for image tests.
- Fix ARG syntax in Dockerfiles.
- Fix curl commands in Dockerfiles.
- Fix some paths in proctor binaries.
- Check error from Walk in search helper.

PiperOrigin-RevId: 269641686
2019-09-17 13:29:19 -07:00
Andrei Vagin 3b7119a7c9 platform/ptrace: log exit code for stub processes
PiperOrigin-RevId: 269631877
2019-09-17 12:45:22 -07:00
Ian Gudger 747320a7aa Update remaining users of LinkEndpoints to not refer to them as an ID.
PiperOrigin-RevId: 269614517
2019-09-17 11:31:00 -07:00
Adin Scannell 191297ecbe Fix artifact pattern for repositories.
PiperOrigin-RevId: 269468986
2019-09-16 18:41:13 -07:00
Nicolas Lacasse 24b7eb2f86 Refactor and clean up image tests.
* Use multi-stage builds in Dockerfiles.
* Combine all proctor binaries into a single binary.
* Change the TestRunner interface to reduce code duplication.

PiperOrigin-RevId: 269462101
2019-09-16 17:51:22 -07:00
Michael Pratt 56cb004218 Migrate from gflags to absl flags
absl flags are more modern and we can easily depend on them directly.

The repo now successfully builds with --incompatible_load_cc_rules_from_bzl.

PiperOrigin-RevId: 269387081
2019-09-16 11:58:27 -07:00
Fabricio Voznika 010b093258 Bring back to life features lost in recent refactor
- Sandbox logs are generated when running tests
- Kokoro uploads the sandbox logs
- Supports multiple parallel runs
- Revive script to install locally built runsc with docker

PiperOrigin-RevId: 269337274
2019-09-16 08:17:00 -07:00
Andrei Vagin 239a07aabf gvisor: return ENOTDIR from the unlink syscall
ENOTDIR has to be returned when a component used as a directory in
pathname is not, in  fact,  a directory.

PiperOrigin-RevId: 269037893
2019-09-13 21:44:57 -07:00
Adin Scannell 2bbf73d9ed Remove stale configurations.
PiperOrigin-RevId: 268947847
2019-09-13 11:39:55 -07:00
Adin Scannell a8834fc555 Update p9 to support flipcall.
PiperOrigin-RevId: 268845090
2019-09-12 23:37:31 -07:00
Adin Scannell 7c6ab6a219 Implement splice methods for pipes and sockets.
This also allows the tee(2) implementation to be enabled, since dup can now be
properly supported via WriteTo.

Note that this change necessitated some minor restructoring with the
fs.FileOperations splice methods. If the *fs.File is passed through directly,
then only public API methods are accessible, which will deadlock immediately
since the locking is already done by fs.Splice. Instead, we pass through an
abstract io.Reader or io.Writer, which elide locks and use the underlying
fs.FileOperations directly.

PiperOrigin-RevId: 268805207
2019-09-12 17:43:27 -07:00
Michael Pratt df5d377521 Remove go_test from go_stateify and go_marshal
They are no-ops, so the standard rule works fine.

PiperOrigin-RevId: 268776264
2019-09-12 15:10:17 -07:00
gVisor bot 78cfbbda4b Merge pull request #843 from nlacasse:version
PiperOrigin-RevId: 268772451
2019-09-12 14:55:24 -07:00
Ghanan Gowripalan 857940d30d Automated rollback of changelist 268047073
PiperOrigin-RevId: 268757842
2019-09-12 13:52:25 -07:00
Adin Scannell 574eda8880 Update repository directory structure.
Currently it will not work with apt out of the box, as we
require the dists/ prefix, along with a distribution name.

This tweaks the overall structure to allow for the same URL
prefix to be used for all repositories, and enables multiple
architectures.

Fixes #852

PiperOrigin-RevId: 268756104
2019-09-12 13:44:18 -07:00
Nicolas Lacasse e2528cae76 Bump bazel toolchain.
This fixes the RBE warning:
DEBUG: rbe_default not using checked in configs; Bazel version 0.29.1 was
picked/selected but no checked in config was found in map[...]
PiperOrigin-RevId: 268747944
2019-09-12 13:04:50 -07:00
Adin Scannell 69f2c41b7a Drop unavailable package.
PiperOrigin-RevId: 268614014
2019-09-11 22:30:28 -07:00
Adin Scannell 96a25e080c Ensure appropriate tools are installed on image.
PiperOrigin-RevId: 268608466
2019-09-11 22:13:26 -07:00
Adin Scannell 1e6bdd5855 Update key environment variables.
PiperOrigin-RevId: 268604220
2019-09-11 21:04:42 -07:00
Adin Scannell c06ef5131f Fix authorization for continuous integration.
The credentials must be explicitly refreshed for pushing to
the repository on the Go branch.

PiperOrigin-RevId: 268589817
2019-09-11 18:49:53 -07:00
Nicolas Lacasse f52dd451ed Bump bazel, rules_go, gazelle, and go toolchain to latest versions.
PiperOrigin-RevId: 268486127
2019-09-11 10:19:50 -07:00
Ian Lewis d24be656c9 Update required Bazel version in README.
PiperOrigin-RevId: 268397389
2019-09-11 00:09:59 -07:00
Nicolas Lacasse c1a8275b47 Fix `runsc --version` and add a test.
We need to include the `--stamp` flag in `tools/workspace_status.sh` for
the version to be picked up by the linker. Not sure why.

Also changes the VERSION string to STABLE_VERSION, which will cause the
program to be re-linked if the string changes.

Fixes #830
2019-09-10 15:41:47 -07:00
Adin Scannell 849c57314f Fix minor Kokoro issues.
A recent Kokoro change pointed to go_tests.cfg (in line with the
other configurations), which unfortunately broke the presubmits.

This change also enabled the KVM tests, which were still using a
remote execution strategy.

This fixes both of these issues and allows presubmits to pass.

One additional test was caught with this case, which seems to
have been broken. It's unclear why this was not being caught.

PiperOrigin-RevId: 268166291
2019-09-10 00:38:52 -07:00
Ian Gudger 9dfcd8b09f Fix ephemeral port leak.
Fix a bug where udp.(*endpoint).Disconnect [accessible in gVisor via
epsocket.(*SocketOperations).Connect with AF_UNSPEC] would leak a port
reservation if the socket/endpoint had an ephemeral port assigned to it.

glibc's getaddrinfo uses connect with AF_UNSPEC, causing each call of
getaddrinfo to leak a port. Call getaddrinfo too many times and you run out of
ports (shows up as connect returning EAGAIN and getaddrinfo returning
EAI_NONAME "Name or service not known").

PiperOrigin-RevId: 268071160
2019-09-09 14:02:00 -07:00
Rahat Mahmood 3733b9b893 go_marshal: Implement automatic generation of ABI marshalling code.
This CL implements go_marshal, a code generation utility for
automatically serializing and deserializing ABI structs.

The go_marshal tool automatically generates implementations of the new
marshal interface. Unlike binary.Marshal/Unmarshal, the generated
interface implementations use no runtime reflection, and translates to
a single memcpy for most structs. See go_marshal/README.md for
details.

PiperOrigin-RevId: 268065475
2019-09-09 13:36:39 -07:00
gVisor bot 6af9a9850a Merge pull request #824 from amscanne:fix_build2
PiperOrigin-RevId: 268062329
2019-09-09 13:21:41 -07:00
Ghanan Gowripalan a8943325db Join IPv6 all-nodes and solicited-node multicast addresses where appropriate.
The IPv6 all-nodes multicast address will be joined on NIC enable, and the
appropriate IPv6 solicited-node multicast address will be joined when IPv6
addresses are added.

Tests: Test receiving packets destined to the IPv6 link-local all-nodes
multicast address and the IPv6 solicted node address of an added IPv6 address.
PiperOrigin-RevId: 268047073
2019-09-09 12:06:06 -07:00
Ian Gudger fe1f521077 Remove reundant global tcpip.LinkEndpointID.
PiperOrigin-RevId: 267709597
2019-09-06 18:01:14 -07:00
Jamie Liu 9e1cbdf565 Indicate flipcall synchronization to the Go race detector.
Since each Endpoint has a distinct mapping of the packet window, the Go race
detector does not recognize accesses by connected Endpoints to be related. This
means that this change isn't necessary for the Go race detector to accept
accesses of flipcall.Endpoint.Data(), but it *is* necessary for it to accept
accesses to shared variables outside the scope of flipcall that are
synchronized by flipcall.Endpoint state; see updated test for an example.

RaceReleaseMerge is needed (instead of RaceRelease) because calls to
raceBecomeInactive() from *unrelated* Endpoints can occur in any order.
(DowngradableRWMutex.RUnlock() has a similar property: calls to RUnlock() on
the same DowngradableRWMutex from different goroutines can occur in any order.
Remove the TODO asking to explain this now that this is understood.)

PiperOrigin-RevId: 267705325
2019-09-06 17:25:07 -07:00
Adin Scannell 1a0a940587 Fix repository build scripts
This has the following fixes:

* Packages are passed to the tools/make_repository.sh command.
* All matching tags are built, for commits with multiple.
* The binary path is generated by the build command.
* Output from signing the repository is supressed.
* Allow a release author.

Change-Id: I2d08954ba76e35612f352be99d5bb99080f80892
2019-09-06 14:28:12 -07:00
Michael Pratt 98f7fbb59f Load C++ rules from @rules_cc
See https://github.com/bazelbuild/bazel/issues/8743. This will be required in
Bazel 1.0.

Protobuf was updated in
bf0c69e130 (diff-96239ee297e0a92ac6ff96a6bc434ef0).

GoogleTest was updated in
6fd262ecf7.

gflags has not yet been updated, so the repo still won't build with
--incompatible_load_cc_rules_from_bzl.

Tested with buildifier -warnings=native-cc -lint=warn **/BUILD.

PiperOrigin-RevId: 267638515
2019-09-06 11:29:00 -07:00
Ian Lewis 0bfffbcb01 Ignore the root container when calculating oom_score_adj for the sandbox.
This is done because the root container for CRI is the infrastructure (pause)
container and always gets a low oom_score_adj. We do this to ensure that only
the oom_score_adj of user containers is used to calculated the sandbox
oom_score_adj.

Implemented in runsc rather than the containerd shim as it's a bit cleaner to
implement here (in the shim it would require overwriting the oomScoreAdj and
re-writing out the config.json again). This processing is Kubernetes(CRI)
specific but we are currently only supporting CRI for multi-container support
anyway.

PiperOrigin-RevId: 267507706
2019-09-05 19:21:25 -07:00
Nicolas Lacasse 7e94f171f4 Better strace logs for statx.
PiperOrigin-RevId: 267498537
2019-09-05 18:03:53 -07:00
gVisor bot 487c6f0344 Merge pull request #809 from google:amscanne-patch-1
PiperOrigin-RevId: 267484802
2019-09-05 16:46:15 -07:00
Bhasker Hariharan eb074a61f2 Fix bug in proc_test.
TestNoDuplicates is racy as it tries to read the /proc file system
while the test is running. But it's possible that from the time a
directory entries are read and each entry processed something could
change and in some cases the entry being processed could have been
deleted. In such cases we should not fail the test but just
ignore the error and move on.

PiperOrigin-RevId: 267483094
2019-09-05 16:40:46 -07:00
Jamie Liu fbdd3ff1da Deflake aio_test.
- Most AIO tests call io_setup(nr_events = 128). sizeof(struct io_event)
(128*32 = 4096). However, the actual size of the mapping created by
io_setup() is determined by:

(from fs/aio.c:ioctx_alloc())
/*
 * We keep track of the number of available ringbuffer slots, to prevent
 * overflow (reqs_available), and we also use percpu counters for this.
 *
 * So since up to half the slots might be on other cpu's percpu counters
 * and unavailable, double nr_events so userspace sees what they
 * expected: additionally, we move req_batch slots to/from percpu
 * counters at a time, so make sure that isn't 0:
 */
nr_events = max(nr_events, num_possible_cpus() * 4);
nr_events *= 2;

(from fs/aio.c:aio_setup_ring())
/* Compensate for the ring buffer's head/tail overlap entry */
nr_events += 2; /* 1 is required, 2 for good luck */
size = sizeof(struct aio_ring);
size += sizeof(struct io_event) * nr_events;
nr_pages = PFN_UP(size);

When we mremap() only the first page of a multi-page AIO ring buffer
mapping, fs/aio.c:aio_ring_mremap() updates struct kioctx::mmap_base -
but struct kioctx::mmap_size is untouched, so sys_io_destroy() =>
kill_ioctx() vm_unmaps() the mremapped page, plus some number of pages
after it. Just get the actual size of the mapping from /proc/self/maps.

- Delete test case MremapOver; while it is correct that Linux will not
complain if you overwrite the AIO ring buffer with another mapping, it
won't actually work in the sense that AIO events will not be written to
the new mapping, because Linux stores the struct pages of the ring
buffer in struct kioctx::ring_pages and writes to those through kmap()
rather than using userspace addresses.

- Don't munmap() after mremap(MREMAP_FIXED) returns EFAULT; see new
comment in factored-out test case MremapExpansion.

PiperOrigin-RevId: 267482903
2019-09-05 16:36:44 -07:00
Robert Tonic c2ae77a607 Apply go fmt to the fsgofer changes. 2019-09-05 17:12:09 -04:00
Robert Tonic 4288a57883 Remove seccomp permissions, and clean up the Attach logic. 2019-09-05 15:26:16 -04:00
Robert Tonic 4573efe84b Switch from net to unet to open Unix Domain Sockets. 2019-09-05 07:16:36 -04:00
Bin Lu a25a97624c Change syscall.Dup2 to syscall.Dup3
Signed-off-by: Bin Lu <bin.lu@arm.com>
2019-09-05 07:51:12 +00:00
Ian Lewis e31686500d Allow non-unique group IDs in bazel docker containers
Allow non-unique group IDs in the bazel docker container in order to avoid
failures using host group IDs that are already present in the image.

Issue #801

PiperOrigin-RevId: 267306089
2019-09-04 23:20:12 -07:00
Adin Scannell 91518fd553
Fix build when no tags are present
This should correct the continuous build.
2019-09-04 22:24:42 -07:00
Ian Gudger fbbb2f7ed6 Run proc_net tests.
PiperOrigin-RevId: 267280086
2019-09-04 19:08:12 -07:00