Commit Graph

1808 Commits

Author SHA1 Message Date
gVisor bot 78cfbbda4b Merge pull request #843 from nlacasse:version
PiperOrigin-RevId: 268772451
2019-09-12 14:55:24 -07:00
Ghanan Gowripalan 857940d30d Automated rollback of changelist 268047073
PiperOrigin-RevId: 268757842
2019-09-12 13:52:25 -07:00
Adin Scannell 574eda8880 Update repository directory structure.
Currently it will not work with apt out of the box, as we
require the dists/ prefix, along with a distribution name.

This tweaks the overall structure to allow for the same URL
prefix to be used for all repositories, and enables multiple
architectures.

Fixes #852

PiperOrigin-RevId: 268756104
2019-09-12 13:44:18 -07:00
Nicolas Lacasse e2528cae76 Bump bazel toolchain.
This fixes the RBE warning:
DEBUG: rbe_default not using checked in configs; Bazel version 0.29.1 was
picked/selected but no checked in config was found in map[...]
PiperOrigin-RevId: 268747944
2019-09-12 13:04:50 -07:00
Adin Scannell 69f2c41b7a Drop unavailable package.
PiperOrigin-RevId: 268614014
2019-09-11 22:30:28 -07:00
Adin Scannell 96a25e080c Ensure appropriate tools are installed on image.
PiperOrigin-RevId: 268608466
2019-09-11 22:13:26 -07:00
Adin Scannell 1e6bdd5855 Update key environment variables.
PiperOrigin-RevId: 268604220
2019-09-11 21:04:42 -07:00
Adin Scannell c06ef5131f Fix authorization for continuous integration.
The credentials must be explicitly refreshed for pushing to
the repository on the Go branch.

PiperOrigin-RevId: 268589817
2019-09-11 18:49:53 -07:00
Nicolas Lacasse f52dd451ed Bump bazel, rules_go, gazelle, and go toolchain to latest versions.
PiperOrigin-RevId: 268486127
2019-09-11 10:19:50 -07:00
Ian Lewis d24be656c9 Update required Bazel version in README.
PiperOrigin-RevId: 268397389
2019-09-11 00:09:59 -07:00
Nicolas Lacasse c1a8275b47 Fix `runsc --version` and add a test.
We need to include the `--stamp` flag in `tools/workspace_status.sh` for
the version to be picked up by the linker. Not sure why.

Also changes the VERSION string to STABLE_VERSION, which will cause the
program to be re-linked if the string changes.

Fixes #830
2019-09-10 15:41:47 -07:00
Adin Scannell 849c57314f Fix minor Kokoro issues.
A recent Kokoro change pointed to go_tests.cfg (in line with the
other configurations), which unfortunately broke the presubmits.

This change also enabled the KVM tests, which were still using a
remote execution strategy.

This fixes both of these issues and allows presubmits to pass.

One additional test was caught with this case, which seems to
have been broken. It's unclear why this was not being caught.

PiperOrigin-RevId: 268166291
2019-09-10 00:38:52 -07:00
Ian Gudger 9dfcd8b09f Fix ephemeral port leak.
Fix a bug where udp.(*endpoint).Disconnect [accessible in gVisor via
epsocket.(*SocketOperations).Connect with AF_UNSPEC] would leak a port
reservation if the socket/endpoint had an ephemeral port assigned to it.

glibc's getaddrinfo uses connect with AF_UNSPEC, causing each call of
getaddrinfo to leak a port. Call getaddrinfo too many times and you run out of
ports (shows up as connect returning EAGAIN and getaddrinfo returning
EAI_NONAME "Name or service not known").

PiperOrigin-RevId: 268071160
2019-09-09 14:02:00 -07:00
Rahat Mahmood 3733b9b893 go_marshal: Implement automatic generation of ABI marshalling code.
This CL implements go_marshal, a code generation utility for
automatically serializing and deserializing ABI structs.

The go_marshal tool automatically generates implementations of the new
marshal interface. Unlike binary.Marshal/Unmarshal, the generated
interface implementations use no runtime reflection, and translates to
a single memcpy for most structs. See go_marshal/README.md for
details.

PiperOrigin-RevId: 268065475
2019-09-09 13:36:39 -07:00
gVisor bot 6af9a9850a Merge pull request #824 from amscanne:fix_build2
PiperOrigin-RevId: 268062329
2019-09-09 13:21:41 -07:00
Ghanan Gowripalan a8943325db Join IPv6 all-nodes and solicited-node multicast addresses where appropriate.
The IPv6 all-nodes multicast address will be joined on NIC enable, and the
appropriate IPv6 solicited-node multicast address will be joined when IPv6
addresses are added.

Tests: Test receiving packets destined to the IPv6 link-local all-nodes
multicast address and the IPv6 solicted node address of an added IPv6 address.
PiperOrigin-RevId: 268047073
2019-09-09 12:06:06 -07:00
Ian Gudger fe1f521077 Remove reundant global tcpip.LinkEndpointID.
PiperOrigin-RevId: 267709597
2019-09-06 18:01:14 -07:00
Jamie Liu 9e1cbdf565 Indicate flipcall synchronization to the Go race detector.
Since each Endpoint has a distinct mapping of the packet window, the Go race
detector does not recognize accesses by connected Endpoints to be related. This
means that this change isn't necessary for the Go race detector to accept
accesses of flipcall.Endpoint.Data(), but it *is* necessary for it to accept
accesses to shared variables outside the scope of flipcall that are
synchronized by flipcall.Endpoint state; see updated test for an example.

RaceReleaseMerge is needed (instead of RaceRelease) because calls to
raceBecomeInactive() from *unrelated* Endpoints can occur in any order.
(DowngradableRWMutex.RUnlock() has a similar property: calls to RUnlock() on
the same DowngradableRWMutex from different goroutines can occur in any order.
Remove the TODO asking to explain this now that this is understood.)

PiperOrigin-RevId: 267705325
2019-09-06 17:25:07 -07:00
Adin Scannell 1a0a940587 Fix repository build scripts
This has the following fixes:

* Packages are passed to the tools/make_repository.sh command.
* All matching tags are built, for commits with multiple.
* The binary path is generated by the build command.
* Output from signing the repository is supressed.
* Allow a release author.

Change-Id: I2d08954ba76e35612f352be99d5bb99080f80892
2019-09-06 14:28:12 -07:00
Michael Pratt 98f7fbb59f Load C++ rules from @rules_cc
See https://github.com/bazelbuild/bazel/issues/8743. This will be required in
Bazel 1.0.

Protobuf was updated in
bf0c69e130 (diff-96239ee297e0a92ac6ff96a6bc434ef0).

GoogleTest was updated in
6fd262ecf7.

gflags has not yet been updated, so the repo still won't build with
--incompatible_load_cc_rules_from_bzl.

Tested with buildifier -warnings=native-cc -lint=warn **/BUILD.

PiperOrigin-RevId: 267638515
2019-09-06 11:29:00 -07:00
Ian Lewis 0bfffbcb01 Ignore the root container when calculating oom_score_adj for the sandbox.
This is done because the root container for CRI is the infrastructure (pause)
container and always gets a low oom_score_adj. We do this to ensure that only
the oom_score_adj of user containers is used to calculated the sandbox
oom_score_adj.

Implemented in runsc rather than the containerd shim as it's a bit cleaner to
implement here (in the shim it would require overwriting the oomScoreAdj and
re-writing out the config.json again). This processing is Kubernetes(CRI)
specific but we are currently only supporting CRI for multi-container support
anyway.

PiperOrigin-RevId: 267507706
2019-09-05 19:21:25 -07:00
Nicolas Lacasse 7e94f171f4 Better strace logs for statx.
PiperOrigin-RevId: 267498537
2019-09-05 18:03:53 -07:00
gVisor bot 487c6f0344 Merge pull request #809 from google:amscanne-patch-1
PiperOrigin-RevId: 267484802
2019-09-05 16:46:15 -07:00
Bhasker Hariharan eb074a61f2 Fix bug in proc_test.
TestNoDuplicates is racy as it tries to read the /proc file system
while the test is running. But it's possible that from the time a
directory entries are read and each entry processed something could
change and in some cases the entry being processed could have been
deleted. In such cases we should not fail the test but just
ignore the error and move on.

PiperOrigin-RevId: 267483094
2019-09-05 16:40:46 -07:00
Jamie Liu fbdd3ff1da Deflake aio_test.
- Most AIO tests call io_setup(nr_events = 128). sizeof(struct io_event)
(128*32 = 4096). However, the actual size of the mapping created by
io_setup() is determined by:

(from fs/aio.c:ioctx_alloc())
/*
 * We keep track of the number of available ringbuffer slots, to prevent
 * overflow (reqs_available), and we also use percpu counters for this.
 *
 * So since up to half the slots might be on other cpu's percpu counters
 * and unavailable, double nr_events so userspace sees what they
 * expected: additionally, we move req_batch slots to/from percpu
 * counters at a time, so make sure that isn't 0:
 */
nr_events = max(nr_events, num_possible_cpus() * 4);
nr_events *= 2;

(from fs/aio.c:aio_setup_ring())
/* Compensate for the ring buffer's head/tail overlap entry */
nr_events += 2; /* 1 is required, 2 for good luck */
size = sizeof(struct aio_ring);
size += sizeof(struct io_event) * nr_events;
nr_pages = PFN_UP(size);

When we mremap() only the first page of a multi-page AIO ring buffer
mapping, fs/aio.c:aio_ring_mremap() updates struct kioctx::mmap_base -
but struct kioctx::mmap_size is untouched, so sys_io_destroy() =>
kill_ioctx() vm_unmaps() the mremapped page, plus some number of pages
after it. Just get the actual size of the mapping from /proc/self/maps.

- Delete test case MremapOver; while it is correct that Linux will not
complain if you overwrite the AIO ring buffer with another mapping, it
won't actually work in the sense that AIO events will not be written to
the new mapping, because Linux stores the struct pages of the ring
buffer in struct kioctx::ring_pages and writes to those through kmap()
rather than using userspace addresses.

- Don't munmap() after mremap(MREMAP_FIXED) returns EFAULT; see new
comment in factored-out test case MremapExpansion.

PiperOrigin-RevId: 267482903
2019-09-05 16:36:44 -07:00
Robert Tonic c2ae77a607 Apply go fmt to the fsgofer changes. 2019-09-05 17:12:09 -04:00
Robert Tonic 4288a57883 Remove seccomp permissions, and clean up the Attach logic. 2019-09-05 15:26:16 -04:00
Robert Tonic 4573efe84b Switch from net to unet to open Unix Domain Sockets. 2019-09-05 07:16:36 -04:00
Bin Lu a25a97624c Change syscall.Dup2 to syscall.Dup3
Signed-off-by: Bin Lu <bin.lu@arm.com>
2019-09-05 07:51:12 +00:00
Ian Lewis e31686500d Allow non-unique group IDs in bazel docker containers
Allow non-unique group IDs in the bazel docker container in order to avoid
failures using host group IDs that are already present in the image.

Issue #801

PiperOrigin-RevId: 267306089
2019-09-04 23:20:12 -07:00
Adin Scannell 91518fd553
Fix build when no tags are present
This should correct the continuous build.
2019-09-04 22:24:42 -07:00
Ian Gudger fbbb2f7ed6 Run proc_net tests.
PiperOrigin-RevId: 267280086
2019-09-04 19:08:12 -07:00
Fabricio Voznika 0f5cdc1e00 Resolve flakes with TestMultiContainerDestroy
Some processes are reparented to the root container depending
on the kill order and the root container would not reap in time.
So some zombie processes were still present when the test checked.

Fix it by running the second container inside a PID namespace.

PiperOrigin-RevId: 267278591
2019-09-04 18:56:49 -07:00
Adin Scannell bcddd0a477 Fix continuous build breakage
PiperOrigin-RevId: 267277711
2019-09-04 18:49:19 -07:00
Bhasker Hariharan 3dc3cffb2d Fix RST generation bugs.
There are a few cases addressed by this change

- We no longer generate a RST in response to a RST packet.

- When we receive a RST we cleanup and release all reservations immediately as
  the connection is now aborted.

- An ACK received by a listening socket generates a RST when SYN cookies are not
  in-use. The only reason an ACK should land at the listening socket is if we
  are using SYN cookies otherwise the goroutine for the handshake in progress
  should have gotten the packet and it should never have arrived at the
  listening endpoint.

- Also fixes the error returned when a connection times out due to a
  Keepalive timer expiration from ECONNRESET to a ETIMEDOUT.

PiperOrigin-RevId: 267238427
2019-09-04 14:59:53 -07:00
Chris Kuiper 7bf1d426d5 Handle subnet and broadcast addresses correctly with NIC.subnets
This also renames "subnet" to "addressRange" to avoid any more confusion with
an interface IP's subnet.

Lastly, this also removes the Stack.ContainsSubnet(..) API since it isn't used
by anyone. Plus the same information can be obtained from
Stack.NICAddressRanges().

PiperOrigin-RevId: 267229843
2019-09-04 14:19:32 -07:00
Adin Scannell 67a2ab1438 Impose order on test scripts.
The simple test script has gotten out of control. Shard this script into
different pieces and attempt to impose order on overall test structure. This
change helps lay some of the foundations for future improvements.

 * The runsc/test directories are moved into just test/.
 * The runsc/test/testutil package is split into logical pieces.
 * The scripts/ directory contains new top-level targets.
 * Each test is now responsible for building targets it requires.
 * The install functionality is moved into `runsc` itself for simplicity.
 * The existing kokoro run_tests.sh file now just calls all (can be split).

After this change is merged,  I will create multiple distinct workflows for
Kokoro, one for each of the scripts currently targeted by `run_tests.sh` today,
which should dramatically reduce the time-to-run for the Kokoro tests, and
provides a better foundation for further improvements to the infrastructure.

PiperOrigin-RevId: 267081397
2019-09-03 22:02:43 -07:00
Ghanan Gowripalan 144127e5e1 Validate IPv6 Hop Limit field for received NDP packets
Make sure that NDP packets are only received if their IP header's hop limit
field is set to 255, as per RFC 4861.

PiperOrigin-RevId: 267061457
2019-09-03 18:43:12 -07:00
Bhasker Hariharan 3789c34b22 Make UDP traceroute work.
Adds support to generate Port Unreachable messages for UDP
datagrams received on a port for which there is no valid
endpoint.

Fixes #703

PiperOrigin-RevId: 267034418
2019-09-03 16:01:17 -07:00
Jamie Liu eb94066ef2 Ensure that flipcall.Endpoint.Shutdown() shuts down inactive peers.
PiperOrigin-RevId: 267022978
2019-09-03 15:10:51 -07:00
gVisor bot 648170f527 Merge pull request #782 from xiaobo55x:pkg_rawfile
PiperOrigin-RevId: 266976315
2019-09-03 11:31:57 -07:00
Haibo Xu fa151e3971 Remove duplicated file in pkg/tcpip/link/rawfile.
The blockingpoll_unsafe.go was copied to blockingpoll_noyield_unsafe.go
during merging commit 7206202bb9. If it still stay here, it would
cause build errors on non-amd64 platform.

ERROR:
pkg/tcpip/link/rawfile/BUILD:5:1:
GoCompilePkg
pkg/tcpip/link/rawfile.a
failed (Exit 1) builder failed: error executing command
bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk
external/go_sdk -installsuffix linux_arm64 -src
pkg/tcpip/link/rawfile/blockingpoll_noyield_unsafe.go -src ...
(remaining 33 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
compilepkg: error running subcommand: exit status 2
pkg/tcpip/link/rawfile/blockingpoll_yield_unsafe.go:35:6:
BlockingPoll redeclared in this block
        previous declaration at
pkg/tcpip/link/rawfile/blockingpoll_unsafe.go:26:78
Target //pkg/tcpip/link/rawfile:rawfile failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 25.531s, Critical Path: 21.08s
INFO: 262 processes: 262 linux-sandbox.
FAILED: Build did NOT complete successfully

Signed-off-by: Haibo Xu <haibo.xu@arm.com>
Change-Id: I4e21f82984225d0aa173de456f7a7c66053a053e
2019-09-02 02:49:41 +00:00
Jamie Liu 0352cf5866 Remove support for non-incremental mapped accounting.
PiperOrigin-RevId: 266496644
2019-08-30 19:06:55 -07:00
Bhasker Hariharan 54bf2e8eff Automated rollback of changelist 261387276
PiperOrigin-RevId: 266491264
2019-08-30 18:15:32 -07:00
Jamie Liu f3dabdfc48 Fix async-signal-unsafety in MlockallTest_Future.
PiperOrigin-RevId: 266491246
2019-08-30 18:11:15 -07:00
Chris Kuiper afbdf2f212 Fix data race accessing referencedNetworkEndpoint.kind
Wrapping "kind" into atomic access functions.

Fixes #789

PiperOrigin-RevId: 266485501
2019-08-30 17:23:53 -07:00
Fabricio Voznika 502c47f7a7 Return correct buffer size for ioctl(socket, FIONREAD)
Ioctl was returning just the buffer size from epsocket.endpoint
and it was not considering data from epsocket.SocketOperations
that was read from the endpoint, but not yet sent to the caller.

PiperOrigin-RevId: 266485461
2019-08-30 17:19:09 -07:00
Adin Scannell 3ec0b64d8c Update image to install docker and fix permissions
PiperOrigin-RevId: 266467361
2019-08-30 15:27:47 -07:00
Adin Scannell 888e87909e Add C++ toolchain and fix compile issues.
This was accidentally introduced in 31f05d5d4f.

Fixes #788.

PiperOrigin-RevId: 266462843
2019-08-30 15:03:15 -07:00
Rahat Mahmood f74affe203 Handle new representation of abstract UDS paths.
When abstract unix domain socket paths are displayed in
/proc/net/unix, Linux historically emitted null bytes as padding at
the end of the path. Newer versions of Linux (v4.9,
e7947ea770d0de434d38a0f823e660d3fd4bebb5) display these as '@'
characters.

Update proc_net_unix test to handle both version of the padding.

PiperOrigin-RevId: 266230200
2019-08-29 14:37:47 -07:00