Commit Graph

1685 Commits

Author SHA1 Message Date
Ian Gudger fe1f521077 Remove reundant global tcpip.LinkEndpointID.
PiperOrigin-RevId: 267709597
2019-09-06 18:01:14 -07:00
Jamie Liu 9e1cbdf565 Indicate flipcall synchronization to the Go race detector.
Since each Endpoint has a distinct mapping of the packet window, the Go race
detector does not recognize accesses by connected Endpoints to be related. This
means that this change isn't necessary for the Go race detector to accept
accesses of flipcall.Endpoint.Data(), but it *is* necessary for it to accept
accesses to shared variables outside the scope of flipcall that are
synchronized by flipcall.Endpoint state; see updated test for an example.

RaceReleaseMerge is needed (instead of RaceRelease) because calls to
raceBecomeInactive() from *unrelated* Endpoints can occur in any order.
(DowngradableRWMutex.RUnlock() has a similar property: calls to RUnlock() on
the same DowngradableRWMutex from different goroutines can occur in any order.
Remove the TODO asking to explain this now that this is understood.)

PiperOrigin-RevId: 267705325
2019-09-06 17:25:07 -07:00
Michael Pratt 98f7fbb59f Load C++ rules from @rules_cc
See https://github.com/bazelbuild/bazel/issues/8743. This will be required in
Bazel 1.0.

Protobuf was updated in
bf0c69e130 (diff-96239ee297e0a92ac6ff96a6bc434ef0).

GoogleTest was updated in
6fd262ecf7.

gflags has not yet been updated, so the repo still won't build with
--incompatible_load_cc_rules_from_bzl.

Tested with buildifier -warnings=native-cc -lint=warn **/BUILD.

PiperOrigin-RevId: 267638515
2019-09-06 11:29:00 -07:00
Ian Lewis 0bfffbcb01 Ignore the root container when calculating oom_score_adj for the sandbox.
This is done because the root container for CRI is the infrastructure (pause)
container and always gets a low oom_score_adj. We do this to ensure that only
the oom_score_adj of user containers is used to calculated the sandbox
oom_score_adj.

Implemented in runsc rather than the containerd shim as it's a bit cleaner to
implement here (in the shim it would require overwriting the oomScoreAdj and
re-writing out the config.json again). This processing is Kubernetes(CRI)
specific but we are currently only supporting CRI for multi-container support
anyway.

PiperOrigin-RevId: 267507706
2019-09-05 19:21:25 -07:00
Nicolas Lacasse 7e94f171f4 Better strace logs for statx.
PiperOrigin-RevId: 267498537
2019-09-05 18:03:53 -07:00
gVisor bot 487c6f0344 Merge pull request #809 from google:amscanne-patch-1
PiperOrigin-RevId: 267484802
2019-09-05 16:46:15 -07:00
Bhasker Hariharan eb074a61f2 Fix bug in proc_test.
TestNoDuplicates is racy as it tries to read the /proc file system
while the test is running. But it's possible that from the time a
directory entries are read and each entry processed something could
change and in some cases the entry being processed could have been
deleted. In such cases we should not fail the test but just
ignore the error and move on.

PiperOrigin-RevId: 267483094
2019-09-05 16:40:46 -07:00
Jamie Liu fbdd3ff1da Deflake aio_test.
- Most AIO tests call io_setup(nr_events = 128). sizeof(struct io_event)
(128*32 = 4096). However, the actual size of the mapping created by
io_setup() is determined by:

(from fs/aio.c:ioctx_alloc())
/*
 * We keep track of the number of available ringbuffer slots, to prevent
 * overflow (reqs_available), and we also use percpu counters for this.
 *
 * So since up to half the slots might be on other cpu's percpu counters
 * and unavailable, double nr_events so userspace sees what they
 * expected: additionally, we move req_batch slots to/from percpu
 * counters at a time, so make sure that isn't 0:
 */
nr_events = max(nr_events, num_possible_cpus() * 4);
nr_events *= 2;

(from fs/aio.c:aio_setup_ring())
/* Compensate for the ring buffer's head/tail overlap entry */
nr_events += 2; /* 1 is required, 2 for good luck */
size = sizeof(struct aio_ring);
size += sizeof(struct io_event) * nr_events;
nr_pages = PFN_UP(size);

When we mremap() only the first page of a multi-page AIO ring buffer
mapping, fs/aio.c:aio_ring_mremap() updates struct kioctx::mmap_base -
but struct kioctx::mmap_size is untouched, so sys_io_destroy() =>
kill_ioctx() vm_unmaps() the mremapped page, plus some number of pages
after it. Just get the actual size of the mapping from /proc/self/maps.

- Delete test case MremapOver; while it is correct that Linux will not
complain if you overwrite the AIO ring buffer with another mapping, it
won't actually work in the sense that AIO events will not be written to
the new mapping, because Linux stores the struct pages of the ring
buffer in struct kioctx::ring_pages and writes to those through kmap()
rather than using userspace addresses.

- Don't munmap() after mremap(MREMAP_FIXED) returns EFAULT; see new
comment in factored-out test case MremapExpansion.

PiperOrigin-RevId: 267482903
2019-09-05 16:36:44 -07:00
Ian Lewis e31686500d Allow non-unique group IDs in bazel docker containers
Allow non-unique group IDs in the bazel docker container in order to avoid
failures using host group IDs that are already present in the image.

Issue #801

PiperOrigin-RevId: 267306089
2019-09-04 23:20:12 -07:00
Adin Scannell 91518fd553
Fix build when no tags are present
This should correct the continuous build.
2019-09-04 22:24:42 -07:00
Ian Gudger fbbb2f7ed6 Run proc_net tests.
PiperOrigin-RevId: 267280086
2019-09-04 19:08:12 -07:00
Fabricio Voznika 0f5cdc1e00 Resolve flakes with TestMultiContainerDestroy
Some processes are reparented to the root container depending
on the kill order and the root container would not reap in time.
So some zombie processes were still present when the test checked.

Fix it by running the second container inside a PID namespace.

PiperOrigin-RevId: 267278591
2019-09-04 18:56:49 -07:00
Adin Scannell bcddd0a477 Fix continuous build breakage
PiperOrigin-RevId: 267277711
2019-09-04 18:49:19 -07:00
Bhasker Hariharan 3dc3cffb2d Fix RST generation bugs.
There are a few cases addressed by this change

- We no longer generate a RST in response to a RST packet.

- When we receive a RST we cleanup and release all reservations immediately as
  the connection is now aborted.

- An ACK received by a listening socket generates a RST when SYN cookies are not
  in-use. The only reason an ACK should land at the listening socket is if we
  are using SYN cookies otherwise the goroutine for the handshake in progress
  should have gotten the packet and it should never have arrived at the
  listening endpoint.

- Also fixes the error returned when a connection times out due to a
  Keepalive timer expiration from ECONNRESET to a ETIMEDOUT.

PiperOrigin-RevId: 267238427
2019-09-04 14:59:53 -07:00
Chris Kuiper 7bf1d426d5 Handle subnet and broadcast addresses correctly with NIC.subnets
This also renames "subnet" to "addressRange" to avoid any more confusion with
an interface IP's subnet.

Lastly, this also removes the Stack.ContainsSubnet(..) API since it isn't used
by anyone. Plus the same information can be obtained from
Stack.NICAddressRanges().

PiperOrigin-RevId: 267229843
2019-09-04 14:19:32 -07:00
Adin Scannell 67a2ab1438 Impose order on test scripts.
The simple test script has gotten out of control. Shard this script into
different pieces and attempt to impose order on overall test structure. This
change helps lay some of the foundations for future improvements.

 * The runsc/test directories are moved into just test/.
 * The runsc/test/testutil package is split into logical pieces.
 * The scripts/ directory contains new top-level targets.
 * Each test is now responsible for building targets it requires.
 * The install functionality is moved into `runsc` itself for simplicity.
 * The existing kokoro run_tests.sh file now just calls all (can be split).

After this change is merged,  I will create multiple distinct workflows for
Kokoro, one for each of the scripts currently targeted by `run_tests.sh` today,
which should dramatically reduce the time-to-run for the Kokoro tests, and
provides a better foundation for further improvements to the infrastructure.

PiperOrigin-RevId: 267081397
2019-09-03 22:02:43 -07:00
Ghanan Gowripalan 144127e5e1 Validate IPv6 Hop Limit field for received NDP packets
Make sure that NDP packets are only received if their IP header's hop limit
field is set to 255, as per RFC 4861.

PiperOrigin-RevId: 267061457
2019-09-03 18:43:12 -07:00
Bhasker Hariharan 3789c34b22 Make UDP traceroute work.
Adds support to generate Port Unreachable messages for UDP
datagrams received on a port for which there is no valid
endpoint.

Fixes #703

PiperOrigin-RevId: 267034418
2019-09-03 16:01:17 -07:00
Jamie Liu eb94066ef2 Ensure that flipcall.Endpoint.Shutdown() shuts down inactive peers.
PiperOrigin-RevId: 267022978
2019-09-03 15:10:51 -07:00
gVisor bot 648170f527 Merge pull request #782 from xiaobo55x:pkg_rawfile
PiperOrigin-RevId: 266976315
2019-09-03 11:31:57 -07:00
Haibo Xu fa151e3971 Remove duplicated file in pkg/tcpip/link/rawfile.
The blockingpoll_unsafe.go was copied to blockingpoll_noyield_unsafe.go
during merging commit 7206202bb9. If it still stay here, it would
cause build errors on non-amd64 platform.

ERROR:
pkg/tcpip/link/rawfile/BUILD:5:1:
GoCompilePkg
pkg/tcpip/link/rawfile.a
failed (Exit 1) builder failed: error executing command
bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk
external/go_sdk -installsuffix linux_arm64 -src
pkg/tcpip/link/rawfile/blockingpoll_noyield_unsafe.go -src ...
(remaining 33 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
compilepkg: error running subcommand: exit status 2
pkg/tcpip/link/rawfile/blockingpoll_yield_unsafe.go:35:6:
BlockingPoll redeclared in this block
        previous declaration at
pkg/tcpip/link/rawfile/blockingpoll_unsafe.go:26:78
Target //pkg/tcpip/link/rawfile:rawfile failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 25.531s, Critical Path: 21.08s
INFO: 262 processes: 262 linux-sandbox.
FAILED: Build did NOT complete successfully

Signed-off-by: Haibo Xu <haibo.xu@arm.com>
Change-Id: I4e21f82984225d0aa173de456f7a7c66053a053e
2019-09-02 02:49:41 +00:00
Jamie Liu 0352cf5866 Remove support for non-incremental mapped accounting.
PiperOrigin-RevId: 266496644
2019-08-30 19:06:55 -07:00
Bhasker Hariharan 54bf2e8eff Automated rollback of changelist 261387276
PiperOrigin-RevId: 266491264
2019-08-30 18:15:32 -07:00
Jamie Liu f3dabdfc48 Fix async-signal-unsafety in MlockallTest_Future.
PiperOrigin-RevId: 266491246
2019-08-30 18:11:15 -07:00
Chris Kuiper afbdf2f212 Fix data race accessing referencedNetworkEndpoint.kind
Wrapping "kind" into atomic access functions.

Fixes #789

PiperOrigin-RevId: 266485501
2019-08-30 17:23:53 -07:00
Fabricio Voznika 502c47f7a7 Return correct buffer size for ioctl(socket, FIONREAD)
Ioctl was returning just the buffer size from epsocket.endpoint
and it was not considering data from epsocket.SocketOperations
that was read from the endpoint, but not yet sent to the caller.

PiperOrigin-RevId: 266485461
2019-08-30 17:19:09 -07:00
Adin Scannell 3ec0b64d8c Update image to install docker and fix permissions
PiperOrigin-RevId: 266467361
2019-08-30 15:27:47 -07:00
Adin Scannell 888e87909e Add C++ toolchain and fix compile issues.
This was accidentally introduced in 31f05d5d4f.

Fixes #788.

PiperOrigin-RevId: 266462843
2019-08-30 15:03:15 -07:00
Rahat Mahmood f74affe203 Handle new representation of abstract UDS paths.
When abstract unix domain socket paths are displayed in
/proc/net/unix, Linux historically emitted null bytes as padding at
the end of the path. Newer versions of Linux (v4.9,
e7947ea770d0de434d38a0f823e660d3fd4bebb5) display these as '@'
characters.

Update proc_net_unix test to handle both version of the padding.

PiperOrigin-RevId: 266230200
2019-08-29 14:37:47 -07:00
Rahat Mahmood 863e11ac4d Implement /proc/net/udp.
PiperOrigin-RevId: 266229756
2019-08-29 14:30:41 -07:00
gVisor bot 0789b9cc08 Merge pull request #655 from praveensastry:feature/runsc-ref-chk-leak
PiperOrigin-RevId: 266226714
2019-08-29 14:17:32 -07:00
Nicolas Lacasse eb4aa40342 Compile procter binaries during image creation.
Using "go run ..." in the ENTRYPOINT causes the go compiler to run each time
the container is started. We can just compile the binary once as part of the
image.

PiperOrigin-RevId: 266212462
2019-08-29 14:02:32 -07:00
gVisor bot 31f05d5d4f Internal change.
PiperOrigin-RevId: 266199211
2019-08-29 14:01:47 -07:00
Jamie Liu 36a8949b2a Add limit_host_fd_translation Gofer mount option.
PiperOrigin-RevId: 266177409
2019-08-29 14:01:03 -07:00
Nicolas Lacasse 6e8fc27dc5 Don't log an error when stopping the container if it is not running.
This removes a lot of confusing messages from the test logs.

PiperOrigin-RevId: 266164001
2019-08-29 14:00:18 -07:00
Adin Scannell 599136b25e Internal change.
PiperOrigin-RevId: 266073190
2019-08-29 13:59:33 -07:00
Adin Scannell f048c8be1e Fix permissions on .ssh files
PiperOrigin-RevId: 266066839
2019-08-28 22:36:16 -07:00
Adin Scannell 06ae36185e Fix kokoro image build scripts
The /tmpfs directory needs to be writable, and kokoro needs passwordless sudo
access to execute the test scripts.

PiperOrigin-RevId: 266063723
2019-08-28 22:07:12 -07:00
Tamir Duberstein 24ecce5dbf Export generated linkAddrEntryEntry
PiperOrigin-RevId: 266000128
2019-08-28 14:56:33 -07:00
Adin Scannell 784f48a78d kokoro: Add scripts to rebuild images.
These scripts generated the following images:

gvisor-kokoro-testing/image-a53bac71541a209e (ubuntu 18.04)
gvisor-kokoro-testing/image-f5b20c5fbd23f448 (ubuntu 16.04)

Any modifications to these scripts should generate new images.

PiperOrigin-RevId: 265843929
2019-08-27 23:29:02 -07:00
Tamir Duberstein 313c767b00 Populate link address cache at dispatch
This allows the stack to learn remote link addresses on incoming
packets, reducing the need to ARP to send responses.

This also reduces the number of round trips to the system clock,
since that may also prove to be performance-sensitive.

Fixes #739.

PiperOrigin-RevId: 265815816
2019-08-27 18:54:56 -07:00
Zach Koopmans f64d9a7d93 Fix pwritev2 flaky test.
Fix a uninitialized memory bug in pwritev2 test.

PiperOrigin-RevId: 265772176
2019-08-27 14:50:03 -07:00
Michael Pratt 9679f9891f Fix comment typo
PiperOrigin-RevId: 265731735
2019-08-27 11:44:06 -07:00
Fabricio Voznika 8fd89fd7a2 Fix sendfile(2) error code
When output file is in append mode, sendfile(2) should fail
with EINVAL and not EBADF.

Closes #721

PiperOrigin-RevId: 265718958
2019-08-27 10:52:46 -07:00
Fabricio Voznika c39564332b Mount volumes as super user
This used to be the case, but regressed after a recent change.
Also made a few fixes around it and clean up the code a bit.

Closes #720

PiperOrigin-RevId: 265717496
2019-08-27 10:47:16 -07:00
Adin Scannell b4cdaef4a1 Don't lose test output.
PiperOrigin-RevId: 265606322
2019-08-26 21:36:34 -07:00
Rahat Mahmood 1fdefd41c5 netstack/tcp: Add LastAck transition.
Add missing state transition to LastAck, which should happen when the
endpoint has already recieved a FIN from the remote side, and is
sending its own FIN.

PiperOrigin-RevId: 265568314
2019-08-26 16:39:13 -07:00
Michael Pratt 904b156962 Add support for Intel cache CPUID leafs
This exposes L1, L2, etc. cache sizes, cache line size, etc.

Across S/R, everything except cache line size can differ from the host. This is
because cache line size is critical for correct use of CLFLUSH / CLFLUSHOPT,
but as far as I know, the other cache parameters can only affect performance,
not correctness.

AMD uses different leafs for cache information, which are not yet supported.

fail. There are no known cases of cache line size other than 64 in the fleet.

PiperOrigin-RevId: 265544786
2019-08-26 14:47:05 -07:00
gVisor bot baf4d8aaca Internal change.
PiperOrigin-RevId: 265535438
2019-08-26 14:07:17 -07:00
gVisor bot 7206202bb9 Merge pull request #696 from xiaobo55x:tcpip_link
PiperOrigin-RevId: 265534854
2019-08-26 14:03:30 -07:00