Commit Graph

848 Commits

Author SHA1 Message Date
Nicolas Lacasse d2f8586a1f Kokoro: Build all packages as part of presubmit.
We now build all packages (including //test/...) with RBE as part of the Kokoro
presubmit.

The tests do not yet use RBE, because there are some failures. The Golang unit,
integration, and image tests still run locally.

The syscall test suite needs even more work to make it pass on RBE. Those will
be enabled in follow-up CLs. They currently are not enabled at all on Kokoro.

PiperOrigin-RevId: 226562208
Change-Id: Idd2b81b3e8f07bf300c77e68990493ba97d16e23
2018-12-21 17:13:34 -08:00
Nicolas Lacasse 7a3b4b7bdd Configure Kokoro build with RBE.
PiperOrigin-RevId: 226552586
Change-Id: I078502d46e2e4a8252ef0477309af8d3ffbbf88d
2018-12-21 15:42:59 -08:00
Andrei Vagin 89b18f7108 Remove gunit_no_heapcheck from deps
heapcheck works fine now.

PiperOrigin-RevId: 226552252
Change-Id: Ifcf17a377b3d6512a576f54462dd81c1d4c7b17c
2018-12-21 15:40:08 -08:00
Michael Pratt 71f0d5108b Internal Change
PiperOrigin-RevId: 226542979
Change-Id: Ife11ebd0a85b8a63078e6daa71b4a99a82080ac9
2018-12-21 14:29:35 -08:00
Ian Gudger b515556519 Implement SO_KEEPALIVE, TCP_KEEPIDLE, and TCP_KEEPINTVL.
Within gVisor, plumb new socket options to netstack.

Within netstack, fix GetSockOpt and SetSockOpt return value logic.

PiperOrigin-RevId: 226532229
Change-Id: If40734e119eed633335f40b4c26facbebc791c74
2018-12-21 13:13:45 -08:00
Michael Pratt 0e9b324856 Fix test util tests
We actually need to include test main!

PiperOrigin-RevId: 226524447
Change-Id: I9d9e631611183c7c1fbae1c20a222c9aeef269b2
2018-12-21 12:15:21 -08:00
Fabricio Voznika 1679ef31ef inotify notifies watchers when control events bit are set
The code that matches the event being published with events watchers
was wronly matching all watchers in case any of the control event bits
were set.

Issue #121

PiperOrigin-RevId: 226521230
Change-Id: Ie2c42bc4366faaf59fbf80a74e9297499bd93f9e
2018-12-21 11:54:02 -08:00
Michael Pratt 5c7f70a07d Update rules_go, gazelle, and Go
PiperOrigin-RevId: 226511383
Change-Id: I3a42572d44582d5713841799ec1fd9695d050938
2018-12-21 10:48:56 -08:00
Jamie Liu 9a442fa4b5 Automated rollback of changelist 226224230
PiperOrigin-RevId: 226493053
Change-Id: Ia98d1cb6dd0682049e4d907ef69619831de5c34a
2018-12-21 08:23:34 -08:00
Nicolas Lacasse 8ba450363f Deflake gofer_test.
We must wait for all lazy resources to be released before closing the rootFile.

PiperOrigin-RevId: 226419499
Change-Id: I1d4d961a92b3816e02690cf3eaf0a88944d730cc
2018-12-20 17:23:26 -08:00
Ian Gudger f6274804e1 Make read and write respect SO_RCVTIMEO and SO_SNDTIMEO
PiperOrigin-RevId: 226387521
Change-Id: I0579ab262320fde6c72d2994dd38437f01a99ea5
2018-12-20 13:48:52 -08:00
Jamie Liu 194ef586fc Rename limits.MemoryPagesLocked to limits.MemoryLocked.
"RLIMIT_MEMLOCK: This is the maximum number of bytes of memory that may
be locked into RAM." - getrlimit(2)

PiperOrigin-RevId: 226384346
Change-Id: Iefac4a1bb69f7714dc813b5b871226a8344dc800
2018-12-20 13:28:46 -08:00
Zhaozhong Ni 25d62850ce test: deflake socket_inet_loopback.
PiperOrigin-RevId: 226350590
Change-Id: Idff080705f644f8f78ce92b53c77eecc37e002e7
2018-12-20 09:52:27 -08:00
Googler 86c9bd2547 Automated rollback of changelist 225861605
PiperOrigin-RevId: 226224230
Change-Id: Id24c7d3733722fd41d5fe74ef64e0ce8c68f0b12
2018-12-19 13:30:08 -08:00
Zach Koopmans ff7178a4d1 Implement pwritev2.
Implement pwritev2 and associated unit tests.
Clean up preadv2 unit tests.
Tag RWF_ flags in both preadv2 and pwritev2 with associated bug tickets.

PiperOrigin-RevId: 226222119
Change-Id: Ieb22672418812894ba114bbc88e67f1dd50de620
2018-12-19 13:16:06 -08:00
Jamie Liu 898838e34d Fix mremap expansion with mm.checkInvariants = true.
Also remove useless RSS changes in mm.movePMAsLocked().

PiperOrigin-RevId: 226052996
Change-Id: If59fd259b93238fb2f15c1c8ebfeda14cb590a87
2018-12-18 13:50:33 -08:00
Jamie Liu 0759f3afee Allow PosixErrorOr<T> to be implicitly constructible from copyable T.
PiperOrigin-RevId: 226037787
Change-Id: I5fb5f55f68b67dd86bbff46349b46e3e8e6b9d1b
2018-12-18 12:16:25 -08:00
Jamie Liu 3b3f026278 Truncate ar before calling mm.breakCopyOnWriteLocked().
... as required by the latter's precondition.

PiperOrigin-RevId: 226033824
Change-Id: I6bc46d0e100c61cc58cb5fc69e70c4ca905cd92d
2018-12-18 11:52:31 -08:00
Googler 41903586fd Deflake socket tests
PiperOrigin-RevId: 226029237
Change-Id: I6fcdb241d635468beb6d6c173f2e7f96d715bf26
2018-12-18 11:27:36 -08:00
Fabricio Voznika 03226cd950 Add BPFAction type with Stringer
PiperOrigin-RevId: 226018694
Change-Id: I98965e26fe565f37e98e5df5f997363ab273c91b
2018-12-18 10:28:28 -08:00
Jamie Liu e7b47844d9 Correctly handle filenames containing spaces in ParseProcMapsLine.
PiperOrigin-RevId: 225992500
Change-Id: Icc8b1675f1cb625fc5e8ef7389beb42fa7bfaa13
2018-12-18 07:23:53 -08:00
Ian Gudger 12c7430a01 Fix recv blocking for connectionless Unix sockets.
Connectionless Unix sockets (DGRAM Unix sockets created with the socket system
call) inherently only have a read queue. They do not establish bidirectional
connections, instead, the connect system call only sets a default send
location. Writes give the data to the other endpoint which has its own read
queue.

To simplify the code, connectionless Unix sockets still get read and write
queues, but the write queue is a dummy and never waited on. The read queue is
the connectionless endpoint's queue. This change fixes a bug where the dummy
queue was incorrectly set as the read queue and the endpoint's queue was
incorrectly set as the write queue. This meant that read notifications went
to the dummy queue and were black holed.

PiperOrigin-RevId: 225921042
Change-Id: I8d9059def787a2c3c305185b92d05093fbd2be2a
2018-12-17 17:53:22 -08:00
Michael Pratt b62591e6a8 Expose internal testing flag
Never to used outside of runsc tests!

PiperOrigin-RevId: 225919013
Change-Id: Ib3b14aa2a2564b5246fb3f8933d95e01027ed186
2018-12-17 17:35:06 -08:00
Rahat Mahmood e9c4f69f36 Enable recently-fixed test case for shm_test.
PiperOrigin-RevId: 225887356
Change-Id: Iee000dcfc9e0168c2566edf41c66108be9b68cd6
2018-12-17 14:12:24 -08:00
Nicolas Lacasse faabaad5a2 Turn off kokoro key-fetching until the key is available.
PiperOrigin-RevId: 225883292
Change-Id: I7e7f139d5865914c51866ea1d23220fcf221748c
2018-12-17 13:52:45 -08:00
Nicolas Lacasse d3ae74d2a5 overlayBoundEndpoint must be recursive if there is an overlay in the lower.
The old overlayBoundEndpoint assumed that the lower is not an overlay.  It
should check if the lower is an overlay and handle that case.

PiperOrigin-RevId: 225882303
Change-Id: I60660c587d91db2826e0719da0983ec8ad024cb8
2018-12-17 13:46:57 -08:00
Nicolas Lacasse f7e8dc57c5 Refactor kokoro/run_tests.sh
This will make it easier to add RBE to bazel.

PiperOrigin-RevId: 225865250
Change-Id: I530b5e09875267c18dc6e7e16590fe9e128253ac
2018-12-17 12:00:30 -08:00
Jamie Liu 2421006426 Implement mlock(), kind of.
Currently mlock() and friends do nothing whatsoever. However, mlocking
is directly application-visible in a number of ways; for example,
madvise(MADV_DONTNEED) and msync(MS_INVALIDATE) both fail on mlocked
regions. We handle this inconsistently: MADV_DONTNEED is too important
to not work, but MS_INVALIDATE is rejected.

Change MM to track mlocked regions in a manner consistent with Linux.
It still will not actually pin pages into host physical memory, but:

- mlock() will now cause sentry memory management to precommit mlocked
pages.

- MADV_DONTNEED and MS_INVALIDATE will interact with mlocked pages as
described above.

PiperOrigin-RevId: 225861605
Change-Id: Iee187204979ac9a4d15d0e037c152c0902c8d0ee
2018-12-17 11:38:59 -08:00
Nicolas Lacasse 54694086df Use more descriptive service account key name.
PiperOrigin-RevId: 225854218
Change-Id: I7843e9e04c58a87d3a7e20d43e64c677ad9c961b
2018-12-17 10:59:28 -08:00
Chris Kuiper e491ebbacf Allow sending of multicast and IPv6 link-local packets w/o route.
Same as with broadcast packets, sending of a multicast packet shouldn't require
accessing the route table. The same applies to IPv6 link-local addresses, which
aren't routable at all (they don't belong to any subnet by definition).

PiperOrigin-RevId: 225775870
Change-Id: Ic53e6560c125a83be2be9c3d112e66b36e8dfe7b
2018-12-16 23:05:59 -08:00
Ian Gudger f74eed464b Add blocking recv tests
PiperOrigin-RevId: 225646045
Change-Id: Ic712ebc627587ef4a9486f0b39fe8c96100f10ff
2018-12-14 22:11:26 -08:00
Nicolas Lacasse 13e4930948 Use containerd at HEAD until better tagged version is available.
Go 1.11.3 has a bug:
https://github.com/golang/go/issues/29241

This is fixed/workarounded in containerd:
52de371700

Until that commit has made it into a tagged version, we will use containerd at
head.

PiperOrigin-RevId: 225636987
Change-Id: I7e32beb7751f566f5b41682a29a14442c1aa56c2
2018-12-14 19:15:01 -08:00
Anne Bertucio 02f86d74f3 Adds Code of Conduct
Adds a code of conduct to the gVisor project that follows
the Google Open Source Office recommended CoC.

Change-Id: Iec85e93991765177aa681c4179ed0e4f2eb7b4d5
PiperOrigin-RevId: 225633127
2018-12-14 18:13:52 -08:00
Adin Scannell 5d8cf31346 Move fdnotifier package to reduce internal confusion.
PiperOrigin-RevId: 225632398
Change-Id: I909e7e2925aa369adc28e844c284d9a6108e85ce
2018-12-14 18:05:01 -08:00
Andrei Vagin 3cf84e3bef Mark sync.Mutex in TTYFileOperations as nosave
PiperOrigin-RevId: 225621767
Change-Id: Ie3a42cdf0b0de22a020ff43e307bf86409cff329
2018-12-14 16:24:21 -08:00
Ian Gudger e1dcf92ec5 Implement SO_SNDTIMEO
PiperOrigin-RevId: 225620490
Change-Id: Ia726107b3f58093a5f881634f90b071b32d2c269
2018-12-14 16:15:06 -08:00
Nicolas Lacasse ed930354ef Give Kokoro access to RBE service.
PiperOrigin-RevId: 225599728
Change-Id: I70cd9f9d7375e56ae8d0a531ad4efb41418e7402
2018-12-14 16:14:02 -08:00
Nicolas Lacasse cd1e9a3fd4 Shard the syscall tests.
PiperOrigin-RevId: 225574278
Change-Id: If5060a37e8a9b0120bec2b5de4037354f0eaba16
2018-12-14 11:25:36 -08:00
Nicolas Lacasse 5301cbf843 Bump rules_go to v0.16.4 and go toolchain to v1.11.3.
PiperOrigin-RevId: 225465835
Change-Id: Iee467e493e5df0a9e149b131f54b0af84d221051
2018-12-13 17:37:55 -08:00
Nicolas Lacasse 8f18b69cb1 Remove unused variable.
PiperOrigin-RevId: 225455503
Change-Id: I327fc6e7ba26532b628f343dece3bd9fc4d3b524
2018-12-13 16:21:41 -08:00
Nicolas Lacasse 1775a0e11e container.Destroy should clean up container metadata even if other cleanups fail
If the sandbox process is dead (because of a panic or some other problem),
container.Destroy will never remove the container metadata file, since it will
always fail when calling container.stop().

This CL changes container.Destroy() to always perform the three necessary
cleanup operations:
* Stop the sandbox and gofer processes.
* Remove the container fs on the host.
* Delete the container metadata directory.

Errors from these three operations will be concatenated and returned from
Destroy().

PiperOrigin-RevId: 225448164
Change-Id: I99c6311b2e4fe5f6e2ca991424edf1ebeae9df32
2018-12-13 15:38:10 -08:00
Rahat Mahmood a0c8aeb73d Clean up shm segments created by shm_test.
This test suite was creating shm segments without ensuring they were
cleaned up. Shm segments outlive the process creating them, so on a
standard linux machine the test was leaving segments behind after each
run. This would often cause failures as test cases would be affected
by the cases that ran before them and left unexpected segments lying
around.

Also skip some assertions around memory usage when running on a Linux
host, as we can't reason about external users of shm segments.

PiperOrigin-RevId: 225435523
Change-Id: Ia299dacf59045002436f5e30dcc131f679bb7272
2018-12-13 14:24:51 -08:00
Ian Gudger 4659f7ed1a Fix WAITALL and RCVTIMEO interaction
PiperOrigin-RevId: 225424296
Change-Id: I60fcc2b859339dca9963cb32227a287e719ab765
2018-12-13 13:20:46 -08:00
Ian Gudger 6253d32cc9 transport/tcp: remove unused error return values
PiperOrigin-RevId: 225421480
Change-Id: I1e9259b0b7e8490164e830b73338a615129c7f0e
2018-12-13 13:02:49 -08:00
Ian Gudger f484b6d4c2 Flush socket test description
This ensures that we know what type of socket caused a timeout.

PiperOrigin-RevId: 225294255
Change-Id: I9033bd0f3791d3b5714aa08d111cf58a3014d252
2018-12-12 18:41:14 -08:00
Rahat Mahmood ccce1d4281 Filesystems shouldn't be saving references to Platform.
Platform objects are not savable, storing references to them in
filesystem datastructures would cause save to fail if someone actually
passed in a Platform.

Current implementations work because everywhere a Platform is
expected, we currently pass in a Kernel object which embeds Platform
and thus satisfies the interface.

Eliminate this indirection and save pointers to Kernel directly.

PiperOrigin-RevId: 225288336
Change-Id: Ica399ff43f425e15bc150a0d7102196c3d54a2ab
2018-12-12 17:47:55 -08:00
Rahat Mahmood f93c288dd7 Fix a data race on Shm.key.
PiperOrigin-RevId: 225240907
Change-Id: Ie568ce3cd643f3e4a0eaa0444f4ed589dcf6031f
2018-12-12 13:18:48 -08:00
Rahat Mahmood 75e39eaa74 Pass information about map writableness to filesystems.
This is necessary to implement file seals for memfds.

PiperOrigin-RevId: 225239394
Change-Id: Ib3f1ab31385afc4b24e96cd81a05ef1bebbcbb70
2018-12-12 13:09:59 -08:00
Jamie Liu a2c868a098 Add rvalue ref-qualified PosixErrorOr<T>::ValueOrDie() overloads.
This allows ValueOrDie() to be called on PosixErrorOr rvalues (e.g.
temporaries) holding move-only types without extraneous std::move()s.

PiperOrigin-RevId: 225098036
Change-Id: I662862e4f3562141f941845fc6e197edb27ce29b
2018-12-11 17:05:53 -08:00
Michael Pratt 24c1158b9c Add "trace signal" option
This option is effectively equivalent to -panic-signal, except that the
sandbox does not die after logging the traceback.

PiperOrigin-RevId: 225089593
Change-Id: Ifb1c411210110b6104613f404334bd02175e484e
2018-12-11 16:12:41 -08:00