Commit Graph

748 Commits

Author SHA1 Message Date
Zhaozhong Ni 18d8992453 state: pretty-print primitive type arrays.
PiperOrigin-RevId: 201269072
Change-Id: Ia542c5a42b5b5d21c1104a003ddff5279644d309
2018-06-19 17:13:35 -07:00
Kevin Krakauer 33f29c730f runsc: Fix flakey container_test.
Verified that this is no longer flakey over 10K repetitions.

PiperOrigin-RevId: 201267499
Change-Id: I793c916fe725412aec25953f764cb4f52c9fbed3
2018-06-19 17:04:51 -07:00
Adin Scannell be76cad5bc Make KVM more scalable by removing CPU cap.
Instead, CPUs will be created dynamically. We also allow a relatively
efficient mechanism for stealing and notifying when a vCPU becomes
available via unlock.

Since the number of vCPUs is no longer fixed at machine creation time,
we make the dirtySet packing more efficient. This has the pleasant side
effect of cutting out the unsafe address space code.

PiperOrigin-RevId: 201266691
Change-Id: I275c73525a4f38e3714b9ac0fd88731c26adfe66
2018-06-19 17:00:30 -07:00
Zhaozhong Ni aa14a2c1be sentry: futex S/R optimization.
No need to save thousands of zerovalue buckets.

PiperOrigin-RevId: 201258598
Change-Id: I5d3ea7b6a5345117ab4f610332d5288ca550be33
2018-06-19 16:08:00 -07:00
Justine Olshan a6dbef045f Added a resume command to unpause a paused container.
Resume checks the status of the container and unpauses the kernel
if its status is paused. Otherwise nothing happens.
Tests were added to ensure that the process is in the correct state
after various commands.

PiperOrigin-RevId: 201251234
Change-Id: Ifd11b336c33b654fea6238738f864fcf2bf81e19
2018-06-19 15:23:36 -07:00
Brian Geffon bda2a1ed35 Rpcinet is racy around shutdown flags.
Correct a data race in rpcinet where a shutdown and recvmsg can
race around shutown flags.

PiperOrigin-RevId: 201238366
Change-Id: I5eb06df4a2b4eba331eeb5de19076213081d581f
2018-06-19 14:12:52 -07:00
Nicolas Lacasse 9db7cfad93 Add a new cache policy FSCACHE_WRITETHROUGH.
The new policy is identical to FSCACHE (which caches everything in memory), but
it also flushes writes to the backing fs agent immediately.

All gofer cache policy decisions have been moved into the cachePolicy type.
Previously they were sprinkled around the codebase.

There are many different things that we cache (page cache, negative dirents,
dirent LRU, unstable attrs, readdir results....), and I don't think we should
have individual flags to control each of these.  Instead, we should have a few
high-level cache policies that are consistent and useful to users.  This
refactoring makes it easy to add more such policies.

PiperOrigin-RevId: 201206937
Change-Id: I6e225c382b2e5e1b0ad4ccf8ca229873f4cd389d
2018-06-19 11:10:11 -07:00
Zhaozhong Ni 5581256f87 state: include I/O and protobuf time in kernel S/R timing stats.
PiperOrigin-RevId: 201205733
Change-Id: I300307b0668989ba7776ab9e3faee71efdd33f46
2018-06-19 11:04:54 -07:00
Brian Geffon 4fd1d40e1d Rpcinet needs to track shutdown state for blocking sockets.
Because rpcinet will emulate a blocking socket backed by an rpc based
non-blocking socket. In the event of a shutdown(SHUT_RD) followed by a
read a non-blocking socket is allowed to return an EWOULDBLOCK however
since a blocking socket knows it cannot receive anymore data it would
block indefinitely and in this situation linux returns 0. We have to
track this on the rpcinet sentry side so we can emulate that behavior
because the remote side has no way to know if the socket is actually
blocking within the sentry.

PiperOrigin-RevId: 201201618
Change-Id: I4ac3a7b74b5dae471ab97c2e7d33b83f425aedac
2018-06-19 10:43:30 -07:00
Justine Olshan 873ec0c414 Modified boot.go to allow for restores.
A file descriptor was added as a flag to boot so a state file can restore a
container that was checkpointed.

PiperOrigin-RevId: 201068699
Change-Id: I18e96069488ffa3add468861397f3877725544aa
2018-06-18 15:20:36 -07:00
Fabricio Voznika 7eaca1074b Add pointer to dated builds in README
PiperOrigin-RevId: 201068427
Change-Id: If03c8c22907e6ef623f39c8ae5316fdd76cf80cb
2018-06-18 15:18:43 -07:00
Lantao Liu f3727528e5 runsc: support symlink to the exec path.
PiperOrigin-RevId: 201049912
Change-Id: Idd937492217a4c2ca3d59c602e41576a3b203dd9
2018-06-18 13:37:59 -07:00
Lantao Liu 821aaf531d runsc: support "rw" mount option.
PiperOrigin-RevId: 201018483
Change-Id: I52fe3d01c83c8a2f0e9275d9d88c37e46fa224a2
2018-06-18 10:34:11 -07:00
Fabricio Voznika 775982ed4b Automated rollback of changelist 200770591
PiperOrigin-RevId: 201012131
Change-Id: I5cd69e795555129319eb41135ecf26db9a0b1fcb
2018-06-18 10:00:04 -07:00
Brian Geffon 563a71ef24 Add rpcinet support for control messages.
Add support for control messages, but at this time the only
control message that the sentry will support here is SO_TIMESTAMP.

PiperOrigin-RevId: 200922230
Change-Id: I63a852d9305255625d9df1d989bd46a66e93c446
2018-06-17 17:06:40 -07:00
Justine Olshan 0786707cd9 Added code for a pause command for a container process.
Like runc, the pause command will pause the processes of the given container.
It will set that container's status to "paused."
A resume command will be be added to unpause and continue running the process.

PiperOrigin-RevId: 200789624
Change-Id: I72a5d7813d90ecfc4d01cc252d6018855016b1ea
2018-06-15 16:09:09 -07:00
Michael Pratt bd2d1aaa16 Replace crypto/rand with internal rand package
PiperOrigin-RevId: 200784607
Change-Id: I39aa6ee632936dcbb00fc298adccffa606e9f4c0
2018-06-15 15:36:00 -07:00
Kevin Krakauer 437890dc4b runsc: Make gofer logs show up in test output.
PiperOrigin-RevId: 200770591
Change-Id: Ifc096d88615b63135210d93c2b4cee2eaecf1eee
2018-06-15 14:07:54 -07:00
Lantao Liu 2081c5e7f7 runsc: support /dev bind mount which does not conflict with default /dev mount.
PiperOrigin-RevId: 200768923
Change-Id: I4b8da10bcac296e8171fe6754abec5aabfec5e65
2018-06-15 13:58:39 -07:00
Zhaozhong Ni fc8ca72a32 sentry: do not start delivering external signal immediately.
PiperOrigin-RevId: 200765756
Change-Id: Ie4266f32e4e977df3925eb29f3fbb756e0337606
2018-06-15 13:38:14 -07:00
Brian Geffon fa6db05e0c FIFOs should support O_TRUNC as a no-op.
PiperOrigin-RevId: 200759323
Change-Id: I683b2edcc2188304c4ca563e46af457e23625905
2018-06-15 12:55:29 -07:00
Dmitry Vyukov 52110bfc33 runsc/cmd: fix kill signal parsing
Signal is arg 1, not 2.
Killing with SIGABRT is useful to get Go traces.

Signed-off-by: Dmitry Vyukov <dvyukov@google.com>

Change-Id: I0b78e34a9de3fb3385108e26fdb4ff6e9347aeff
PiperOrigin-RevId: 200742743
2018-06-15 11:06:07 -07:00
Adin Scannell b31ac4e1df Use notify explicitly on unlock path.
There are circumstances under which the redpill call will not generate
the appropriate action and notification. Replace this call with an
explicit notification, which is guaranteed to transition as well as
perform the futex wake.

PiperOrigin-RevId: 200726934
Change-Id: Ie19e008a6007692dd7335a31a8b59f0af6e54aaa
2018-06-15 09:30:08 -07:00
Fabricio Voznika ef5dd4df9b Set kernel.applicationCores to the number of processor on the host
The right number to use is the number of processors assigned to the cgroup. But until
we make the sandbox join the respective cgroup, just use the number of processors on
the host.

Closes #65, closes #66

PiperOrigin-RevId: 200725483
Change-Id: I34a566b1a872e26c66f56fa6e3100f42aaf802b1
2018-06-15 09:19:04 -07:00
Fabricio Voznika 119a302ceb Implement /proc/thread-self
Closes #68

PiperOrigin-RevId: 200725401
Change-Id: I4827009b8aee89d22887c3af67291ccf7058d420
2018-06-15 09:18:00 -07:00
Adin Scannell 1eb1bf8670 Update contributing guidelines with an example.
Fixes #69

PiperOrigin-RevId: 200683809
Change-Id: I1312ebb3775d5f9088e9108359c19e2dedbb7b70
2018-06-15 01:22:08 -07:00
Brielle Broder bd1e83ff60 Fix typo.
PiperOrigin-RevId: 200631795
Change-Id: I297fe3e30fb06b04fccd8358c933e45019dcc1fa
2018-06-14 15:45:10 -07:00
Jamie Liu 657db692b2 Ignore expiration count in kernelCPUClockListener.Notify.
PiperOrigin-RevId: 200590832
Change-Id: I35b817ecccc9414a742dee4815dfc67d0c7d0496
2018-06-14 11:35:11 -07:00
Michael Pratt d71f5ef688 Add nanosleep filter for Go 1.11 support
golang.org/cl/108538 replaces pselect6 with nanosleep in runtime.usleep. Update
the filters accordingly.

PiperOrigin-RevId: 200574612
Change-Id: Ifb2296fcb3781518fc047aabbbffedb9ae488cd7
2018-06-14 10:11:05 -07:00
Ian Gudger f5d0c59f5c Fix reference leak in VDSO validation
PiperOrigin-RevId: 200496070
Change-Id: I33adb717c44e5b4bcadece882be3ab1ee3920556
2018-06-13 20:00:55 -07:00
Brian Geffon 1170039e78 Fix missing returns in rpcinet.
PiperOrigin-RevId: 200472634
Change-Id: I3f0fb9e3b2f8616e6aa1569188258f330bf1ed31
2018-06-13 16:21:23 -07:00
Adin Scannell 7b7b199ed0 Deflake kvm_test.
PiperOrigin-RevId: 200439846
Change-Id: I9970fe0716cb02f0f41b754891d55db7e0729f56
2018-06-13 13:05:33 -07:00
Fabricio Voznika 717f2501c9 Fix failure to mount volume that sandbox process has no access
Boot loader tries to stat mount to determine whether it's a file or not. This
may file if the sandbox process doesn't have access to the file. Instead, add
overlay on top of file, which is better anyway since we don't want to propagate
changes to the host.

PiperOrigin-RevId: 200411261
Change-Id: I14222410e8bc00ed037b779a1883d503843ffebb
2018-06-13 10:20:06 -07:00
Zhaozhong Ni 686093669e sentry: do not treat all save errors as state file errors.
PiperOrigin-RevId: 200410220
Change-Id: I6a8745e33be949e335719083501f18b24f6ba471
2018-06-13 10:14:15 -07:00
Jamie Liu 55b9058456 Log filemem state when panicing due to invalid refcount.
PiperOrigin-RevId: 200408305
Change-Id: I676ee49ec77697105723577928c7f82088cd378e
2018-06-13 10:03:54 -07:00
Ian Gudger ba426f7782 Fix reference leak for negative dirents
PiperOrigin-RevId: 200306715
Change-Id: I7c80059c77ebd3d9a5d7d48b05c8e7a597f10850
2018-06-12 17:04:20 -07:00
Brian Geffon c2b3f04d1c Rpcinet doensn't handle SO_RCVTIMEO properly.
Rpcinet already inherits socket.ReceiveTimeout; however, it's
never set on setsockopt(2). The value is currently forwarded
as an RPC and ignored as all sockets will be non-blocking
on the RPC side.

PiperOrigin-RevId: 200299260
Change-Id: I6c610ea22c808ff6420c63759dccfaeab17959dd
2018-06-12 16:16:15 -07:00
Lantao Liu 2506b9b11f runsc: do not include sub target if it is not started with '/'.
PiperOrigin-RevId: 200274828
Change-Id: I956703217df08d8650a881479b7ade8f9f119912
2018-06-12 13:54:54 -07:00
Brielle Broder 711a9869e5 Runsc checkpoint works.
This is the first iteration of checkpoint that actually saves to a file.
Tests for checkpoint are included.

Ran into an issue when private unix sockets are enabled. An error message
was added for this case and the mutex state was set.

PiperOrigin-RevId: 200269470
Change-Id: I28d29a9f92c44bf73dc4a4b12ae0509ee4070e93
2018-06-12 13:25:23 -07:00
Jamie Liu 7a10df454b Drop MMapOpts.MappingIdentity reference in loader.mapSegment.
PiperOrigin-RevId: 200261995
Change-Id: I7e460b18ceab2c23096bdeb7416159d6e774aaf7
2018-06-12 12:38:02 -07:00
Kevin Krakauer 2dc9cd7bf7 runsc: enable terminals in the sandbox.
runsc now mounts the devpts filesystem, so you get a real terminal using
ssh+sshd.

PiperOrigin-RevId: 200244830
Change-Id: If577c805ad0138fda13103210fa47178d8ac6605
2018-06-12 11:03:25 -07:00
Fabricio Voznika 48335318a2 Enable debug logging in tests
Unit tests call runsc directly now, so all command line arguments
are valid. On the other hand, enabling debug in the test binary
doesn't affect runsc. It needs to be set in the config.

PiperOrigin-RevId: 200237706
Change-Id: I0b5922db17f887f58192dbc2f8dd2fd058b76ec7
2018-06-12 10:25:55 -07:00
Adin Scannell 41f766893a Minor ring0 interface cleanup.
- Remove unused methods.
- Provide declaration for asm function.

PiperOrigin-RevId: 200146850
Change-Id: Ic455c96ffe0d2e78ef15f824eb65d7de705b054a
2018-06-11 18:17:15 -07:00
Adin Scannell 1397a413b4 Make page tables split-safe.
In order to minimize the likelihood of exit during page table
modifications, make the full set of page table functions split-safe.
This is not strictly necessary (and you may still incur splits due to
allocations from the allocator pool) but should make retries a very rare
occurance.

PiperOrigin-RevId: 200146688
Change-Id: I8fa36aa16b807beda2f0b057be60038258e8d597
2018-06-11 18:15:14 -07:00
Adin Scannell 09b0a9c320 Handle all exception vectors.
PiperOrigin-RevId: 200144655
Change-Id: I5a753c74b75007b7714d6fe34aa0d2e845dc5c41
2018-06-11 17:57:19 -07:00
Fabricio Voznika ea4a468fba Set CLOEXEC option to sockets
hostinet/socket.go: the Sentry doesn't spawn new processes, but it doesn't hurt to protect the socket from leaking.
unet/unet.go: should be setting closing on exec. The FD is explicitly donated to children when needed.

PiperOrigin-RevId: 200135682
Change-Id: Ia8a45ced1e00a19420c8611b12e7a8ee770f89cb
2018-06-11 16:45:50 -07:00
Brian Geffon ab2c2575d6 Rpcinet is incorrectly handling MSG_TRUNC with SOCK_STREAM
SOCK_STREAM has special behavior with respect to MSG_TRUNC. Specifically,
the data isn't actually copied back out to userspace when MSG_TRUNC is
provided on a SOCK_STREAM.

According to tcp(7): "Since version 2.4, Linux supports the use of
MSG_TRUNC in the flags argument of recv(2) (and recvmsg(2)). This flag
causes the received bytes of data to be discarded, rather than passed
back in a caller-supplied buffer."

PiperOrigin-RevId: 200134860
Change-Id: I70f17a5f60ffe7794c3f0cfafd131c069202e90d
2018-06-11 16:40:38 -07:00
Brian Geffon 0412f17e06 rpcinet is treating EAGAIN and EWOULDBLOCK as different errnos.
PiperOrigin-RevId: 200124614
Change-Id: I38a7b083f1464a2a586fe24db648e624c455fec5
2018-06-11 15:34:08 -07:00
Fabricio Voznika 7260363751 Add O_TRUNC handling in openat
PiperOrigin-RevId: 200103677
Change-Id: I3efb565c30c64d35f8fd7b5c05ed78dcc2990c51
2018-06-11 13:35:21 -07:00
Kevin Krakauer 032b0398a5 Sentry: split tty.queue into its own file.
Minor refactor. line_discipline.go was home to 2 large structs (lineDiscipline
and queue), and queue is now large enough IMO to get its own file.

Also moves queue locks into the queue struct, making locking simpler.

PiperOrigin-RevId: 200080301
Change-Id: Ia75a0e9b3d9ac8d7e5a0f0099a54e1f5b8bdea34
2018-06-11 11:09:43 -07:00