Commit Graph

4922 Commits

Author SHA1 Message Date
Kevin Krakauer 04bdcc7b65 runsc: Enable waiting on individual containers within a sandbox.
PiperOrigin-RevId: 201742160
Change-Id: Ia9fa1442287c5f9e1196fb117c41536a80f6bb31
2018-06-22 14:31:25 -07:00
Nicolas Lacasse e0e6409812 Simplify some handle logic.
PiperOrigin-RevId: 201738936
Change-Id: Ib75136415e28e8df0c742acd6b9512d4809fe3a8
2018-06-22 14:10:30 -07:00
Jamie Liu fe3fc44da3 Handle mremap(old_size=0).
PiperOrigin-RevId: 201729703
Change-Id: I486900b0c6ec59533b88da225a5829c474e35a70
2018-06-22 13:08:38 -07:00
Brian Geffon 5d45f88f2c Netstack should return EOF on closed read.
The shutdown behavior where we return EAGAIN for sockets
which are non-blocking is only correct for packet based sockets.
SOCK_STREAM sockets should return EOF.

PiperOrigin-RevId: 201703055
Change-Id: I20b25ceca7286c37766936475855959706fc5397
2018-06-22 10:19:25 -07:00
Brielle Broder e1aee51d09 Modified Checkpoint/Restore flags to improve compatibility with Docker.
Added a number of unimplemented flags required for using runsc's
Checkpoint and Restore with Docker. Modified the "image-path" flag to
require a directory instead of a file.

PiperOrigin-RevId: 201697486
Change-Id: I55883df2f1bbc3ec3c395e0ca160ce189e5e7eba
2018-06-22 09:41:26 -07:00
Zhaozhong Ni 0e434b66a6 netstack: tcp socket connected state S/R support.
PiperOrigin-RevId: 201596247
Change-Id: Id22f47b2cdcbe14aa0d930f7807ba75f91a56724
2018-06-21 15:19:45 -07:00
Michael Pratt 2dedbc7211 Drop return from SendExternalSignal
SendExternalSignal is no longer called before CreateProcess, so it can
enforce this simplified precondition.

StartForwarding, and after Kernel.Start.

PiperOrigin-RevId: 201591170
Change-Id: Ib7022ef7895612d7d82a00942ab59fa433c4d6e9
2018-06-21 14:53:55 -07:00
Fabricio Voznika f6be5fe619 Forward SIGUSR2 to the sandbox too
SIGUSR2 was being masked out to be used as a way to dump sentry
stacks. This could cause compatibility problems in cases anyone
uses SIGUSR2 to communicate with the container init process.

PiperOrigin-RevId: 201575374
Change-Id: I312246e828f38ad059139bb45b8addc2ed055d74
2018-06-21 13:22:18 -07:00
Ian Gudger d571a4359c Implement ioctl(FIOASYNC)
FIOASYNC and friends are used to send signals when a file is ready for IO.

This may or may not be needed by Nginx. While Nginx does use it, it is unclear
if the code that uses it has any effect.

PiperOrigin-RevId: 201550828
Change-Id: I7ba05a7db4eb2dfffde11e9bd9a35b65b98d7f50
2018-06-21 10:53:21 -07:00
Justine Olshan f2a687001d Added functionality to create a RestoreEnvironment.
Before a container can be restored, the mounts must be configured.
The root and submounts and their key information is compiled into a
RestoreEnvironment.
Future code will be added to set this created environment before
restoring a container.
Tests to ensure the correct environment were added.

PiperOrigin-RevId: 201544637
Change-Id: Ia894a8b0f80f31104d1c732e113b1d65a4697087
2018-06-21 10:18:11 -07:00
Brielle Broder 7d6149063a Restore implementation added to runsc.
Restore creates a new container and uses the given image-path to load a saved
image of a previous container. Restore command is plumbed through container
and sandbox. This command does not work yet - more to come.

PiperOrigin-RevId: 201541229
Change-Id: I864a14c799ce3717d99bcdaaebc764281863d06f
2018-06-21 09:58:24 -07:00
Nicolas Lacasse 81d13fbd4d runsc: Default umask should be 0.
PiperOrigin-RevId: 201539050
Change-Id: I36cbf270fa5ad25de507ecb919e4005eda6aa16d
2018-06-21 09:43:15 -07:00
Ian Gudger ef4f239c79 Fix typo in runsc gofer flag description
PiperOrigin-RevId: 201529295
Change-Id: I55eb516ec6d14fbcd48593a3d61f724adc253a23
2018-06-21 08:34:51 -07:00
Fabricio Voznika 95cb01e0a9 Reduce test sleep time
PiperOrigin-RevId: 201428433
Change-Id: I72de1e46788ec84f61513416bb690956e515907e
2018-06-20 15:32:15 -07:00
Fabricio Voznika 2f59ba0e2d Include image test as part of kokoro tests
PiperOrigin-RevId: 201427731
Change-Id: I5cbee383ec51c02b7892ec7812cbbdc426be8991
2018-06-20 15:28:12 -07:00
Fabricio Voznika 2b5bdb525e Add end-to-end image tests
PiperOrigin-RevId: 201418619
Change-Id: I7961b027394d98422642f829bc54745838c138bd
2018-06-20 14:38:45 -07:00
Fabricio Voznika 4ad7315b67 Add 'runsc debug' command
It prints sandbox stacks to the log to help debug stuckness. I expect
that many more options will be added in the future.

PiperOrigin-RevId: 201405931
Change-Id: I87e560800cd5a5a7b210dc25a5661363c8c3a16e
2018-06-20 13:31:31 -07:00
Nicolas Lacasse 5aa7615ec9 Kokoro: Change name of uploaded log files.
PiperOrigin-RevId: 201404066
Change-Id: I8a0f2405d0192d412386592c1872a67582921b12
2018-06-20 13:20:36 -07:00
Nicolas Lacasse d93f55e863 Remove some defers in hot paths in the filesystem code.
PiperOrigin-RevId: 201401727
Change-Id: Ia5589882ba58a00efb522ab372e206b7e8e62aee
2018-06-20 13:05:54 -07:00
Fabricio Voznika af6f9f56f8 Add tool to configure runtime settings in docker
This will be used with the upcoming e2e image tests.

PiperOrigin-RevId: 201400832
Change-Id: I49509314e16ea54655ea8060dbf511a04a7a8f79
2018-06-20 13:01:16 -07:00
Zhaozhong Ni 4e9f0e91d7 sentry: pending signals S/R optimization.
Almost all of the hundreds of pending signal queues are empty upon save.

PiperOrigin-RevId: 201380318
Change-Id: I40747072435299de681d646e0862efac0637e172
2018-06-20 11:02:41 -07:00
Kevin Krakauer 5397963b5d runsc: Enable container creation within existing sandboxes.
Containers are created as processes in the sandbox. Of the many things that
don't work yet, the biggest issue is that the fsgofer is launched with its root
as the sandbox's root directory. Thus, when a container is started and wants to
read anything (including the init binary of the container), the gofer tries to
serve from sandbox's root (which basically just has pause), not the container's.

PiperOrigin-RevId: 201294560
Change-Id: I6423aa8830538959c56ae908ce067e4199d627b1
2018-06-19 21:44:33 -07:00
Brian Geffon db66e383c3 Epsocket has incorrect recv(2) behavior after SHUT_RD.
After shutdown(SHUT_RD) calls to recv /w MSG_DONTWAIT or with
O_NONBLOCK should result in a EAGAIN and not 0. Blocking sockets
should return 0 as they would have otherwise blocked indefinitely.

PiperOrigin-RevId: 201271123
Change-Id: If589b69c17fa5b9ff05bcf9e44024da9588c8876
2018-06-19 17:29:11 -07:00
Kevin Krakauer 3ebd0e35f4 runsc: Whitelist lstat, as it is now used in specutils.
When running multi-container, child containers are added after the filters have
been installed. Thus, lstat must be in the set of allowed syscalls.

PiperOrigin-RevId: 201269550
Change-Id: I03f2e6675a53d462ed12a0f651c10049b76d4c52
2018-06-19 17:17:41 -07:00
Zhaozhong Ni 18d8992453 state: pretty-print primitive type arrays.
PiperOrigin-RevId: 201269072
Change-Id: Ia542c5a42b5b5d21c1104a003ddff5279644d309
2018-06-19 17:13:35 -07:00
Kevin Krakauer 33f29c730f runsc: Fix flakey container_test.
Verified that this is no longer flakey over 10K repetitions.

PiperOrigin-RevId: 201267499
Change-Id: I793c916fe725412aec25953f764cb4f52c9fbed3
2018-06-19 17:04:51 -07:00
Adin Scannell be76cad5bc Make KVM more scalable by removing CPU cap.
Instead, CPUs will be created dynamically. We also allow a relatively
efficient mechanism for stealing and notifying when a vCPU becomes
available via unlock.

Since the number of vCPUs is no longer fixed at machine creation time,
we make the dirtySet packing more efficient. This has the pleasant side
effect of cutting out the unsafe address space code.

PiperOrigin-RevId: 201266691
Change-Id: I275c73525a4f38e3714b9ac0fd88731c26adfe66
2018-06-19 17:00:30 -07:00
Zhaozhong Ni aa14a2c1be sentry: futex S/R optimization.
No need to save thousands of zerovalue buckets.

PiperOrigin-RevId: 201258598
Change-Id: I5d3ea7b6a5345117ab4f610332d5288ca550be33
2018-06-19 16:08:00 -07:00
Justine Olshan a6dbef045f Added a resume command to unpause a paused container.
Resume checks the status of the container and unpauses the kernel
if its status is paused. Otherwise nothing happens.
Tests were added to ensure that the process is in the correct state
after various commands.

PiperOrigin-RevId: 201251234
Change-Id: Ifd11b336c33b654fea6238738f864fcf2bf81e19
2018-06-19 15:23:36 -07:00
Brian Geffon bda2a1ed35 Rpcinet is racy around shutdown flags.
Correct a data race in rpcinet where a shutdown and recvmsg can
race around shutown flags.

PiperOrigin-RevId: 201238366
Change-Id: I5eb06df4a2b4eba331eeb5de19076213081d581f
2018-06-19 14:12:52 -07:00
Nicolas Lacasse 9db7cfad93 Add a new cache policy FSCACHE_WRITETHROUGH.
The new policy is identical to FSCACHE (which caches everything in memory), but
it also flushes writes to the backing fs agent immediately.

All gofer cache policy decisions have been moved into the cachePolicy type.
Previously they were sprinkled around the codebase.

There are many different things that we cache (page cache, negative dirents,
dirent LRU, unstable attrs, readdir results....), and I don't think we should
have individual flags to control each of these.  Instead, we should have a few
high-level cache policies that are consistent and useful to users.  This
refactoring makes it easy to add more such policies.

PiperOrigin-RevId: 201206937
Change-Id: I6e225c382b2e5e1b0ad4ccf8ca229873f4cd389d
2018-06-19 11:10:11 -07:00
Zhaozhong Ni 5581256f87 state: include I/O and protobuf time in kernel S/R timing stats.
PiperOrigin-RevId: 201205733
Change-Id: I300307b0668989ba7776ab9e3faee71efdd33f46
2018-06-19 11:04:54 -07:00
Brian Geffon 4fd1d40e1d Rpcinet needs to track shutdown state for blocking sockets.
Because rpcinet will emulate a blocking socket backed by an rpc based
non-blocking socket. In the event of a shutdown(SHUT_RD) followed by a
read a non-blocking socket is allowed to return an EWOULDBLOCK however
since a blocking socket knows it cannot receive anymore data it would
block indefinitely and in this situation linux returns 0. We have to
track this on the rpcinet sentry side so we can emulate that behavior
because the remote side has no way to know if the socket is actually
blocking within the sentry.

PiperOrigin-RevId: 201201618
Change-Id: I4ac3a7b74b5dae471ab97c2e7d33b83f425aedac
2018-06-19 10:43:30 -07:00
Justine Olshan 873ec0c414 Modified boot.go to allow for restores.
A file descriptor was added as a flag to boot so a state file can restore a
container that was checkpointed.

PiperOrigin-RevId: 201068699
Change-Id: I18e96069488ffa3add468861397f3877725544aa
2018-06-18 15:20:36 -07:00
Fabricio Voznika 7eaca1074b Add pointer to dated builds in README
PiperOrigin-RevId: 201068427
Change-Id: If03c8c22907e6ef623f39c8ae5316fdd76cf80cb
2018-06-18 15:18:43 -07:00
Lantao Liu f3727528e5 runsc: support symlink to the exec path.
PiperOrigin-RevId: 201049912
Change-Id: Idd937492217a4c2ca3d59c602e41576a3b203dd9
2018-06-18 13:37:59 -07:00
Lantao Liu 821aaf531d runsc: support "rw" mount option.
PiperOrigin-RevId: 201018483
Change-Id: I52fe3d01c83c8a2f0e9275d9d88c37e46fa224a2
2018-06-18 10:34:11 -07:00
Fabricio Voznika 775982ed4b Automated rollback of changelist 200770591
PiperOrigin-RevId: 201012131
Change-Id: I5cd69e795555129319eb41135ecf26db9a0b1fcb
2018-06-18 10:00:04 -07:00
Brian Geffon 563a71ef24 Add rpcinet support for control messages.
Add support for control messages, but at this time the only
control message that the sentry will support here is SO_TIMESTAMP.

PiperOrigin-RevId: 200922230
Change-Id: I63a852d9305255625d9df1d989bd46a66e93c446
2018-06-17 17:06:40 -07:00
Justine Olshan 0786707cd9 Added code for a pause command for a container process.
Like runc, the pause command will pause the processes of the given container.
It will set that container's status to "paused."
A resume command will be be added to unpause and continue running the process.

PiperOrigin-RevId: 200789624
Change-Id: I72a5d7813d90ecfc4d01cc252d6018855016b1ea
2018-06-15 16:09:09 -07:00
Michael Pratt bd2d1aaa16 Replace crypto/rand with internal rand package
PiperOrigin-RevId: 200784607
Change-Id: I39aa6ee632936dcbb00fc298adccffa606e9f4c0
2018-06-15 15:36:00 -07:00
Kevin Krakauer 437890dc4b runsc: Make gofer logs show up in test output.
PiperOrigin-RevId: 200770591
Change-Id: Ifc096d88615b63135210d93c2b4cee2eaecf1eee
2018-06-15 14:07:54 -07:00
Lantao Liu 2081c5e7f7 runsc: support /dev bind mount which does not conflict with default /dev mount.
PiperOrigin-RevId: 200768923
Change-Id: I4b8da10bcac296e8171fe6754abec5aabfec5e65
2018-06-15 13:58:39 -07:00
Zhaozhong Ni fc8ca72a32 sentry: do not start delivering external signal immediately.
PiperOrigin-RevId: 200765756
Change-Id: Ie4266f32e4e977df3925eb29f3fbb756e0337606
2018-06-15 13:38:14 -07:00
Brian Geffon fa6db05e0c FIFOs should support O_TRUNC as a no-op.
PiperOrigin-RevId: 200759323
Change-Id: I683b2edcc2188304c4ca563e46af457e23625905
2018-06-15 12:55:29 -07:00
Dmitry Vyukov 52110bfc33 runsc/cmd: fix kill signal parsing
Signal is arg 1, not 2.
Killing with SIGABRT is useful to get Go traces.

Signed-off-by: Dmitry Vyukov <dvyukov@google.com>

Change-Id: I0b78e34a9de3fb3385108e26fdb4ff6e9347aeff
PiperOrigin-RevId: 200742743
2018-06-15 11:06:07 -07:00
Adin Scannell b31ac4e1df Use notify explicitly on unlock path.
There are circumstances under which the redpill call will not generate
the appropriate action and notification. Replace this call with an
explicit notification, which is guaranteed to transition as well as
perform the futex wake.

PiperOrigin-RevId: 200726934
Change-Id: Ie19e008a6007692dd7335a31a8b59f0af6e54aaa
2018-06-15 09:30:08 -07:00
Fabricio Voznika ef5dd4df9b Set kernel.applicationCores to the number of processor on the host
The right number to use is the number of processors assigned to the cgroup. But until
we make the sandbox join the respective cgroup, just use the number of processors on
the host.

Closes #65, closes #66

PiperOrigin-RevId: 200725483
Change-Id: I34a566b1a872e26c66f56fa6e3100f42aaf802b1
2018-06-15 09:19:04 -07:00
Fabricio Voznika 119a302ceb Implement /proc/thread-self
Closes #68

PiperOrigin-RevId: 200725401
Change-Id: I4827009b8aee89d22887c3af67291ccf7058d420
2018-06-15 09:18:00 -07:00
Adin Scannell 1eb1bf8670 Update contributing guidelines with an example.
Fixes #69

PiperOrigin-RevId: 200683809
Change-Id: I1312ebb3775d5f9088e9108359c19e2dedbb7b70
2018-06-15 01:22:08 -07:00