Commit Graph

97 Commits

Author SHA1 Message Date
Justine Olshan c05660373e Moved restore code out of create and made to be called after create.
Docker expects containers to be created before they are restored.
However, gVisor restoring requires specificactions regarding the kernel
and the file system. These actions were originally in booting the sandbox.

Now setting up the file system is deferred until a call to a call to
runsc start. In the restore case, the kernel is destroyed and a new kernel
is created in the same process, as we need the same process for Docker.

These changes required careful execution of concurrent processes which
required the use of a channel.

Full docker integration still needs the ability to restore into the same
container.

PiperOrigin-RevId: 205161441
Change-Id: Ie1d2304ead7e06855319d5dc310678f701bd099f
2018-07-18 16:58:30 -07:00
Nicolas Lacasse e5d8f99c60 runsc: Fixes to CheckpointRestoreTest.
We must delete the output file at the beginning of the test, otherwise the test
fails immediately.

Also some minor cleanups in readOutputFile.

PiperOrigin-RevId: 205150525
Change-Id: I6bae1acd5b315320a2c6e25a59afcfc06267fb17
2018-07-18 15:46:37 -07:00
Nicolas Lacasse 9059983fdb runsc: Fix map access race in boot.Loader.waitContainer.
PiperOrigin-RevId: 204522004
Change-Id: I4819dc025f0a1df03ceaaba7951b1902d44562b3
2018-07-13 13:46:14 -07:00
Nicolas Lacasse 6dce46d4c0 Bump the timeout when waiting for python HTTP server.
PiperOrigin-RevId: 204511630
Change-Id: Ib841a7144f3833321b0e69b8585b03c4ed55a265
2018-07-13 12:34:04 -07:00
Nicolas Lacasse 67507bd579 runsc: Don't close the control server in a defer.
Closing the control server will block until all open requests have completed.
If a control server method panics, we end up stuck because the defer'd Destroy
function will never return.

PiperOrigin-RevId: 204354676
Change-Id: I6bb1d84b31242d7c3f20d5334b1c966bd6a61dbf
2018-07-12 13:36:57 -07:00
Bhasker Hariharan c15cb8d432 Automated rollback of changelist 203157739
PiperOrigin-RevId: 204196916
Change-Id: If632750fc6368acb835e22cfcee0ae55c8a04d16
2018-07-11 15:07:19 -07:00
Justine Olshan 81ae5f3df5 Created runsc and docker integration tests.
Moved some of the docker image functions to testutil.go.
Test runsc commands create, start, stop, pause, and resume.

PiperOrigin-RevId: 204138452
Change-Id: Id00bc58d2ad230db5e9e905eed942187e68e7c7b
2018-07-11 09:37:28 -07:00
Brielle Broder b763b3992a Modified error message for clarity.
Previously, error message only showed "<nil>" when child and pid were the
same (since no error is returned by the Wait4 syscall in this case) which
occurs when the process has incorrectly terminated. A new error message
was added to improve clarity for such a case. Tests for this function were
modified to reflect the improved distinction between process termination
and error.

PiperOrigin-RevId: 204018107
Change-Id: Ib38481c9590405e5bafcb6efe27fd49b3948910c
2018-07-10 14:58:12 -07:00
Justine Olshan f107a5b1a0 Tests pause and resume functionality on a Python container.
PiperOrigin-RevId: 203488336
Change-Id: I55e1b646f1fae73c27a49e064875d55f5605b200
2018-07-06 09:39:01 -07:00
Michael Pratt 660f1203ff Fix runsc VDSO mapping
80bdf8a406 accidentally moved vdso into an
inner scope, never assigning the vdso variable passed to the Kernel and
thus skipping VDSO mappings.

Fix this and remove the ability for loadVDSO to skip VDSO mappings,
since tests that do so are gone.

PiperOrigin-RevId: 203169135
Change-Id: Ifd8cadcbaf82f959223c501edcc4d83d05327eba
2018-07-03 12:53:39 -07:00
Fabricio Voznika 52ddb8571c Skip overlay on root when its readonly
PiperOrigin-RevId: 203161098
Change-Id: Ia1904420cb3ee830899d24a4fe418bba6533be64
2018-07-03 12:01:09 -07:00
Lantao Liu 138cb8da50 runsc: `runsc wait` print wait status.
PiperOrigin-RevId: 203160639
Change-Id: I8fb2787ba0efb7eacd9d4c934238a26eb5ae79d5
2018-07-03 11:58:12 -07:00
Fabricio Voznika 0ef6066167 Resend packets back to netstack if destined to itself
Add option to redirect packet back to netstack if it's destined to itself.
This fixes the problem where connecting to the local NIC address would
not work, e.g.:
echo bar | nc -l -p 8080 &
echo foo | nc 192.168.0.2 8080

PiperOrigin-RevId: 203157739
Change-Id: I31c9f7c501e3f55007f25e1852c27893a16ac6c4
2018-07-03 11:39:17 -07:00
Fabricio Voznika c1b4c1ffee Fix flaky image_test
- Some failures were being ignored in run_tests.sh
- Give more time for mysql to setup
- Fix typo with network=host tests
- Change httpd test to wait on http server being available, not only output

PiperOrigin-RevId: 203156896
Change-Id: Ie1801dcd76e9b5fe4722c4d8695c76e40988dd74
2018-07-03 11:34:15 -07:00
Nicolas Lacasse 4500155ffc runsc: Mount "mandatory" mounts right after mounting the root.
The /proc and /sys mounts are "mandatory" in the sense that they should be
mounted in the sandbox even when they are not included in the spec. Runsc
treats /tmp similarly, because it is faster to use the internal tmpfs
implementation instead of proxying to the host.

However, the spec may contain submounts of these mandatory mounts (particularly
for /tmp). In those cases, we must mount our mandatory mounts before the
submount, otherwise the submount will be masked.

Since the mandatory mounts are all top-level directories, we can mount them
right after the root.

PiperOrigin-RevId: 203145635
Change-Id: Id69bae771d32c1a5b67e08c8131b73d9b42b2fbf
2018-07-03 10:36:22 -07:00
Dmitry Vyukov 6144751962 runsc/boot/filter: permit SYS_TIME for race
glibc's malloc also uses SYS_TIME. Permit it.

#0  0x0000000000de6267 in time ()
#1  0x0000000000db19d8 in get_nprocs ()
#2  0x0000000000d8a31a in arena_get2.part ()
#3  0x0000000000d8ab4a in malloc ()
#4  0x0000000000d3c6b5 in __sanitizer::InternalAlloc(unsigned long, __sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator32<0ul, 140737488355328ull, 0ul, __sanitizer::SizeClassMap<3ul, 4ul, 8ul, 17ul, 64ul, 14ul>, 20ul, __sanitizer::TwoLevelByteMap<32768ull, 4096ull, __sanitizer::NoOpMapUnmapCallback>, __sanitizer::NoOpMapUnmapCallback> >*, unsigned long) ()
#5  0x0000000000d4cd70 in __tsan_go_start ()
#6  0x00000000004617a3 in racecall ()
#7  0x00000000010f4ea0 in runtime.findfunctab ()
#8  0x000000000043f193 in runtime.racegostart ()

Signed-off-by: Dmitry Vyukov <dvyukov@google.com>
[mpratt@google.com: updated comments and commit message]
Signed-off-by: Michael Pratt <mpratt@google.com>

Change-Id: Ibe2d0dc3035bf5052d5fb802cfaa37c5e0e7a09a
PiperOrigin-RevId: 203042627
2018-07-02 17:47:32 -07:00
Lantao Liu 126296ce2a runsc: fix panic for `runsc wait` on stopped container.
PiperOrigin-RevId: 203016694
Change-Id: Ic51ef754aa6d7d1b3b35491aff96a63d7992e122
2018-07-02 14:52:21 -07:00
Fabricio Voznika fa64c2a151 Make default limits the same as with runc
Closes #2

PiperOrigin-RevId: 202997196
Change-Id: I0c9f6f5a8a1abe1ae427bca5f590bdf9f82a6675
2018-07-02 12:51:38 -07:00
Brielle Broder ca353b53ed Fix typo.
PiperOrigin-RevId: 202720658
Change-Id: Iff42fd23f831ee7f29ddd6eb867020b76ed1eb23
2018-06-29 15:51:32 -07:00
Justine Olshan 80bdf8a406 Sets the restore environment for restoring a container.
Updated how restoring occurs through boot.go with a separate Restore function.
This prevents a new process and new mounts from being created.
Added tests to ensure the container is restored.
Registered checkpoint and restore commands so they can be used.
Docker support for these commands is still limited.
Working on #80.

PiperOrigin-RevId: 202710950
Change-Id: I2b893ceaef6b9442b1ce3743bd112383cb92af0c
2018-06-29 14:47:40 -07:00
Brielle Broder 25e315c2e1 Added leave-running flag for checkpoint.
The leave-running flag allows the container to continue running after a
checkpoint has occurred by doing an immediate restore into a new
container with the same container ID after the old container is destroyed.

Updates #80.

PiperOrigin-RevId: 202695426
Change-Id: Iac50437f5afda018dc18b24bb8ddb935983cf336
2018-06-29 13:09:33 -07:00
Kevin Krakauer 16d37973eb runsc: Add the "wait" subcommand.
Users can now call "runsc wait <container id>" to wait on a particular process
inside the container. -pid can also be used to wait on a specific PID.

Manually tested the wait subcommand for a single waiter and multiple waiters
(simultaneously 2 processes waiting on the container and 2 processes waiting on
a PID within the container).

PiperOrigin-RevId: 202548978
Change-Id: Idd507c2cdea613c3a14879b51cfb0f7ea3fb3d4c
2018-06-28 14:56:36 -07:00
Fabricio Voznika 5a8e014c3d Add more image tests
PiperOrigin-RevId: 202537696
Change-Id: I900fe8fd36cc7a4edb44fe2d03f8ba6768db53cb
2018-06-28 13:54:04 -07:00
Fabricio Voznika bb31a11903 Wait for sandbox process when waiting for root container
Closes #71

PiperOrigin-RevId: 202532762
Change-Id: I80a446ff638672ff08e6fd853cd77e28dd05d540
2018-06-28 13:23:04 -07:00
Fabricio Voznika 8459390cdd Error out if spec is invalid
Closes #66

PiperOrigin-RevId: 202496258
Change-Id: Ib9287c5bf1279ffba1db21ebd9e6b59305cddf34
2018-06-28 09:57:27 -07:00
Fabricio Voznika 1f207de315 Add option to configure watchdog action
PiperOrigin-RevId: 202494747
Change-Id: I4d4a18e71468690b785060e580a5f83c616bd90f
2018-06-28 09:46:50 -07:00
Brielle Broder f93043615f Added MkdirAll capabilities for Checkpoint's image-path.
Now able to save the state file (checkpoint.img) at an image-path that had
previously not existed. This is important because there can only be one
checkpoint.img file per directory so this will enable users to create as many
directories as needed for proper organization.

PiperOrigin-RevId: 202360414
Change-Id: If5dd2b72e08ab52834a2b605571186d107b64526
2018-06-27 13:32:53 -07:00
Fabricio Voznika c186e408cc Add KVM, overlay and host network to image tests
PiperOrigin-RevId: 202236006
Change-Id: I4ea964a70fc49e8b51c9da27d77301c4eadaae71
2018-06-26 19:05:50 -07:00
Lantao Liu 000fd8d1e4 runsc: set gofer umask to 0.
PiperOrigin-RevId: 202185642
Change-Id: I2eefcc0b2ffadc6ef21d177a8a4ab0cda91f3399
2018-06-26 13:40:04 -07:00
Lantao Liu e8ae2b85e9 runsc: add a `multi-container` flag to enable multi-container support.
PiperOrigin-RevId: 201995800
Change-Id: I770190d135e14ec7da4b3155009fe10121b2a502
2018-06-25 12:08:44 -07:00
Fabricio Voznika cecc1e472c Fix lint errors
PiperOrigin-RevId: 201978212
Change-Id: Ie3df1fd41d5293fff66b546a0c68c3bf98126067
2018-06-25 10:41:27 -07:00
Kevin Krakauer 04bdcc7b65 runsc: Enable waiting on individual containers within a sandbox.
PiperOrigin-RevId: 201742160
Change-Id: Ia9fa1442287c5f9e1196fb117c41536a80f6bb31
2018-06-22 14:31:25 -07:00
Brielle Broder e1aee51d09 Modified Checkpoint/Restore flags to improve compatibility with Docker.
Added a number of unimplemented flags required for using runsc's
Checkpoint and Restore with Docker. Modified the "image-path" flag to
require a directory instead of a file.

PiperOrigin-RevId: 201697486
Change-Id: I55883df2f1bbc3ec3c395e0ca160ce189e5e7eba
2018-06-22 09:41:26 -07:00
Fabricio Voznika f6be5fe619 Forward SIGUSR2 to the sandbox too
SIGUSR2 was being masked out to be used as a way to dump sentry
stacks. This could cause compatibility problems in cases anyone
uses SIGUSR2 to communicate with the container init process.

PiperOrigin-RevId: 201575374
Change-Id: I312246e828f38ad059139bb45b8addc2ed055d74
2018-06-21 13:22:18 -07:00
Justine Olshan f2a687001d Added functionality to create a RestoreEnvironment.
Before a container can be restored, the mounts must be configured.
The root and submounts and their key information is compiled into a
RestoreEnvironment.
Future code will be added to set this created environment before
restoring a container.
Tests to ensure the correct environment were added.

PiperOrigin-RevId: 201544637
Change-Id: Ia894a8b0f80f31104d1c732e113b1d65a4697087
2018-06-21 10:18:11 -07:00
Brielle Broder 7d6149063a Restore implementation added to runsc.
Restore creates a new container and uses the given image-path to load a saved
image of a previous container. Restore command is plumbed through container
and sandbox. This command does not work yet - more to come.

PiperOrigin-RevId: 201541229
Change-Id: I864a14c799ce3717d99bcdaaebc764281863d06f
2018-06-21 09:58:24 -07:00
Nicolas Lacasse 81d13fbd4d runsc: Default umask should be 0.
PiperOrigin-RevId: 201539050
Change-Id: I36cbf270fa5ad25de507ecb919e4005eda6aa16d
2018-06-21 09:43:15 -07:00
Ian Gudger ef4f239c79 Fix typo in runsc gofer flag description
PiperOrigin-RevId: 201529295
Change-Id: I55eb516ec6d14fbcd48593a3d61f724adc253a23
2018-06-21 08:34:51 -07:00
Fabricio Voznika 95cb01e0a9 Reduce test sleep time
PiperOrigin-RevId: 201428433
Change-Id: I72de1e46788ec84f61513416bb690956e515907e
2018-06-20 15:32:15 -07:00
Fabricio Voznika 2f59ba0e2d Include image test as part of kokoro tests
PiperOrigin-RevId: 201427731
Change-Id: I5cbee383ec51c02b7892ec7812cbbdc426be8991
2018-06-20 15:28:12 -07:00
Fabricio Voznika 2b5bdb525e Add end-to-end image tests
PiperOrigin-RevId: 201418619
Change-Id: I7961b027394d98422642f829bc54745838c138bd
2018-06-20 14:38:45 -07:00
Fabricio Voznika 4ad7315b67 Add 'runsc debug' command
It prints sandbox stacks to the log to help debug stuckness. I expect
that many more options will be added in the future.

PiperOrigin-RevId: 201405931
Change-Id: I87e560800cd5a5a7b210dc25a5661363c8c3a16e
2018-06-20 13:31:31 -07:00
Fabricio Voznika af6f9f56f8 Add tool to configure runtime settings in docker
This will be used with the upcoming e2e image tests.

PiperOrigin-RevId: 201400832
Change-Id: I49509314e16ea54655ea8060dbf511a04a7a8f79
2018-06-20 13:01:16 -07:00
Kevin Krakauer 5397963b5d runsc: Enable container creation within existing sandboxes.
Containers are created as processes in the sandbox. Of the many things that
don't work yet, the biggest issue is that the fsgofer is launched with its root
as the sandbox's root directory. Thus, when a container is started and wants to
read anything (including the init binary of the container), the gofer tries to
serve from sandbox's root (which basically just has pause), not the container's.

PiperOrigin-RevId: 201294560
Change-Id: I6423aa8830538959c56ae908ce067e4199d627b1
2018-06-19 21:44:33 -07:00
Kevin Krakauer 3ebd0e35f4 runsc: Whitelist lstat, as it is now used in specutils.
When running multi-container, child containers are added after the filters have
been installed. Thus, lstat must be in the set of allowed syscalls.

PiperOrigin-RevId: 201269550
Change-Id: I03f2e6675a53d462ed12a0f651c10049b76d4c52
2018-06-19 17:17:41 -07:00
Kevin Krakauer 33f29c730f runsc: Fix flakey container_test.
Verified that this is no longer flakey over 10K repetitions.

PiperOrigin-RevId: 201267499
Change-Id: I793c916fe725412aec25953f764cb4f52c9fbed3
2018-06-19 17:04:51 -07:00
Justine Olshan a6dbef045f Added a resume command to unpause a paused container.
Resume checks the status of the container and unpauses the kernel
if its status is paused. Otherwise nothing happens.
Tests were added to ensure that the process is in the correct state
after various commands.

PiperOrigin-RevId: 201251234
Change-Id: Ifd11b336c33b654fea6238738f864fcf2bf81e19
2018-06-19 15:23:36 -07:00
Justine Olshan 873ec0c414 Modified boot.go to allow for restores.
A file descriptor was added as a flag to boot so a state file can restore a
container that was checkpointed.

PiperOrigin-RevId: 201068699
Change-Id: I18e96069488ffa3add468861397f3877725544aa
2018-06-18 15:20:36 -07:00
Lantao Liu f3727528e5 runsc: support symlink to the exec path.
PiperOrigin-RevId: 201049912
Change-Id: Idd937492217a4c2ca3d59c602e41576a3b203dd9
2018-06-18 13:37:59 -07:00
Lantao Liu 821aaf531d runsc: support "rw" mount option.
PiperOrigin-RevId: 201018483
Change-Id: I52fe3d01c83c8a2f0e9275d9d88c37e46fa224a2
2018-06-18 10:34:11 -07:00