Linux kernel before 4.19 doesn't implement a feature that updates
open FD after a file is open for write (and is copied to the upper
layer). Already open FD will continue to read the old file content
until they are reopened. This is especially problematic for gVisor
because it caches open files.
Flag was added to force readonly files to be reopenned when the
same file is open for write. This is only needed if using kernels
prior to 4.19.
Closes#1006
It's difficult to really test this because we never run on tests
on older kernels. I'm adding a test in GKE which uses kernels
with the overlayfs problem for 1.14 and lower.
PiperOrigin-RevId: 275115289
When any of these flags are set, all writes will trigger a subsequent fsync
call. This behavior already existed for "write-through" mounts.
O_DIRECT is treated as an alias for O_SYNC. Better support coming soon.
PiperOrigin-RevId: 275114392
This proc file reports routing information to applications inside the
container.
Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
Change-Id: I498e47f8c4c185419befbb42d849d0b099ec71f3
This proc file contains statistics according to [1].
[1] https://tools.ietf.org/html/rfc2013
Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
Change-Id: I9662132085edd8a7783d356ce4237d7ac0800d94
Right now, we can find more than one process with the 1 PID in /proc.
$ for i in `seq 10`; do
> unshare -fp sleep 1000 &
> done
$ ls /proc
1 1 1 1 12 18 24 29 6 loadavg net sys version
1 1 1 1 16 20 26 32 cpuinfo meminfo self thread-self
1 1 1 1 17 21 28 36 filesystems mounts stat uptime
PiperOrigin-RevId: 272506593
Kernel.cpuClockTicker increments kernel.cpuClock, which tasks use as a clock to
track their CPU usage. This improves latency in the syscall path by avoid
expensive monotonic clock calls on every syscall entry/exit.
However, this timer fires every 10ms. Thus, when all tasks are idle (i.e.,
blocked or stopped), this forces a sentry wakeup every 10ms, when we may
otherwise be able to sleep until the next app-relevant event. These wakeups
cause the sentry to utilize approximately 2% CPU when the application is
otherwise idle.
Updates to clock are not strictly necessary when the app is idle, as there are
no readers of cpuClock. This commit reduces idle CPU by disabling the timer
when tasks are completely idle, and computing its effects at the next wakeup.
Rather than disabling the timer as soon as the app goes idle, we wait until the
next tick, which provides a window for short sleeps to sleep and wakeup without
doing the (relatively) expensive work of disabling and enabling the timer.
PiperOrigin-RevId: 272265822
The gofer's CachingInodeOperations implementation contains an optimization for
the common open-read-close pattern when we have a host FD. In this case, the
host kernel will update the timestamp for us to a reasonably close time, so we
don't need an extra RPC to the gofer.
However, when the app explicitly sets the timestamps (via futimes or similar)
then we actually DO need to update the timestamps, because the host kernel
won't do it for us.
To fix this, a new boolean `forceSetTimestamps` was added to
CachineInodeOperations.SetMaskedAttributes. It is only set by
gofer.InodeOperations.SetTimestamps.
PiperOrigin-RevId: 272048146
How to reproduce:
$ echo "timeout 10 ls" > foo.sh
$ chmod +x foo.sh
$ ./foo.sh
(will hang here for 10 secs, and the output of ls does not show)
When "ls" process writes to stdout, it receives SIGTTOU signal, and
hangs there. Until "timeout" process timeouts, and kills "ls" process.
The expected result is: "ls" writes its output into tty, and terminates
immdedately, then "timeout" process receives SIGCHLD and terminates.
The reason for this failure is that we missed the check for TOSTOP (if
set, background processes will receive the SIGTTOU signal when they do
write).
We use drivers/tty/n_tty.c:n_tty_write() as a reference.
Fixes: #862
Reported-by: chris.zn <chris.zn@antfin.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
Signed-off-by: chenglang.hy <chenglang.hy@antfin.com>
Adresses a deadlock with the rolled back change:
b6a5b950d2
Creating a session from an orphaned process group was causing a lock to be
acquired twice by a single goroutine. This behavior is addressed, and a test
(OrphanRegression) has been added to pty.cc.
Implemented the following ioctls:
- TIOCSCTTY - set controlling TTY
- TIOCNOTTY - remove controlling tty, maybe signal some other processes
- TIOCGPGRP - get foreground process group. Also enables tcgetpgrp().
- TIOCSPGRP - set foreground process group. Also enabled tcsetpgrp().
Next steps are to actually turn terminal-generated control characters (e.g. C^c)
into signals to the proper process groups, and to send SIGTTOU and SIGTTIN when
appropriate.
PiperOrigin-RevId: 270088599
This also allows the tee(2) implementation to be enabled, since dup can now be
properly supported via WriteTo.
Note that this change necessitated some minor restructoring with the
fs.FileOperations splice methods. If the *fs.File is passed through directly,
then only public API methods are accessible, which will deadlock immediately
since the locking is already done by fs.Splice. Instead, we pass through an
abstract io.Reader or io.Writer, which elide locks and use the underlying
fs.FileOperations directly.
PiperOrigin-RevId: 268805207
This used to be the case, but regressed after a recent change.
Also made a few fixes around it and clean up the code a bit.
Closes#720
PiperOrigin-RevId: 265717496
For SOCK_STREAM type unix socket, we shall return ECONNRESET if peer is
closed with data not read.
We explictly set a flag when closing one end, to differentiate from
just shutdown (where zero shall be returned).
Fixes: #735
Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
Previously we were representing socket addresses as an interface{},
which allowed any type which could be binary.Marshal()ed to be used as
a socket address. This is fine when the address is passed to userspace
via the linux ABI, but is problematic when used from within the sentry
such as by networking procfs files.
PiperOrigin-RevId: 262460640
- This also gets rid of pipes for now because pipe does not have vfs2 specific
support yet.
- Added file path resolution logic.
- Fixes testing infrastructure.
- Does not include unit tests yet.
PiperOrigin-RevId: 262213950
(Don't worry, this is mostly tests.)
Implemented the following ioctls:
- TIOCSCTTY - set controlling TTY
- TIOCNOTTY - remove controlling tty, maybe signal some other processes
- TIOCGPGRP - get foreground process group. Also enables tcgetpgrp().
- TIOCSPGRP - set foreground process group. Also enabled tcsetpgrp().
Next steps are to actually turn terminal-generated control characters (e.g. C^c)
into signals to the proper process groups, and to send SIGTTOU and SIGTTIN when
appropriate.
PiperOrigin-RevId: 261387276
We can get the mount namespace from the CreateProcessArgs in all cases where we
need it. This also gets rid of kernel.Destroy method, since the only thing it
was doing was DecRefing the mounts.
Removing the need to call kernel.SetRootMountNamespace also allowed for some
more simplifications in the container fs setup code.
PiperOrigin-RevId: 261357060
This is the source of many warnings like:
AtomicRefCount 0x7f5ff84e3500 owned by "fs.Inode" garbage collected with ref count of 1 (want 0)
PiperOrigin-RevId: 261197093
It gets rid of holding state of the io.Reader offset (which is anyways held by
the vfs.FileDescriptor struct. It is also odd using a io.Reader becuase we
using io.ReaderAt to interact with the device. So making a io.ReaderAt wrapper
makes more sense.
Most importantly, it gets rid of the complexity of extracting the file reader
from a regular file implementation and then using it. Now we can just use the
regular file implementation as a reader which is more intuitive.
PiperOrigin-RevId: 260846927
Adds feature to launch from an open host FD instead of a binary_path.
The FD should point to a valid executable and most likely be statically
compiled. If the executable is not statically compiled, the loader will
search along the interpreter paths, which must be able to be resolved in
the Sandbox's file system or start will fail.
PiperOrigin-RevId: 260756825
This provides the following benefits:
- We can now use pkg/fd package which does not take ownership
of the file descriptor. So it does not close the fd when garbage collected.
This reduces scope of errors from unexpected garbage collection of io.File.
- It enforces the offset parameter in every read call.
It does not affect the fd offset nor is it affected by it. Hence reducing
scope of error of using stale offsets when reading.
- We do not need to serialize the usage of any global file descriptor anymore.
So this drops the mutual exclusion req hence reducing complexity and
congestion.
PiperOrigin-RevId: 260635174
Kevin Krakauer
gVisor bot
Fabricio Voznika
Nicolas Lacasse
Jianfeng Tan
Andrei Vagin
Michael Pratt
Adin Scannell
Jamie Liu
Bhasker Hariharan
Rahat Mahmood
Tamir Duberstein
Ayush Ranjan
Zach Koopmans