Commit Graph

282 Commits

Author SHA1 Message Date
Ayush Ranjan d7bb79b6f1 ext: Add ext2 and ext3 tiny images.
PiperOrigin-RevId: 259657917
2019-07-23 19:01:05 -07:00
Ayush Ranjan bd7708956f ext: Added extent tree building logic.
PiperOrigin-RevId: 259628657
2019-07-23 15:51:50 -07:00
Michael Pratt 6f7e2bb388 Take copyMu in Revalidate
copyMu is required to read child.overlay.upper.

PiperOrigin-RevId: 258662209
2019-07-17 16:12:01 -07:00
Jamie Liu 2bc398bfd8 Separate O_DSYNC and O_SYNC.
PiperOrigin-RevId: 258657913
2019-07-17 15:52:38 -07:00
Ayush Ranjan 84a59de5dc ext: disklayout: extents support.
PiperOrigin-RevId: 258657776
2019-07-17 15:48:58 -07:00
Ayush Ranjan 8e3e021aca ext: Filesystem init implementation.
PiperOrigin-RevId: 258645957
2019-07-17 14:48:04 -07:00
gVisor bot 682fd2d68f Merge pull request #533 from kevinGC:stub-dev-tty
PiperOrigin-RevId: 258607547
2019-07-17 11:28:30 -07:00
Michael Pratt ca829158e3 Properly invalidate cache in rename and remove
We were invalidating the wrong overlayEntry in rename and missing invalidation
in rename and remove if lower exists.

PiperOrigin-RevId: 258604685
2019-07-17 11:14:57 -07:00
gVisor bot 78a2704bde Merge pull request #474 from zhuangel:proctasks
PiperOrigin-RevId: 258479216
2019-07-16 18:12:07 -07:00
Jianfeng Tan cf4fc510fd Support /proc/net/dev
This proc file reports the stats of interfaces. We could use ifconfig
command to check the result.

Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
Change-Id: Ia7c1e637f5c76c30791ffda68ee61e861b6ef827
COPYBARA_INTEGRATE_REVIEW=https://gvisor-review.googlesource.com/c/gvisor/+/18282/
PiperOrigin-RevId: 258303936
2019-07-15 22:51:05 -07:00
gVisor bot eff2c264a4 Merge pull request #282 from zhangningdlut:chris_test_proc
PiperOrigin-RevId: 257855479
2019-07-12 13:11:01 -07:00
Kevin ddef7f8078 Fix license year and remove Read. 2019-07-11 21:31:26 -07:00
Kevin 44427d8e26 Add a stub for /dev/tty.
Actual implementation to follow, but this will satisfy applications that
want it to just exist.
2019-07-11 21:24:27 -07:00
Ayush Ranjan 2eeca68900 Added tiny ext4 image.
The image is of size 64Kb which supports 64 1k blocks
and 16 inodes. This is the smallest size mkfs.ext4 works with.

Added README.md documenting how this was created and included
all files on the device under assets.

PiperOrigin-RevId: 257712672
2019-07-11 17:17:47 -07:00
Ayush Ranjan 5242face2e ext: boilerplate code.
Renamed ext4 to ext since we are targeting ext(2/3/4).
Removed fs.go since we are targeting VFS2.
Added ext.go with filesystem struct.

PiperOrigin-RevId: 257689775
2019-07-11 15:05:36 -07:00
Ayush Ranjan 7965b1272b ext4: disklayout: Directory Entry implementation.
PiperOrigin-RevId: 257314911
2019-07-09 18:36:02 -07:00
Nicolas Lacasse 659bebab8e Don't try to execute a file that is not regular.
PiperOrigin-RevId: 257037608
2019-07-08 12:56:48 -07:00
Ayush Ranjan 8f9b1ca8e7 ext4: disklayout: inode impl.
PiperOrigin-RevId: 257010414
2019-07-08 10:44:11 -07:00
Adin Scannell 753da9604e Remove map from fd_map, change to fd_table.
This renames FDMap to FDTable and drops the kernel.FD type, which had an entire
package to itself and didn't serve much use (it was freely cast between types,
and served as more of an annoyance than providing any protection.)

Based on BenchmarkFDLookupAndDecRef-12, we can expect 5-10 ns per lookup
operation, and 10-15 ns per concurrent lookup operation of savings.

This also fixes two tangential usage issues with the FDMap. Namely, non-atomic
use of NewFDFrom and associated calls to Remove (that are both racy and fail to
drop the reference on the underlying file.)

PiperOrigin-RevId: 256285890
2019-07-02 19:28:59 -07:00
Ayush Ranjan d8ec2fb671 Ext4: DiskLayout: Inode interface.
PiperOrigin-RevId: 256234390
2019-07-02 14:04:31 -07:00
Ian Gudger 45566fa4e4 Add finalizer on AtomicRefCount to check for leaks.
PiperOrigin-RevId: 255711454
2019-06-28 20:07:52 -07:00
Adin Scannell 7dae043fec Drop ashmem and binder.
These are unfortunately unused and unmaintained. They can be brought back in
the future if need requires it.

PiperOrigin-RevId: 255697132
2019-06-28 17:20:25 -07:00
Ayush Ranjan c4da599e22 ext4: disklayout: SuperBlock interface implementations.
PiperOrigin-RevId: 255687771
2019-06-28 16:18:29 -07:00
Nicolas Lacasse 295078fa7a Automated rollback of changelist 255263686
PiperOrigin-RevId: 255679453
2019-06-28 15:28:41 -07:00
Ayush Ranjan 7c13789818 Superblock interface in the disk layout package for ext4.
PiperOrigin-RevId: 255644277
2019-06-28 12:07:28 -07:00
Yong He c61d7761b4 Fix deadloop in proc subtask list
Readdir of /proc/x/task/ will get direntry entries
from tasks of specified taskgroup. Now the tasks
slice is unsorted, use sort.SearchInts search entry
from the slice may cause infinity loops.
The fix is sort the slice before search.
This issue could be easily reproduced via following
steps, revise Readdir in pkg/sentry/fs/proc/task.go,
force set taskInts into test slice
[]int{1, 11, 7, 5, 10, 6, 8, 3, 9, 2, 4},
then run docker image and run ls /proc/1/task, the
command will cause infinity loops.
2019-06-28 22:20:57 +08:00
Fabricio Voznika b2907595e5 Complete pipe support on overlayfs
Get/Set pipe size and ioctl support were missing from
overlayfs. It required moving the pipe.Sizer interface
to fs so that overlay could get access.

Fixes #318

PiperOrigin-RevId: 255511125
2019-06-27 17:22:53 -07:00
Michael Pratt 5b41ba5d0e Fix various spelling issues in the documentation
Addresses obvious typos, in the documentation only.

COPYBARA_INTEGRATE_REVIEW=https://github.com/google/gvisor/pull/443 from Pixep:fix/documentation-spelling 4d0688164eafaf0b3010e5f4824b35d1e7176d65
PiperOrigin-RevId: 255477779
2019-06-27 14:25:50 -07:00
Michael Pratt 085a907565 Cache directory entries in the overlay
Currently, the overlay dirCache is only used for a single logical use of
getdents. i.e., it is discard when the FD is closed or seeked back to
the beginning.

But the initial work of getting the directory contents can be quite
expensive (particularly sorting large directories), so we should keep it
as long as possible.

This is very similar to the readdirCache in fs/gofer.

Since the upper filesystem does not have to allow caching readdir
entries, the new CacheReaddir MountSourceOperations method controls this
behavior.

This caching should be trivially movable to all Inodes if desired,
though that adds an additional copy step for non-overlay Inodes.
(Overlay Inodes already do the extra copy).

PiperOrigin-RevId: 255477592
2019-06-27 14:24:03 -07:00
Fabricio Voznika 42e212f6b7 Preserve permissions when checking lower
The code was wrongly assuming that only read access was
required from the lower overlay when checking for permissions.
This allowed non-writable files to be writable in the overlay.

Fixes #316

PiperOrigin-RevId: 255263686
2019-06-26 14:24:44 -07:00
Michael Pratt e98ce4a2c6 Add TODO reminder to remove tmpfs caching options
Updates #179

PiperOrigin-RevId: 255081565
2019-06-25 17:12:34 -07:00
Andrei Vagin e9ea7230f7 fs: synchronize concurrent writes into files with O_APPEND
For files with O_APPEND, a file write operation gets a file size and uses it as
offset to call an inode write operation. This means that all other operations
which can change a file size should be blocked while the write operation doesn't
complete.

PiperOrigin-RevId: 254873771
2019-06-24 17:45:02 -07:00
Rahat Mahmood 94a6bfab5d Implement /proc/net/tcp.
PiperOrigin-RevId: 254854346
2019-06-24 15:56:36 -07:00
chris.zn f957fb23cf Return ENOENT when reading /proc/{pid}/task of an exited process
There will be a deadloop when we use getdents to read /proc/{pid}/task
of an exited process

Like this:

Process A is running
                         Process B: open /proc/{pid of A}/task
Process A exits
                         Process B: getdents /proc/{pid of A}/task

Then, process B will fall into deadloop, and return "." and ".."
in loops and never ends.

This patch returns ENOENT when use getdents to read /proc/{pid}/task
if the process is just exited.

Signed-off-by: chris.zn <chris.zn@antfin.com>
2019-06-24 15:49:53 +08:00
Nicolas Lacasse 35719d52c7 Implement statx.
We don't have the plumbing for btime yet, so that field is left off. The
returned mask indicates that btime is absent.

Fixes #343

PiperOrigin-RevId: 254575752
2019-06-22 13:29:26 -07:00
Andrei Vagin ab6774cebf gvisor/fs: getdents returns 0 if offset is equal to FileMaxOffset
FileMaxOffset is a special case when lseek(d, 0, SEEK_END) has been called.

PiperOrigin-RevId: 254498777
2019-06-21 17:25:17 -07:00
Ayush Ranjan 727375321f ext4 block group descriptor implementation in disk layout package.
PiperOrigin-RevId: 254482180
2019-06-21 15:42:46 -07:00
Michael Pratt 292f70cbf7 Add package docs to seqfile and ramfs
These are the only packages missing docs:
https://godoc.org/gvisor.dev/gvisor

PiperOrigin-RevId: 254261022
2019-06-20 13:34:33 -07:00
Nicolas Lacasse f7428af9c1 Add MountNamespace to task.
This allows tasks to have distinct mount namespace, instead of all sharing the
kernel's root mount namespace.

Currently, the only way for a task to get a different mount namespace than the
kernel's root is by explicitly setting a different MountNamespace in
CreateProcessArgs, and nothing does this (yet).

In a follow-up CL, we will set CreateProcessArgs.MountNamespace when creating a
new container inside runsc.

Note that "MountNamespace" is a poor term for this thing. It's more like a
distinct VFS tree. When we get around to adding real mount namespaces, this
will need a better naem.

PiperOrigin-RevId: 254009310
2019-06-19 09:21:21 -07:00
Fabricio Voznika ca245a428b Attempt to fix TestPipeWritesAccumulate
Test fails because it's reading 4KB instead of the
expected 64KB. Changed the test to read pipe buffer
size instead of hardcode and added some logging in
case the reason for failure was not pipe buffer size.

PiperOrigin-RevId: 253916040
2019-06-18 19:16:11 -07:00
Andrei Vagin 8ab0848c70 gvisor/fs: don't update file.offset for sockets, pipes, etc
sockets, pipes and other non-seekable file descriptors don't
use file.offset, so we don't need to update it.

With this change, we will be able to call file operations
without locking the file.mu mutex. This is already used for
pipes in the splice system call.

PiperOrigin-RevId: 253746644
2019-06-18 01:43:29 -07:00
Ian Gudger 3e9b8ecbfe Plumb context through more layers of filesytem.
All functions which allocate objects containing AtomicRefCounts will soon need
a context.

PiperOrigin-RevId: 253147709
2019-06-13 18:40:38 -07:00
Adin Scannell add40fd6ad Update canonical repository.
This can be merged after:
https://github.com/google/gvisor-website/pull/77
  or
https://github.com/google/gvisor-website/pull/78

PiperOrigin-RevId: 253132620
2019-06-13 16:50:15 -07:00
Fabricio Voznika fc746efa9a Add support to mount pod shared tmpfs mounts
Parse annotations containing 'gvisor.dev/spec/mount' that gives
hints about how mounts are shared between containers inside a
pod. This information can be used to better inform how to mount
these volumes inside gVisor. For example, a volume that is shared
between containers inside a pod can be bind mounted inside the
sandbox, instead of being two independent mounts.

For now, this information is used to allow the same tmpfs mounts
to be shared between containers which wasn't possible before.

PiperOrigin-RevId: 252704037
2019-06-11 14:54:31 -07:00
Rahat Mahmood a00157cc0e Store more information in the kernel socket table.
Store enough information in the kernel socket table to distinguish
between different types of sockets. Previously we were only storing
the socket family, but this isn't enough to classify sockets. For
example, TCPv4 and UDPv4 sockets are both AF_INET, and ICMP sockets
are SOCK_DGRAM sockets with a particular protocol.

Instead of creating more sub-tables, flatten the socket table and
provide a filtering mechanism based on the socket entry.

Also generate and store a socket entry index ("sl" in linux) which
allows us to output entries in a stable order from procfs.

PiperOrigin-RevId: 252495895
2019-06-10 15:17:43 -07:00
Rahat Mahmood 315cf9a523 Use common definition of SockType.
SockType isn't specific to unix domain sockets, and the current
definition basically mirrors the linux ABI's definition.

PiperOrigin-RevId: 251956740
2019-06-06 17:00:27 -07:00
Fabricio Voznika 02ab1f187c Copy up parent when binding UDS on overlayfs
Overlayfs was expecting the parent to exist when bind(2)
was called, which may not be the case. The fix is to copy
the parent directory to the upper layer before binding
the UDS.

There is not good place to add tests for it. Syscall tests
would be ideal, but it's hard to guarantee that the
directory where the socket is created hasn't been touched
before (and thus copied the parent to the upper layer).
Added it to runsc integration tests for now. If it turns
out we have lots of these kind of tests, we can consider
moving them somewhere more appropriate.

PiperOrigin-RevId: 251954156
2019-06-06 16:45:51 -07:00
Rahat Mahmood 2d2831e354 Track and export socket state.
This is necessary for implementing network diagnostic interfaces like
/proc/net/{tcp,udp,unix} and sock_diag(7).

For pass-through endpoints such as hostinet, we obtain the socket
state from the backend. For netstack, we add explicit tracking of TCP
states.

PiperOrigin-RevId: 251934850
2019-06-06 15:04:47 -07:00
Michael Pratt 57772db2e7 Shutdown host sockets on internal shutdown
This is required to make the shutdown visible to peers outside the
sandbox.

The readClosed / writeClosed fields were dropped, as they were
preventing a shutdown socket from reading the remainder of queued bytes.
The host syscalls will return the appropriate errors for shutdown.

The control message tests have been split out of socket_unix.cc to make
the (few) remaining tests accessible to testing inherited host UDS,
which don't support sending control messages.

Updates #273

PiperOrigin-RevId: 251763060
2019-06-05 18:40:37 -07:00
Michael Pratt d3ed9baac0 Implement dumpability tracking and checks
We don't actually support core dumps, but some applications want to
get/set dumpability, which still has an effect in procfs.

Lack of support for set-uid binaries or fs creds simplifies things a
bit.

As-is, processes started via CreateProcess (i.e., init and sentryctl
exec) have normal dumpability. I'm a bit torn on whether sentryctl exec
tasks should be dumpable, but at least since they have no parent normal
UID/GID checks should protect them.

PiperOrigin-RevId: 251712714
2019-06-05 14:00:13 -07:00