Commit Graph

4168 Commits

Author SHA1 Message Date
Craig Chi 70cfea2377 Fix comments of TODO issues. 2020-09-16 12:19:30 -07:00
Jinmou Li d459bb3372 Add FUSE umount support
This change implements Release for the FUSE filesystem
and expected behaviors of the FUSE devices.
It includes several checks for aborted connection
in the path for making a request and a function
to abort all the ongoing FUSE requests in order.
2020-09-16 12:19:30 -07:00
Jinmou Li 4edc56d3e9 Fix FUSE_RELEASE protocol reply processing
This commit fixes the potential unexpected errors
of original handling of FUSE_RELEASE responses while
keep the same behavior (ignoring any reply).
2020-09-16 12:19:30 -07:00
Jinmou Li 826a685a95 Improve FUSE async/noreply call logic
This change adds bookkeeping variables for the
FUSE request. With them, old insecure confusing
code we used to process async requests is replaced
by new clear compiling ones. Future code can take
advantage of them to have better control of each
requests.
2020-09-16 12:19:30 -07:00
Jinmou Li f1219ec5f1 Refactor FUSE connection for readability and structure
This change decouples the code that is weakly
tied to the connection struct from connection.go,
rename variables and files with more meaningful choices,
adds detailed comments, explains lock orders,
and adds other minor improvement to make
the existing FUSE code more readable and
more organized.

Purpose is to avoid too much code in one file
and provide better structure for the
future commits.
2020-09-16 12:19:30 -07:00
Craig Chi 2fbbe3b768 Add comments for exported attributes 2020-09-16 12:19:30 -07:00
Craig Chi bf8efe8cdf Implement FUSE_SETATTR
This commit implements FUSE_SETATTR command. When a system call modifies
the metadata of a regular file or a folder by chown(2), chmod(2),
truncate(2), utime(2), or utimes(2), they should be translated to
corresponding FUSE_SETATTR command and sent to the FUSE server.

Fixes #3332
2020-09-16 12:19:30 -07:00
Craig Chi 4181e8c974 Add fh support for revise attr and fstat(2) test
According to Linux 4.4's FUSE behavior, the flags and fh attributes in
FUSE_GETATTR are only used in read, write, and lseek. fstat(2) doesn't
use them either. Add tests to ensure the requests sent from FUSE module
are consistent with Linux's.

Updates #3655
2020-09-16 12:19:30 -07:00
Craig Chi 1146ab6bac Add fuse_fd_util library to include common fuse fd test functions 2020-09-16 12:19:30 -07:00
Craig Chi e63abd82dd Add default attr in fuse_util
fuse_util provides utilities for fuse testing. Add a function to return
a stub fuse_attr struct with specified mode and nodeid.
2020-09-16 12:19:30 -07:00
Jinmou Li 98faed55e6 Implement FUSE_WRITE
This commit adds basic write(2) support for FUSE.
2020-09-16 12:19:30 -07:00
Craig Chi 18f1e1c91b Implement FUSE_CREATE
FUSE_CREATE is called when issuing creat(2) or open(2) with O_CREAT. It
creates a new file on the FUSE filesystem.

Fixes #3825
2020-09-16 12:19:30 -07:00
Jinmou Li 7ed4e46a71 FUSE device: clean up readLocked
This change removes the unnecessary loop and avoids
the recursive call. It also fixes minor bugs in this
function.
2020-09-16 12:19:30 -07:00
Jinmou Li e91c026672 Downgrade FUSE minor version support and clarify comments 2020-09-16 12:19:30 -07:00
Craig Chi 1d8029022e fuse: remove unused marshalling functions
This commit removes unused marshalling functions in linux abi package
and moves self-defined FUSEInitRes wrapper to fuse package.

Updates #3707
2020-09-16 12:19:30 -07:00
Boyuan He & Ridwan Sharif cb9a2a1ad4 fuse: add benchmarking support for FUSE
This change adds the following:
-  Add support for containerizing syscall tests for FUSE
-  Mount tmpfs in the container so we can run benchmarks against it
-  Run the server in a background process
-  benchmarks for fuse syscall

Co-authored-by: Ridwan Sharif <ridwanmsharif@google.com>
2020-09-16 12:19:30 -07:00
Craig Chi 449986264f Support multiple FUSE kernel versions of FUSE_INIT response struct
The fuse_init_out struct changes in different FUSE kernel versions. A
FUSE server may implement older versions of fuse_init_out, but they
share common attributes from the beginning. Implement variable-length
marshallable interface to support older versions of ABI.

Fixes #3707
2020-09-16 12:19:30 -07:00
Craig Chi 983e30c016 Implementing inode.Getlink
kernfs uses inode.Getlink to resolve symlink when look up paths.

Updates #3452
2020-09-16 12:19:30 -07:00
Craig Chi 21cac9dd04 Fix FUSE_READDIR offset issue
According to readdir(3), the offset attribute in struct dirent is the
offset to the next dirent instead of the offset of itself. Send the
successive FUSE_READDIR requests with the offset retrieved from the last
entry.

Updates #3255
2020-09-16 12:19:30 -07:00
Ridwan Sharif d51ddcefdc fuse: use safe go_marshal API for FUSE
Until #3698 is resolved, this change is needed to ensure we're not
corrupting memory anywhere.
2020-09-16 12:19:30 -07:00
Ridwan Sharif 4a5857d644 fuse: Implement IterDirents for directory file description
Fixes #3255.

This change adds support for IterDirents. You can now use `ls` in
the FUSE sandbox.

Co-authored-by: Craig Chi <craigchi@google.com>
2020-09-16 12:19:30 -07:00
Ridwan Sharif bc07df8887 Implement FUSE_RMDIR
Fixes #3587

Co-authored-by: Craig Chi <craigchi@google.com>
2020-09-16 12:19:30 -07:00
Jinmou Li 713400d6b0 Implement FUSE_READ
Fixes #3206
2020-09-16 12:19:30 -07:00
Boyuan He 4d26c9929d Implement FUSE_MKDIR
Fixes #3392
2020-09-16 12:19:30 -07:00
Boyuan He 733d013f97 Implement FUSE_READLINK
Fixes #3316
2020-09-16 12:19:30 -07:00
Boyuan He b50c03b571 Implement FUSE_SYMLINK
Fixes #3452
2020-09-16 12:19:30 -07:00
Boyuan He b53e10f391 Implement FUSE_MKNOD
Fixes #3492
2020-09-16 12:19:30 -07:00
Boyuan He 947088e10a Implement FUSE_RELEASE/RELEASEDIR
Fixes #3314
2020-09-16 12:19:30 -07:00
Boyuan He 32044f94e9 Implement FUSE_OPEN/OPENDIR
Fixes #3174
2020-09-16 12:19:30 -07:00
Andrei Vagin d6ee3ae6d7 Implement FUSE_LOOKUP
Fixes #3231

Co-authored-by: Boyuan He <heboyuan@google.com>
2020-09-16 12:19:30 -07:00
Craig Chi 717b661c45 Add function to create a fake inode in FUSE integration test
Adds a function for the testing thread to set up a fake inode with a
specific path under mount point. After this function is called, each
subsequent FUSE_LOOKUP request with the same path will be served with
the fixed stub response.

Fixes #3539
2020-09-16 12:19:30 -07:00
Craig Chi d928d3c00a Add function generating array of iovec with different FUSE structs
This commit adds a function in the newly created fuse_util library,
which accepts a variable number of arguments and data structures.

Fixes #3609
2020-09-16 12:19:30 -07:00
Craig Chi a289c38626 Add functions in FUSE integration test to get metrics from FUSE server
This commit adds 3 utility functions to ensure all received requests
and preset responses are consumed.

1. Get number of unconsumed requests (received by the FUSE server but
   not consumed by the testing thread).
2. Get number of unsent responses (set by the testing thread but not
   processed by the FUSE server).
3. Get total bytes of the received requests (to ensure some operations
   don't trigger FUSE requests).

Fixes #3607
2020-09-16 12:19:30 -07:00
Craig Chi 15ff2893d9 Extend integration test to test sequence of FUSE operation
Original FUSE integration test has limited capabilities. To test more
situations, the new integration test framework introduces a protocol
to communicate between testing thread and the FUSE server. In summary,
this change includes:

1. Remove CompareResult() and break SetExpected() into
   SetServerResponse() and GetServerActualRequest(). We no longer set
   up an expected request because we want to retrieve the actual FUSE
   request made to the FUSE server and check in the testing thread.

2. Declare a serial buffer data structure to save the received requests
   and expected responses sequentially. The data structure contains a
   cursor to indicate the progress of accessing. This change makes
   sequential SetServerResponse() and GetServerActualRequest() possible.

3. Replace 2 single directional pipes with 1 bi-directional socketpair.
   A protocol which starts with FuseTestCmd is used between the testing
   thread and the FUSE server to provide various functionality.

Fixes #3405
2020-09-16 12:19:30 -07:00
Fabricio Voznika 326a1dbb73 Refactor removed default test dimension
ptrace was always selected as a dimension before, but not
anymore. Some tests were specifying "overlay" expecting that
to be in addition to the default.

PiperOrigin-RevId: 332004111
2020-09-16 07:47:28 -07:00
Rahat Mahmood 9ef1c79922 Rename marshal.Task to marshal.CopyContext.
CopyContext is a better name for the interface because from
go-marshal's perspective, the interface has nothing to do with a
task. A kernel.Task happens to implement the interface, but so can
other things like MemoryManager and IO sequences.

PiperOrigin-RevId: 331959678
2020-09-16 02:10:12 -07:00
Rahat Mahmood d201feb8c5 Enable automated marshalling for the syscall package.
PiperOrigin-RevId: 331940975
2020-09-15 23:38:57 -07:00
Ian Lewis dcd532e2e4 Add support for OCI seccomp filters in the sandbox.
OCI configuration includes support for specifying seccomp filters. In runc,
these filter configurations are converted into seccomp BPF programs and loaded
into the kernel via libseccomp. runsc needs to be a static binary so, for
runsc, we cannot rely on a C library and need to implement the functionality
in Go.

The generator added here implements basic support for taking OCI seccomp
configuration and converting it into a seccomp BPF program with the same
behavior as a program generated by libseccomp.

- New conditional operations were added to pkg/seccomp to support operations
  available in OCI.
- AllowAny and AllowValue were renamed to MatchAny and EqualTo to better reflect
  that syscalls matching the conditionals result in the provided action not
  simply SCMP_RET_ALLOW.
- BuildProgram in pkg/seccomp no longer panics if provided an empty list of
  rules. It now builds a program with the architecture sanity check only.
- ProgramBuilder now allows adding labels that are unused. However, backwards
  jumps are still not permitted.

Fixes #510

PiperOrigin-RevId: 331938697
2020-09-15 23:19:17 -07:00
Ian Lewis c053c4bb03 Fix GitHub issue template.
runsc -v doesn't work. It should be runsc -version

PiperOrigin-RevId: 331911035
2020-09-15 19:49:56 -07:00
Chong Cai cb2e3c946a Implement gvisor verity fs ioctl with GETFLAGS
PiperOrigin-RevId: 331905347
2020-09-15 19:01:59 -07:00
Jamie Liu 8b15effd9e Improve syserror_test.
- It's very difficult to prevent returnErrnoAsError and returnError from being
  optimized out. Instead, replace BenchmarkReturn* with BenchmarkAssign*, which
  store to globalError.

- Compare to a non-nil globalError in BenchmarkCompare* and BenchmarkSwitch*.

New results:
BenchmarkAssignErrno
BenchmarkAssignErrno-12     	1000000000	         0.615 ns/op
BenchmarkAssignError
BenchmarkAssignError-12     	1000000000	         0.626 ns/op
BenchmarkCompareErrno
BenchmarkCompareErrno-12    	1000000000	         0.522 ns/op
BenchmarkCompareError
BenchmarkCompareError-12    	1000000000	         3.54 ns/op
BenchmarkSwitchErrno
BenchmarkSwitchErrno-12     	1000000000	         1.45 ns/op
BenchmarkSwitchError
BenchmarkSwitchError-12     	536315757	        10.9 ns/op

PiperOrigin-RevId: 331875387
2020-09-15 15:59:25 -07:00
Jamie Liu 456c6c33e1 Invert dependency between the context and amutex packages.
This is to allow the syserror package to depend on the context package in a
future change.

PiperOrigin-RevId: 331866252
2020-09-15 15:14:53 -07:00
Dean Deng a004f0d082 Support setting STATX_SIZE for kernfs.InodeAttrs.
Make setting STATX_SIZE a no-op, if it is valid for the given permissions and
file type.

Also update proc tests, which were overfitted before.

Fixes #3842.
Updates #1193.

PiperOrigin-RevId: 331861087
2020-09-15 14:55:28 -07:00
Arthur Sfez 72a30b1148 Move reusable IPv4 test code into a testutil module and refactor it
The refactor aims to simplify the package, by replacing the Go channel with a
PacketBuffer slice.

This code will be reused by tests for IPv6 fragmentation.

PiperOrigin-RevId: 331860411
2020-09-15 14:49:29 -07:00
Nayana Bidari 7f89a26e18 Release FDTable lock before dropping the fds.
This is needed for SO_LINGER, where close() is blocked for linger timeout and
we are holding the FDTable lock for the entire timeout which will not allow
us to create/delete other fds. We have to release the locks and then drop the
fds.

PiperOrigin-RevId: 331844185
2020-09-15 13:43:50 -07:00
Jamie Liu 0d790cbaea Read vfs2 epoll events atomically.
Discovered by ayushranjan@:

VFS2 was employing the following algorithm for fetching ready events from an
epoll instance:
- Create a statically sized EpollEvent slice on the stack of size 16.
- Pass that to EpollInstance.ReadEvents() to populate.
   - EpollInstance.ReadEvents() requeues level-triggered events that it returns
     back into the ready queue.
- Write the results to usermem.
- If the number of results were = 16 then recall EpollInstance.ReadEvents() in
  the hopes of getting more. But this will cause duplication of the "requeued"
  ready level-triggered events.

So if the ready queue has >= 16 ready events, the EpollWait for loop will spin
until it fills the usermem with `maxEvents` events.

Fixes #3521

PiperOrigin-RevId: 331840527
2020-09-15 13:25:58 -07:00
Jamie Liu 86b31a8077 RFC: design for a 9P replacement
Tentatively `lisafs` (LInux SAndbox FileSystem).

PiperOrigin-RevId: 331839246
2020-09-15 13:19:36 -07:00
gVisor bot 84d48c0fdd Merge pull request #3895 from btw616:fix/issue-3894
PiperOrigin-RevId: 331824411
2020-09-15 12:12:28 -07:00
Ghanan Gowripalan d3880b76cb Don't conclude broadcast from route destination
The routing table (in its current) form should not be used to make
decisions about whether a remote address is a broadcast address or
not (for IPv4).

Note, a destination subnet does not always map to a network.
E.g. RouterA may have a route to 192.168.0.0/22 through RouterB,
but RouterB may be configured with 4x /24 subnets on 4 different
interfaces.

See https://github.com/google/gvisor/issues/3938.

PiperOrigin-RevId: 331819868
2020-09-15 11:53:00 -07:00
Tiwei Bie 1adedad81c Fix proc.(*fdDir).IterDirents for VFS2
Currently the returned offset is an index, and we can't
use it to find the next fd to serialize, because getdents
should iterate correctly despite mutation of fds. Instead,
we can return the next fd to serialize plus 2 (which
accounts for "." and "..") as the offset.

Fixes: #3894

Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
2020-09-15 11:12:29 +08:00