Commit Graph

3553 Commits

Author SHA1 Message Date
Dean Deng e8f1a5c1f6 Port GETOWN, SETOWN fcntls to vfs2.
Also make some fixes to vfs1's F_SETOWN. The fcntl test now entirely passes
on vfs2.

Fixes #2920.

PiperOrigin-RevId: 318669529
2020-06-27 21:33:37 -07:00
Dean Deng 02d552d07c Support sticky bit in vfs2.
Updates #2923.

PiperOrigin-RevId: 318648128
2020-06-27 14:39:41 -07:00
Dean Deng 691c04278e Add documentation for vfs2 inotify.
Updates #1479.

PiperOrigin-RevId: 318631247
2020-06-27 09:20:04 -07:00
Ian Gudger bab3c36efb Add style guide.
PiperOrigin-RevId: 318591900
2020-06-26 21:10:37 -07:00
Dean Deng 85be13d9a3 Add tests for eventfd/timerfd/inotify operations that should return ESPIPE.
PiperOrigin-RevId: 318585377
2020-06-26 19:42:27 -07:00
Kevin Krakauer 66d1665441 IPv6 raw sockets. Needed for ip6tables.
IPv6 raw sockets never include the IPv6 header.

PiperOrigin-RevId: 318582989
2020-06-26 19:07:02 -07:00
gVisor bot 8dbeac53ce Implement SO_NO_CHECK socket option.
SO_NO_CHECK is used to skip the UDP checksum generation on a TX socket
(UDP checksum is optional on IPv4).

Test:
 - TestNoChecksum
 - SoNoCheckOffByDefault (UdpSocketTest)
 - SoNoCheck (UdpSocketTest)

Fixes #3055

PiperOrigin-RevId: 318575215
2020-06-26 17:51:04 -07:00
Zeling Feng aed7183976 Packetimpact test for IPv6 unknown options action
The Option Type identifiers are internally encoded such that their
highest-order two bits specify the action that must be taken if the
processing IPv6 node does not recognize the Option Type:

  00 - skip over this option and continue processing the header.
  01 - discard the packet.
  10 - discard the packet and, regardless of whether or not the
       packet's Destination Address was a multicast address, send an
       ICMP Parameter Problem, Code 2, message to the packet's
       Source Address, pointing to the unrecognized Option Type.
  11 - discard the packet and, only if the packet's Destination
       Address was not a multicast address, send an ICMP Parameter
       Problem, Code 2, message to the packet's Source Address,
       pointing to the unrecognized Option Type.

PiperOrigin-RevId: 318566613
2020-06-26 16:43:53 -07:00
Kevin Krakauer 9cfc154975 Require CAP_SYS_ADMIN in the root user namespace for TTY theft
PiperOrigin-RevId: 318563543
2020-06-26 16:24:39 -07:00
Zeling Feng e6a90baef1 Support IPv6 extension headers in packetimpact tests.
- IPv6 HopByHop Options Extension Header
- IPv6 Destination Options Extension Header

PiperOrigin-RevId: 318551425
2020-06-26 15:08:51 -07:00
Dean Deng 54a31e219c Support inotify IN_ONESHOT.
Also, while we're here, make sure that gofer inotify events are generated when
files are created in remote revalidating mode.

Updates #1479.

PiperOrigin-RevId: 318536354
2020-06-26 13:47:48 -07:00
gVisor bot cfd049da87 Merge pull request #2931 from ridwanmsharif:ridwanmsharif/fuse-char-device
PiperOrigin-RevId: 318511615
2020-06-26 11:37:23 -07:00
Kevin Krakauer 47ac142dc9 Enable docker IPv6 in kokoro
We need docker IPv6 for ip6tables tests. Docker has spotty IPv6 support, so just
enabling IPv6 in daemon.json breaks the Ruby image test (AFAICT it breaks
anything that tries to use IPv6 to reach the internet). An ip6tables NAT rule is
added to handle this.

We could make these changes as part of scripts/, but these would overwrite
global values and possibly break users' systems.

PiperOrigin-RevId: 318508209
2020-06-26 11:19:12 -07:00
Kevin Krakauer 7fb6cc286f conntrack refactor, no behavior changes
- Split connTrackForPacket into 2 functions instead of switching on flag
- Replace hash with struct keys.
- Remove prefixes where possible
- Remove unused connStatus, timeout
- Flatten ConnTrack struct a bit - some intermediate structs had no meaning
  outside of the context of their parent.
- Protect conn.tcb with a mutex
- Remove redundant error checking (e.g. when is pkt.NetworkHeader valid)
- Clarify that HandlePacket and CreateConnFor are the expected entrypoints for
  ConnTrack

PiperOrigin-RevId: 318407168
2020-06-25 21:21:57 -07:00
Tamir Duberstein 4069461877 Avoid an allocation in epoll
PiperOrigin-RevId: 318346153
2020-06-25 14:18:33 -07:00
Ridwan Sharif 2828806fb0 Test that the fuse device can be opened 2020-06-25 15:46:30 -04:00
Ridwan Sharif a63db7d903 Moved FUSE device under the fuse directory 2020-06-25 14:22:21 -04:00
Ridwan Sharif bd5f0e2dc4 Add FUSE character device
This change adds a FUSE character device backed by devtmpfs. This
device will be used to establish a connection between the FUSE
server daemon and fusefs. The FileDescriptionImpl methods will
be implemented as we flesh out fusefs some more. The tests assert
that the device can be opened and used.
2020-06-25 14:22:21 -04:00
Adin Scannell 00ee5abaa7 Drop unused markdown links.
PiperOrigin-RevId: 318284693
2020-06-25 09:18:30 -07:00
Dean Deng b5e814445a Fix procfs bugs in vfs2.
- Support writing on proc/[pid]/{uid,gid}map
- Return EIO for writing to static files.

Updates #2923.

PiperOrigin-RevId: 318188503
2020-06-24 19:22:12 -07:00
gVisor bot ac6f7b600b Internal change.
PiperOrigin-RevId: 318180382
2020-06-24 18:14:07 -07:00
Nicolas Lacasse 58880bf551 Port /dev/net/tun device to VFS2.
Updates #2912 #1035

PiperOrigin-RevId: 318162565
2020-06-24 16:23:44 -07:00
Tamir Duberstein 10930b0f8c Remove waiter.Entry.Context
This field is redundant since state can be stored in the callback.

PiperOrigin-RevId: 318134855
2020-06-24 13:56:38 -07:00
Bhasker Hariharan b070e218c6 Add support for Stack level options.
Linux controls socket send/receive buffers using a few sysctl variables
  - net.core.rmem_default
  - net.core.rmem_max
  - net.core.wmem_max
  - net.core.wmem_default
  - net.ipv4.tcp_rmem
  - net.ipv4.tcp_wmem

The first 4 control the default socket buffer sizes for all sockets
raw/packet/tcp/udp and also the maximum permitted socket buffer that can be
specified in setsockopt(SOL_SOCKET, SO_(RCV|SND)BUF,...).

The last two control the TCP auto-tuning limits and override the default
specified in rmem_default/wmem_default as well as the max limits.

Netstack today only implements tcp_rmem/tcp_wmem and incorrectly uses it
to limit the maximum size in setsockopt() as well as uses it for raw/udp
sockets.

This changelist introduces the other 4 and updates the udp/raw sockets to use
the newly introduced variables. The values for min/max match the current
tcp_rmem/wmem values and the default value buffers for UDP/RAW sockets is
updated to match the linux value of 212KiB up from the really low current value
of 32 KiB.

Updates #3043
Fixes #3043

PiperOrigin-RevId: 318089805
2020-06-24 10:24:20 -07:00
Adin Scannell 364ac92baf Support for saving pointers to fields in the state package.
Previously, it was not possible to encode/decode an object graph which
contained a pointer to a field within another type. This was because the
encoder was previously unable to disambiguate a pointer to an object and a
pointer within the object.

This CL remedies this by constructing an address map tracking the full memory
range object occupy. The encoded Refvalue message has been extended to allow
references to children objects within another object. Because the encoding
process may learn about object structure over time, we cannot encode any
objects under the entire graph has been generated.

This CL also updates the state package to use standard interfaces intead of
reflection-based dispatch in order to improve performance overall. This
includes a custom wire protocol to significantly reduce the number of
allocations and take advantage of structure packing.

As part of these changes, there are a small number of minor changes in other
places of the code base:

* The lists used during encoding are changed to use intrusive lists with the
  objectEncodeState directly, which required that the ilist Len() method is
  updated to work properly with the ElementMapper mechanism.

* A bug is fixed in the list code wherein Remove() called on an element that is
  already removed can corrupt the list (removing the element if there's only a
  single element). Now the behavior is correct.

* Standard error wrapping is introduced.

* Compressio was updated to implement the new wire.Reader and wire.Writer
  inteface methods directly. The lack of a ReadByte and WriteByte caused issues
  not due to interface dispatch, but because underlying slices for a Read or
  Write call through an interface would always escape to the heap!

* Statify has been updated to support the new APIs.

See README.md for a description of how the new mechanism works.

PiperOrigin-RevId: 318010298
2020-06-23 23:34:06 -07:00
Dean Deng 399c52888d Resolve remaining inotify TODOs.
Also refactor HandleDeletion().

Updates #1479.

PiperOrigin-RevId: 317989000
2020-06-23 20:05:28 -07:00
Dean Deng 2189e0a660 Clean up hostfs TODOs.
This CL does a handful of things:
- Support O_DSYNC, O_SYNC
- Support O_APPEND and document an unavoidable race condition
- Ignore O_DIRECT; we probably don't want to allow applications to set O_DIRECT
  on the host fd itself.
- Leave a TODO for supporting O_NONBLOCK, which is a simple fix once RWF_NOWAIT
  is supported.
- Get rid of caching TODO; force_page_cache is not configurable for host fs in
  vfs1 or vfs2 after whitelist fs was removed.
- For the remaining TODOs, link to more specific bugs.

Fixes #1672.

PiperOrigin-RevId: 317985269
2020-06-23 19:24:43 -07:00
Ian Gudger 2141013dce Add support for SO_REUSEADDR to TCP sockets/endpoints.
For TCP sockets, SO_REUSEADDR relaxes the rules for binding addresses.

gVisor/netstack already supported a behavior similar to SO_REUSEADDR, but did
not allow disabling it. This change brings the SO_REUSEADDR behavior closer to
the behavior implemented by Linux and adds a new SO_REUSEADDR disabled
behavior. Like Linux, SO_REUSEADDR is now disabled by default.

PiperOrigin-RevId: 317984380
2020-06-23 19:15:38 -07:00
Nicolas Lacasse 0f328beb0d Port /dev/tty device to VFS2.
Support is limited to the functionality that exists in VFS1.

Updates #2923 #1035

PiperOrigin-RevId: 317981417
2020-06-23 18:48:37 -07:00
Dean Deng 65a587dedf Complete inotify IN_EXCL_UNLINK implementation in VFS2.
Events were only skipped on parent directories after their children were
unlinked; events on the unlinked file itself need to be skipped as well.

As a result, all Watches.Notify() calls need to know whether the dentry where
the call came from was unlinked.

Updates #1479.

PiperOrigin-RevId: 317979476
2020-06-23 18:33:04 -07:00
Ting-Yu Wang acf519a77b Nit fix: Create and use a std::string object for `const char*`.
PiperOrigin-RevId: 317973144
2020-06-23 17:44:34 -07:00
Dean Deng 0c628c3152 Support inotify in vfs2 gofer fs.
Because there is no inode structure stored in the sandbox, inotify watches
must be held on the dentry. This would be an issue in the presence of hard
links, where multiple dentries would need to share the same set of watches,
but in VFS2, we do not support the internal creation of hard links on gofer
fs. As a result, we make the assumption that every dentry corresponds to a
unique inode.

Furthermore, dentries can be cached and then evicted, even if the underlying
file has not be deleted. We must prevent this from occurring if there are any
watches that would be lost. Note that if the dentry was deleted or invalidated
(d.vfsd.IsDead()), we should still destroy it along with its watches.

Additionally, when a dentry’s last watch is removed, we cache it if it also
has zero references. This way, the dentry can eventually be evicted from
memory if it is no longer needed. This is accomplished with a new dentry
method, OnZeroWatches(), which is called by Inotify.RmWatch and
Inotify.Release. Note that it must be called after all inotify locks are
released to avoid violating lock order. Stress tests are added to make sure
that inotify operations don't deadlock with gofer.OnZeroWatches.

Updates #1479.

PiperOrigin-RevId: 317958034
2020-06-23 16:14:56 -07:00
Ting-Yu Wang 793edf4cb4 Deflake proc test: Don't fail on DT_UNKNOWN.
Per manual page:
"All applications must properly handle a return of DT_UNKNOWN."

PiperOrigin-RevId: 317957013
2020-06-23 16:07:26 -07:00
Nicolas Lacasse edea9a8d02 Port readahead to VFS2.
It preserves the same functionality (almost none) as in VFS1.

Updates #2923 #1035

PiperOrigin-RevId: 317943522
2020-06-23 14:53:33 -07:00
gVisor bot ca1bc46f15 Internal change.
PiperOrigin-RevId: 317941748
2020-06-23 14:42:15 -07:00
gVisor bot 44dd65ce1e Merge pull request #2272 from lubinszARM:pr_serr_injection
PiperOrigin-RevId: 317933650
2020-06-23 13:53:56 -07:00
Dean Deng 38d7b2fe56 Only allow regular files, sockets, pipes, and char devices to be imported.
PiperOrigin-RevId: 317796028
2020-06-22 21:30:54 -07:00
Ayush Ranjan c9e6978062 Fix the way PR build clones gVisor.
Copybara force-pushes to the PR immediately before merging which triggers a PR
build. Since the PR is merged, the refspec +refs/pull/{pr_num}/merge is not
available and the build fails causing all master commit CI builds to show a
failure.

This change remove the clone step from travis and clone manually in a way
which always would succeed. We fetch +refs/pull/{pr_num}/head and cherry pick
that onto the target branch.

I have tested this in https://github.com/ayushr2/gvisor/pull/1 and
https://github.com/ayushr2/gvisor/pull/2.

PiperOrigin-RevId: 317759891
2020-06-22 16:36:52 -07:00
Dean Deng 4573e7d863 Check for invalid trailing / when traversing path in gofer OpenAt.
Updates #2923.

PiperOrigin-RevId: 317700049
2020-06-22 11:39:41 -07:00
Bruno Dal Bo 282a6aea1b Extract common nested LinkEndpoint pattern
... and unify logic for detached netsted endpoints.

sniffer.go caused crashes if a packet delivery is attempted when the dispatcher
is nil.

Extracted the endpoint nesting logic into a common composable type so it can be
used by the Fuchsia Netstack (the pattern is widespread there).

PiperOrigin-RevId: 317682842
2020-06-22 10:31:45 -07:00
Nicolas Lacasse a480b4faf4 Allow readdir(/proc/[tid]/net) to return EINVAL on a zombie task.
Despite what the man page says, linux will return EINVAL when calling
getdents() an a /proc/[tid]/net file corresponding to a zombie task. This
causes readdir() to return a null pointer AND errno=EINVAL.

See fs/proc/proc_net.c:proc_tgid_net_readdir() for where this occurs.

We have tests that recursively read /proc, and are likely to hit this when
running natively, so we must catch and handle this case.

PiperOrigin-RevId: 317674168
2020-06-22 09:54:22 -07:00
Dean Deng 00928d142d Fix vfs2 extended attributes.
Correct behavior when given zero size arguments and trying to set user.* xattrs
on files other than regular files or directories.

Updates #2923.

PiperOrigin-RevId: 317590409
2020-06-21 21:48:19 -07:00
Dean Deng f46f4a2af9 Enable passing vfs2 tests.
I forgot to update getdents earlier. Several thousand runs of the fsync and
proc_net_unix tests all passed as well.

Updates #2923.

PiperOrigin-RevId: 317415488
2020-06-19 18:27:29 -07:00
Dean Deng ad9f469174 Fix bugs in vfs2 to make symlink tests pass.
- Return ENOENT if target path is empty.
- Make sure open(2) with O_CREAT|O_EXCL returns EEXIST when necessary.
- Correctly update atime in tmpfs using touchATime().

Updates #2923.

PiperOrigin-RevId: 317382655
2020-06-19 14:41:14 -07:00
Nicolas Lacasse f0feada89c Use internal tmpfs in test runner, even when running with overlay.
PiperOrigin-RevId: 317377571
2020-06-19 14:14:21 -07:00
Dean Deng 8655fb7248 Fix vfs2 proc/self/fd dirent iteration.
Make proc/self/fd iteration work properly. Also, the comment on
kernfs.Inode.IterDirents did not accurately reflect how parameters should be
used/were used in kernfs.Inode impls other than fdDir.

Updates #2923.

PiperOrigin-RevId: 317370325
2020-06-19 13:36:18 -07:00
Dean Deng 7db196c4db Port fadvise64 to vfs2.
Like vfs1, we have a trivial implementation that ignores all valid advice.

Updates #2923.

PiperOrigin-RevId: 317349505
2020-06-19 11:50:09 -07:00
gVisor bot d962f9f384 Implement UDP cheksum verification.
Test:
 - TestIncrementChecksumErrors

Fixes #2943

PiperOrigin-RevId: 317348158
2020-06-19 11:43:20 -07:00
Dean Deng a609fff9d1 Fix vfs2 handling of preadv2/pwritev2 flags.
Check for unsupported flags, and silently support RWF_HIPRI by doing nothing.
From pkg/abi/linux/file.go: "gVisor does not implement the RWF_HIPRI feature,
but the flag is accepted as a valid flag argument for preadv2/pwritev2."

Updates #2923.

PiperOrigin-RevId: 317330631
2020-06-19 10:19:46 -07:00
Dean Deng f40d023ad6 Don't adjust parent link count if we replace a child dir with another.
Updates #2923.

PiperOrigin-RevId: 317314460
2020-06-19 08:45:40 -07:00