Commit Graph

3703 Commits

Author SHA1 Message Date
Rahat Mahmood f23f62c2c2 Correctly set the test VFS environment variable.
Also fix test bugs uncovered now that they aren't silently skipped on
VFS2.

Updates #1487.

PiperOrigin-RevId: 316415807
2020-06-15 01:09:38 -07:00
Ian Gudger 3b5eaad3c8 Allow reading IP_MULTICAST_LOOP and IP_MULTICAST_TTL on TCP sockets.
I am not really sure what the point of this is, but someone filed a bug about
it, so I assume something relies on it.

PiperOrigin-RevId: 316225127
2020-06-12 20:49:02 -07:00
Tony Gong 82313667ea Make GenerateRandomPayload available to all tests
Moved the function for generating a payload of random byets of a specified
length into the testbench package so that it's availbale for all tests to use.

Added a test case to the IPv4 ID uniqueness test which uses a payload length
of 512 bytes. This test case passes for gVisor currently, whereas the test case
with a small payload of 11 bytes fails because gVisor only assigns the ID field
if the IP payload is sufficiently large.

PiperOrigin-RevId: 316185097
2020-06-12 15:07:17 -07:00
Andrei Vagin 6ec9d60403 vfs2: implement fcntl(fd, F_SETFL, flags)
PiperOrigin-RevId: 316148074
2020-06-12 11:58:15 -07:00
Mithun Iyer 61d6c059ac Replace use of %v in packetimpact tests
PiperOrigin-RevId: 316027588
2020-06-11 19:46:49 -07:00
Ian Lewis 8ea99d58ff Set the HOME environment variable for sub-containers.
Fixes #701

PiperOrigin-RevId: 316025635
2020-06-11 19:31:24 -07:00
Adin Scannell 5a894e35a0 Remove generated logs when test succeeds.
PiperOrigin-RevId: 316022884
2020-06-11 19:05:18 -07:00
Jamie Liu 77c206e371 Add //pkg/sentry/fsimpl/overlay.
Major differences from existing overlay filesystems:

- Linux allows lower layers in an overlay to require revalidation, but not the
  upper layer. VFS1 allows the upper layer in an overlay to require
  revalidation, but not the lower layer. VFS2 does not allow any layers to
  require revalidation. (Now that vfs.MkdirOptions.ForSyntheticMountpoint
  exists, no uses of overlay in VFS1 are believed to require upper layer
  revalidation; in particular, the requirement that the upper layer support the
  creation of "trusted." extended attributes for whiteouts effectively required
  the upper filesystem to be tmpfs in most cases.)

- Like VFS1, but unlike Linux, VFS2 overlay does not attempt to make mutations
  of the upper layer atomic using a working directory and features like
  RENAME_WHITEOUT. (This may change in the future, since not having a working
  directory makes error recovery for some operations, e.g. rmdir, particularly
  painful.)

- Like Linux, but unlike VFS1, VFS2 represents whiteouts using character
  devices with rdev == 0; the equivalent of the whiteout attribute on
  directories is xattr trusted.overlay.opaque = "y"; and there is no equivalent
  to the whiteout attribute on non-directories since non-directories are never
  merged with lower layers.

- Device and inode numbers work as follows:

    - In Linux, modulo the xino feature and a special case for when all layers
      are the same filesystem:

        - Directories use the overlay filesystem's device number and an
          ephemeral inode number assigned by the overlay.

        - Non-directories that have been copied up use the device and inode
          number assigned by the upper filesystem.

        - Non-directories that have not been copied up use a per-(overlay,
          layer)-pair device number and the inode number assigned by the lower
          filesystem.

    - In VFS1, device and inode numbers always come from the lower layer unless
      "whited out"; this has the adverse effect of requiring interaction with
      the lower filesystem even for non-directory files that exist on the upper
      layer.

    - In VFS2, device and inode numbers are assigned as in Linux, except that
      xino and the samefs special case are not supported.

- Like Linux, but unlike VFS1, VFS2 does not attempt to maintain memory mapping
  coherence across copy-up. (This may have to change in the future, as users
  may be dependent on this property.)

- Like Linux, but unlike VFS1, VFS2 uses the overlayfs mounter's credentials
  when interacting with the overlay's layers, rather than the caller's.

- Like Linux, but unlike VFS1, VFS2 permits multiple lower layers in an
  overlay.

- Like Linux, but unlike VFS1, VFS2's overlay filesystem is
  application-mountable.

Updates #1199

PiperOrigin-RevId: 316019067
2020-06-11 18:34:53 -07:00
Ian Gudger dc4e0157ef Add test for reordering.
Tests the effect of reordering on retransmission and window size.

Test covers the expected behavior of both Linux and netstack, however, netstack
does not behave as expected. Further, the current expected behavior of netstack
is not ideal and should be adjusted in the future.

PiperOrigin-RevId: 316015184
2020-06-11 18:04:43 -07:00
Fabricio Voznika fbe41987c9 Enable nogo again
PiperOrigin-RevId: 316011323
2020-06-11 17:34:02 -07:00
Ghanan Gowripalan 4c0a8bdaf5 Do not use tentative addresses for routes
Tentative addresses should not be used when finding a route. This change
fixes a bug where a tentative address may have been used.

Test: stack_test.TestDADResolve
PiperOrigin-RevId: 315997624
2020-06-11 16:09:45 -07:00
gVisor bot 4f111b6384 Merge pull request #2863 from lubinszARM:pr_sndbuf
PiperOrigin-RevId: 315991648
2020-06-11 15:36:43 -07:00
Fabricio Voznika d58d57606a Don't copy structs with sync.Mutex during initialization
During inititalization inode struct was copied around, but
it isn't great pratice to copy it around since it contains
ref count and sync.Mutex.

Updates #1480

PiperOrigin-RevId: 315983788
2020-06-11 14:56:19 -07:00
gVisor bot 11dc95e6c5 Merge pull request #2705 from lubinszARM:pr_sigfp_nested
PiperOrigin-RevId: 315979564
2020-06-11 14:35:46 -07:00
gVisor bot 13f2664cfe Merge pull request #2946 from avagin:travis-pipefail
PiperOrigin-RevId: 315972822
2020-06-11 14:04:26 -07:00
gVisor bot 44575bf726 Refactor packetimpact Connection types
Reorganize the Connection types such that the defined types no longer expose
the lower-level functions SendFrame and CreateFrame. These methods are still
exported on the underlying Connection type, and thus can be accessed via a
type-cast. In future, defined types should have one or more type-safe versions
of the send() method on Connection, e.g. UDPIPv4 has Send() which allows the UDP
header to be overridden and SendIP() which allows both the IPv4 and UDP headers
to be modified.

testbench.Connection gets a SendFrameStateless method which sends frames
without updating the state of any of the layers. This should be used when
sending out-of-band control messages such as ICMP messages, as using the
normal Send method can result in errors when attempting to update the TCP
state using an ICMP packet.

Also remove the localAddr field of testbench.Connection and instead compute
it on the fly as needed for UDPIPv4 and TCPIPv4.

PiperOrigin-RevId: 315969714
2020-06-11 13:48:30 -07:00
gVisor bot af6ec7b734 Add Generate method in merkletree
A method is added to generate a merkle tree for data, and store the
generated tree in the output.

PiperOrigin-RevId: 315966571
2020-06-11 13:33:09 -07:00
gVisor bot d2cc9a888e Factor out flipcall mmap for internal use
PiperOrigin-RevId: 315959279
2020-06-11 12:57:56 -07:00
gVisor bot b39cc6a800 Add merkle tree size measure
This change creates a merkletree package which will be used in the future
for an implementation of file system API.

PiperOrigin-RevId: 315952451
2020-06-11 12:23:25 -07:00
Andrei Vagin c45420a3f5 travis: set the pipefail option
The travis job has to fail if make smoke-test fails.

Reported-by: Bin Lu <bin.lu@arm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2020-06-11 11:26:22 -07:00
Michael Pratt 0c7a5bc69c Add nogo TODO.
PiperOrigin-RevId: 315911025
2020-06-11 09:13:01 -07:00
Ian Gudger a085e562d0 Add support for SO_REUSEADDR to UDP sockets/endpoints.
On UDP sockets, SO_REUSEADDR allows multiple sockets to bind to the same
address, but only delivers packets to the most recently bound socket. This
differs from the behavior of SO_REUSEADDR on TCP sockets. SO_REUSEADDR for TCP
sockets will likely need an almost completely independent implementation.

SO_REUSEADDR has some odd interactions with the similar SO_REUSEPORT. These
interactions are tested fairly extensively and all but one particularly odd
one (that honestly seems like a bug) behave the same on gVisor and Linux.

PiperOrigin-RevId: 315844832
2020-06-10 23:49:26 -07:00
Nicolas Lacasse a87c74bc54 Remove duplicate colon from warning log.
doAction()->log.TracebackAll() will append a colon.

PiperOrigin-RevId: 315842611
2020-06-10 23:23:07 -07:00
Bin Lu cbfb537eef Deleting the maxSendBufferSize from fs/host
When I do high-performance networking,
the value of wmem_max is often set very high,
specially for 10/25/50 Gigabit NIC.

I think maybe this restriction is not suitable.

Signed-off-by: Bin Lu <bin.lu@arm.com>
2020-06-10 21:53:46 -04:00
gVisor bot b436b9717e Merge pull request #2711 from lubinszARM:pr_mmio
PiperOrigin-RevId: 315812219
2020-06-10 18:51:41 -07:00
gVisor bot 508e7c3a79 Merge pull request #2763 from gaurav1086:sentry_kernel_timekeeper_use_buffered_channel
PiperOrigin-RevId: 315803553
2020-06-10 17:43:16 -07:00
Fabricio Voznika ab4c851893 Cgroup fixes
- Set hugetlb related fields
- Add realtime scheduler related fields
- Beef up unit tests

Updates #2713

PiperOrigin-RevId: 315797979
2020-06-10 17:05:23 -07:00
Fabricio Voznika 41d9e536d5 Skip pids.max if value is zero
LinuxPids.Limit is the only optional cgroup field in OCI that
is not a pointer. If value is 0 or negative it should be
skipped.

PiperOrigin-RevId: 315791909
2020-06-10 16:30:06 -07:00
Nayana Bidari 9338854ea3 Fix the error code for syscall test with null TOS.
The setsockopt with nullptr can fail with either EFAULT or zero.

PiperOrigin-RevId: 315777107
2020-06-10 15:06:45 -07:00
Nayana Bidari 4b9652d63b {S,G}etsockopt for TCP_KEEPCNT option.
TCP_KEEPCNT is used to set the maximum keepalive probes to be
sent before dropping the connection.

WANT_LGTM=jchacon
PiperOrigin-RevId: 315758094
2020-06-10 13:37:27 -07:00
Andrei Vagin a5a4f80487 socket/unix: handle sendto address argument for connected sockets
In case of SOCK_SEQPACKET, it has to be ignored.
In case of SOCK_STREAM, EISCONN or EOPNOTSUPP has to be returned.

PiperOrigin-RevId: 315755972
2020-06-10 13:26:54 -07:00
Michael Pratt fadbfd83d9 Include panic message in log
PiperOrigin-RevId: 315745386
2020-06-10 12:38:20 -07:00
gVisor bot 6d43ac9574 Merge pull request #2787 from lubinszARM:pr_race_time
PiperOrigin-RevId: 315734425
2020-06-10 11:45:16 -07:00
Tamir Duberstein f004bb8703 Remove duplicate and incorrect size check
Minimum header sizes are already checked in each `case` arm below. Worse, the
ICMP entries in transportProtocolMinSizes are incorrect, and produce false "raw
packet" logs.

PiperOrigin-RevId: 315730073
2020-06-10 11:25:58 -07:00
Tamir Duberstein 9d2b2c121f Replace use of %v in sniffer
PiperOrigin-RevId: 315711208
2020-06-10 10:02:41 -07:00
Fabricio Voznika 203dc121f6 Redirect TODOs to more specific issues
Closes #1623

PiperOrigin-RevId: 315681993
2020-06-10 06:51:32 -07:00
Gaurav Singh f1f85f475d sentry: use defer wg.Done() unconditionally
Signed-off-by: Gaurav Singh <gaurav1086@gmail.com>
2020-06-09 22:56:39 -04:00
Fabricio Voznika 67565078bb Implement flock(2) in VFS2
LockFD is the generic implementation that can be embedded in
FileDescriptionImpl implementations. Unique lock ID is
maintained in vfs.FileDescription and is created on demand.

Updates #1480

PiperOrigin-RevId: 315604825
2020-06-09 18:46:42 -07:00
gVisor bot 52c922f7c0 Merge pull request #2712 from lubinszARM:pr_sigfp_init
PiperOrigin-RevId: 315599736
2020-06-09 18:05:01 -07:00
gVisor bot e3cbfbf349 Merge pull request #2907 from lubinszARM:pr_minor
PiperOrigin-RevId: 315595602
2020-06-09 17:34:41 -07:00
gVisor bot 4950ccde75 Fix write hang bug found by syzkaller.
After this change e.mu is only promoted to exclusively locked during
route.Resolve. It downgrades back to read-lock afterwards.
This prevents the second RLock() call gets stuck later in the stack.

https://syzkaller.appspot.com/bug?id=065b893bd8d1d04a4e0a1d53c578537cde1efe99

Syzkaller logs does not contain interesting stack traces.
The following stack trace is obtained by running repro locally.

goroutine 53 [semacquire, 3 minutes]:
runtime.gopark(0xfd4278, 0x1896320, 0xc000301912, 0x4)
        GOROOT/src/runtime/proc.go:304 +0xe0 fp=0xc0000e25f8 sp=0xc0000e25d8 pc=0x437170
runtime.goparkunlock(...)
        GOROOT/src/runtime/proc.go:310
runtime.semacquire1(0xc0001220b0, 0xc00000a300, 0x1, 0x0)
        GOROOT/src/runtime/sema.go:144 +0x1c0 fp=0xc0000e2660 sp=0xc0000e25f8 pc=0x4484e0
sync.runtime_Semacquire(0xc0001220b0)
        GOROOT/src/runtime/sema.go:56 +0x42 fp=0xc0000e2690 sp=0xc0000e2660 pc=0x448132
gvisor.dev/gvisor/pkg/sync.(*RWMutex).RLock(...)
        pkg/sync/rwmutex_unsafe.go:76
gvisor.dev/gvisor/pkg/tcpip/transport/udp.(*endpoint).HandleControlPacket(0xc000122000, 0x7ee5, 0xc00053c16c, 0x4, 0x5e21, 0xc00053c224, 0x4, 0x1, 0x0, 0xc00007ed00)
        pkg/tcpip/transport/udp/endpoint.go:1345 +0x169 fp=0xc0000e26d8 sp=0xc0000e2690 pc=0x9843f9
......
gvisor.dev/gvisor/pkg/tcpip/transport/udp.(*protocol).HandleUnknownDestinationPacket(0x18bb5a0, 0xc000556540, 0x5e21, 0xc00053c16c, 0x4, 0x7ee5, 0xc00053c1ec, 0x4, 0xc00007e680, 0x4)
        pkg/tcpip/transport/udp/protocol.go:143 +0xb9a fp=0xc0000e8260 sp=0xc0000e7510 pc=0x9859ba
......
gvisor.dev/gvisor/pkg/tcpip/transport/udp.sendUDP(0xc0001220d0, 0xc00053ece0, 0x1, 0x1, 0x883, 0x1405e217ee5, 0x11100a0, 0xc000592000, 0xf88780)
        pkg/tcpip/transport/udp/endpoint.go:924 +0x3b0 fp=0xc0000ed390 sp=0xc0000ec750 pc=0x981af0
gvisor.dev/gvisor/pkg/tcpip/transport/udp.(*endpoint).write(0xc000122000, 0x11104e0, 0xc00020a460, 0x0, 0x0, 0x0, 0x0, 0x0)
        pkg/tcpip/transport/udp/endpoint.go:510 +0x4ad fp=0xc0000ed658 sp=0xc0000ed390 pc=0x97f2dd

PiperOrigin-RevId: 315590041
2020-06-09 17:02:14 -07:00
Fabricio Voznika 6722b1e56f Don't WriteOut to readonly mounts
When the file closes, it attempts to write dirty cached
attributes to the file. This should not be done when the
mount is readonly.

PiperOrigin-RevId: 315585058
2020-06-09 16:36:56 -07:00
Fabricio Voznika 0ae5bd24d7 Mount root and volumes as read-only if --overlay is enabled
PiperOrigin-RevId: 315583963
2020-06-09 16:31:38 -07:00
gVisor bot ea2c11fcf8 Merge pull request #2885 from google:dependabot/bundler/benchmarks/workloads/ruby_template/puma-3.12.6
PiperOrigin-RevId: 315558580
2020-06-09 14:21:35 -07:00
Ghanan Gowripalan 2d3b9d18e7 Handle removed NIC in NDP timer for packet tx
NDP packets are sent periodically from NDP timers. These timers do not
hold the NIC lock when sending packets as the packet write operation
may take some time. While the lock is not held, the NIC may be removed
by some other goroutine. This change handles that scenario gracefully.

Test: stack_test.TestRemoveNICWhileHandlingRSTimer
PiperOrigin-RevId: 315524143
2020-06-09 11:33:20 -07:00
Jamie Liu ecff24930c Ensure pgalloc.MemoryFile.fileSize is always chunk-aligned.
findAvailableLocked() may return a non-aligned FileRange.End after expansion
since it may round FileRange.Start down to a hugepage boundary.

PiperOrigin-RevId: 315520321
2020-06-09 11:16:05 -07:00
Kevin Krakauer 20afd66e01 Invoke bazel query via bash function.
PiperOrigin-RevId: 315514034
2020-06-09 10:48:24 -07:00
Bin Lu b792cc1ca9 minor change in kvm module for Arm64
Signed-off-by: Bin Lu <bin.lu@arm.com>
2020-06-09 05:42:41 -04:00
Bin Lu 5ad1025d2d initialize an empty fp state area for sentry on Arm64
We need to initialize an empty fp state area for the sentry.

Signed-off-by: Bin Lu <bin.lu@arm.com>
2020-06-09 05:01:03 -04:00
Fabricio Voznika 4e96b94915 Combine executable lookup code
Run vs. exec, VFS1 vs. VFS2 were executable lookup were
slightly different from each other. Combine them all
into the same logic.

PiperOrigin-RevId: 315426443
2020-06-08 23:08:23 -07:00