This prevents setting stale errno on responses.
Also fixes TestDiscardsUDPPacketsWithMcastSourceAddressV6 to use correct
multicast addresses in test.
Fixes#3793
PiperOrigin-RevId: 329391155
Some syscall tests, namely uname_test_* modify the host and domain
name, which modifies the execution environment and can have unintended
consequences on other tests. For example, modifying the hostname
causes some networking tests to fail DNS lookups. Run all syscall
tests in their own uts namespaces to isolate these changes.
PiperOrigin-RevId: 329348127
Currently the logs produce
TestOne: packetimpact_test.go:182: listing devices on ... container: process terminated with status: 126
which is not actionable; presumably the `ip` command output is interesting.
PiperOrigin-RevId: 329032105
An earlier change considered the loopback bound to all addresses in an
assigned subnet. This should have only be done for IPv4 to maintain
compatability with Linux:
```
$ ip addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group ...
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
$ ping 2001:db8::1
PING 2001:db8::1(2001:db8::1) 56 data bytes
^C
--- 2001:db8::1 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3062ms
$ ping 2001:db8::2
PING 2001:db8::2(2001:db8::2) 56 data bytes
^C
--- 2001:db8::2 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2030ms
$ sudo ip addr add 2001:db8::1/64 dev lo
$ ping 2001:db8::1
PING 2001:db8::1(2001:db8::1) 56 data bytes
64 bytes from 2001:db8::1: icmp_seq=1 ttl=64 time=0.055 ms
64 bytes from 2001:db8::1: icmp_seq=2 ttl=64 time=0.074 ms
64 bytes from 2001:db8::1: icmp_seq=3 ttl=64 time=0.073 ms
64 bytes from 2001:db8::1: icmp_seq=4 ttl=64 time=0.071 ms
^C
--- 2001:db8::1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3075ms
rtt min/avg/max/mdev = 0.055/0.068/0.074/0.007 ms
$ ping 2001:db8::2
PING 2001:db8::2(2001:db8::2) 56 data bytes
From 2001:db8::1 icmp_seq=1 Destination unreachable: No route
From 2001:db8::1 icmp_seq=2 Destination unreachable: No route
From 2001:db8::1 icmp_seq=3 Destination unreachable: No route
From 2001:db8::1 icmp_seq=4 Destination unreachable: No route
^C
--- 2001:db8::2 ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 3070ms
```
Test: integration_test.TestLoopbackAcceptAllInSubnet
PiperOrigin-RevId: 329011566
Fixes *.sh Java runtime tests, where splice()-ing from a pipe to /dev/zero
would not actually empty the pipe.
There was no guarantee that the data would actually be consumed on a splice
operation unless the output file's implementation of Write/PWrite actually
called VFSPipeFD.CopyIn. Now, whatever bytes are "written" are consumed
regardless of whether CopyIn is called or not.
Furthermore, the number of bytes in the IOSequence for reads is now capped at
the amount of data actually available. Before, splicing to /dev/zero would
always return the requested splice size without taking the actual available
data into account.
This change also refactors the case where an input file is spliced into an
output pipe so that it follows a similar pattern, which is arguably cleaner
anyway.
Updates #3576.
PiperOrigin-RevId: 328843954
BadSocketPair test will return several errnos (EPREM, ESOCKTNOSUPPORT,
EAFNOSUPPORT) meaning the test is just too specific. Checking the syscall
fails is appropriate.
PiperOrigin-RevId: 328813071
...while we figure out of we want to consider the loopback interface
bound to all IPs in an assigned IPv6 subnet, or not (to maintain
compatibility with Linux).
PiperOrigin-RevId: 328807974
ioctl calls with TIOCSCTTY fail if the calling process already has a
controlling terminal, which occurs on a 5.4 kernel like our Ubuntu 18 CI.
Thus, run tests calling ioctl TTOCSCTTY in clean subprocess.
Also, while we're here, switch out non-inclusive master/slave for main/replica.
PiperOrigin-RevId: 328756598
Added a few tests for write(2) and pwrite(2)
1. Regular Files
For write(2)
- write zero bytes should not move the offset
- write non-zero bytes should increment the offset the exact amount
- write non-zero bytes after a lseek() should move the offset the exact amount after the seek
- write non-zero bytes with O_APPEND should move the offset the exact amount after original EOF
For pwrite(2), offset is not affected when
- pwrite zero bytes
- pwrite non-zero bytes
For EOF, added a test asserting the EOF (indicated by lseek(SEEK_END)) is updated properly after writing non-zero bytes
2. Symlink
Added one pwite64() call for symlink that is written as a counterpart of the existing test using pread64()
In Linux, a kernel configuration is set that compiles the kernel with a
custom function that is called at the beginning of every basic block, which
updates the memory-mapped coverage information. The Go coverage tool does not
allow us to inject arbitrary instructions into basic blocks, but it does
provide data that we can convert to a kcov-like format and transfer them to
userspace through a memory mapping.
Note that this is not a strict implementation of kcov, which is especially
tricky to do because we do not have the same coverage tools available in Go
that that are available for the actual Linux kernel. In Linux, a kernel
configuration is set that compiles the kernel with a custom function that is
called at the beginning of every basic block to write program counters to the
kcov memory mapping. In Go, however, coverage tools only give us a count of
basic blocks as they are executed. Every time we return to userspace, we
collect the coverage information and write out PCs for each block that was
executed, providing userspace with the illusion that the kcov data is always
up to date. For convenience, we also generate a unique synthetic PC for each
block instead of using actual PCs. Finally, we do not provide thread-specific
coverage data (each kcov instance only contains PCs executed by the thread
owning it); instead, we will supply data for any file specified by --
instrumentation_filter.
Also, fix issue in nogo that was causing pkg/coverage:coverage_nogo
compilation to fail.
PiperOrigin-RevId: 328426526
iptables sockopts were kludged into an unnecessary check, this properly
relegates them to the {get,set}SockOptIP functions.
PiperOrigin-RevId: 328395135
When SO_LINGER option is enabled, the close will not return until all the
queued messages are sent and acknowledged for the socket or linger timeout is
reached. If the option is not set, close will return immediately. This option
is mainly supported for connection oriented protocols such as TCP.
PiperOrigin-RevId: 328350576
These tests print disk_free_space()/disk_total_space() and expect the printed
result to be an integer (despite the fact that both the documented and returned
type is float). After cl/297213789, free/total disk space on tmpfs is
sufficiently large that PHP prints the result in scientific notation instead:
========DIFF========
012+ float(9.2233720368548E+18)
013+ float(9.2233720368548E+18)
012- float(%d)
013- float(%d)
========DONE========
FAIL disk_total_space() and disk_free_space() tests [ext/standard/tests/file/disk.phpt]
PiperOrigin-RevId: 328349906
We still deviate a bit from linux in how long we will actually wait in
FIN-WAIT-2. Linux seems to cap it with TIME_WAIT_LEN and it's not completely
obvious as to why it's done that way. For now I think we can ignore that and
fix it if it really is an issue.
PiperOrigin-RevId: 328324922
When a loopback interface is configurd with an address and associated
subnet, the loopback should treat all addresses in that subnet as an
address it owns.
This is mimicking linux behaviour as seen below:
```
$ ip addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group ...
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
$ ping 192.0.2.1
PING 192.0.2.1 (192.0.2.1) 56(84) bytes of data.
^C
--- 192.0.2.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1018ms
$ ping 192.0.2.2
PING 192.0.2.2 (192.0.2.2) 56(84) bytes of data.
^C
--- 192.0.2.2 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2039ms
$ sudo ip addr add 192.0.2.1/24 dev lo
$ ip addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group ...
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.0.2.1/24 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
$ ping 192.0.2.1
PING 192.0.2.1 (192.0.2.1) 56(84) bytes of data.
64 bytes from 192.0.2.1: icmp_seq=1 ttl=64 time=0.131 ms
64 bytes from 192.0.2.1: icmp_seq=2 ttl=64 time=0.046 ms
64 bytes from 192.0.2.1: icmp_seq=3 ttl=64 time=0.048 ms
^C
--- 192.0.2.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2042ms
rtt min/avg/max/mdev = 0.046/0.075/0.131/0.039 ms
$ ping 192.0.2.2
PING 192.0.2.2 (192.0.2.2) 56(84) bytes of data.
64 bytes from 192.0.2.2: icmp_seq=1 ttl=64 time=0.131 ms
64 bytes from 192.0.2.2: icmp_seq=2 ttl=64 time=0.069 ms
64 bytes from 192.0.2.2: icmp_seq=3 ttl=64 time=0.049 ms
64 bytes from 192.0.2.2: icmp_seq=4 ttl=64 time=0.035 ms
^C
--- 192.0.2.2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3049ms
rtt min/avg/max/mdev = 0.035/0.071/0.131/0.036 ms
```
Test: integration_test.TestLoopbackAcceptAllInSubnet
PiperOrigin-RevId: 328188546
Add tests for socket re-bind/listen of client and server sockets
with the older connection still in TIME_WAIT state and with
SO_REUSEADDR enabled.
PiperOrigin-RevId: 327924702
This is done to ease troubleshooting when tests fail. runsc
logs are not stored when tests passe, so this will only
affect failing tests and should not increase log storage
too badly.
PiperOrigin-RevId: 327717551
Accept 128 + SIGNAL as well as SIGNAL as valid
returns for fork/exec tests.
Also, make changes so that test compiles in opensource. Test
had compile errors on latest Ubuntu 16.04 image with updated bazel to
3.4.0 (as well as base 2.0) used for Kokoro tests.
PiperOrigin-RevId: 327510310
Skip InvalidOffset and InvalidLength for Linux as the test is invalid for
later Kernel versions.
Add UnsupportedFile test as this check is in all kernel versions.
PiperOrigin-RevId: 327248035
Setting timeouts for sockets on GCP images (debian) for usecs only
respects multiples of 4K. Set the test with a multiple of 4K with a comment.
PiperOrigin-RevId: 327093848
Fixes python runtime test test_glob.
Updates #3515
We were checking is the to-be-opened dentry is a dir or not before resolving
symlinks. We should check that after resolving symlinks.
This was preventing us from opening a symlink which pointed to a directory
with O_DIRECTORY.
Also added this check in tmpfs and removed a duplicate check.
PiperOrigin-RevId: 327085895
This is a preparatory commit for a larger commit working on
ICMP generation in error cases.
This is removal of technical debt and cleanup in the gvisor code
as part of gvisor issue 2211.
Updates #2211.
PiperOrigin-RevId: 326615389
Fixes php runtime test ext/standard/tests/file/readfile_basic.phpt
Fixes#3516
fsgofers only want the access mode in the OpenFlags passed to Create(). If more
flags are supplied (like O_APPEND in this case), read/write from that fd will
fail with EBADF. See runsc/fsgofer/fsgofer.go:WriteAt()
VFS2 was providing more than just access modes. So filtering the flags using
p9.OpenFlagsModeMask == linux.O_ACCMODE fixes the issue.
Gofer in VFS1 also only extracts the access mode flags while making the create
RPC. See pkg/sentry/fs/gofer/path.go:Create()
Even in VFS2, when we open a handle, we extract out only the access mode flags
+ O_TRUNC.
See third_party/gvisor/pkg/sentry/fsimpl/gofer/handle.go:openHandle()
Added a test for this.
PiperOrigin-RevId: 326574829
Netstack's TIME-WAIT state for a TCP socket could be terminated prematurely if
the socket entered TIME-WAIT using shutdown(..., SHUT_RDWR) and then was closed
using close(). This fixes that bug and updates the tests to verify that Netstack
correctly honors TIME-WAIT under such conditions.
Fixes#3106
PiperOrigin-RevId: 326456443
Fixes php test ext/standard/tests/file/touch_variation5.phpt on vfs2.
Updates #3516
Also spotted a bug with O_EXCL, where we did not return EEXIST when we tried
to open the root of the filesystem with O_EXCL | O_CREAT.
Added some more tests for open() corner cases.
PiperOrigin-RevId: 326346863
The code was deleting logs for all tests when a single test
passed. Change it to delete only the logs relevant to the
test at hand.
Also fixed the benchmark lookup code, which was always generating
a single empty benchmark entry if there were not benchmarks.
PiperOrigin-RevId: 326311477
Mark all tests passing for VFS2 in:
image_test
integration_test
There's no way to do negative look ahead/behind in golang test regex,
so check if the tests uses VFS2 and skip CheckPointRestore if it does.
PiperOrigin-RevId: 326050915
//test/iptables:iptables_test runs 30 seconds faster on my machine.
* Using contexts instead of many smaller timeouts makes the tests less
likely to flake and removes unnecessary complexity.
* We also use context to properly shut down concurrent goroutines and
the test container.
* Container logs are always logged.
Remove the old benchmark-tools directory, including
imports in the WORKSPACE file and associated bazel rules.
The new Golang benchmark-tools can be found at //test/benchmarks
and it is functionally equivalent, excepting syscall_test
which can be found in //test/perf/linux.
PiperOrigin-RevId: 325529075
It was changed in the Linux kernel:
commit f0628c524fd188c3f9418e12478dfdfadacba815
Date: Fri Apr 24 16:06:16 2020 +0800
net: Replace the limit of TCP_LINGER2 with TCP_FIN_TIMEOUT_MAX
PiperOrigin-RevId: 325493859
test_httplib was un-excluded in cl/325137657. Exclude it again.
test_httplib fails consistently when running in a kokoro job. Could not repro
with Docker.
There are some java tests that pass locally on Docker but fail when running
within a kokoro job.
Also make shard count a multiple of 8 to minimize idle cores and minimize test
time.
PiperOrigin-RevId: 325267071
- un-exclude passing tests to increase testing surface.
- create/update bugs for tests that fail on runsc but pass on runc.
PiperOrigin-RevId: 325137657
This required minor fixes to the bazel wrapper. The "dut_platform" is
also changed to "native" to line-up with the system call tests and
remove the hard-coded "linux" and "netstack" strings.
The newer version of FUSE_INIT checks the response from the FUSE server
if its major number is equal to 7. If it's not, then FUSE_INIT fails and
further filesystem operations will get ECONNREFUSED. To mitigate this
issue, we can send back a response with major version equals to 7 when
consuming the first FUSE_INIT request.
Fixes#3500
This commit adds an integration test framework for FUSE support. Please
refer to the test example and test/fuse/README.md for further details.
Fixes#3098
- Exclude flaky tests.
- Bump timeout.
- Un-exclude passing tests to increase testing surface.
- Create/Update bugs for tests that pass on runc but fail on runsc.
PiperOrigin-RevId: 324830840
This change adds a new reverse HTTP test where the HTTPD server runs
in a native container but the client runs inside gVisor. It allows
us to test download performance under varying levels of concurrency.
Also tweaks the concurrent request numbers to test for high levels
of concurrency.
PiperOrigin-RevId: 324651203
Before kernel version 4.16-rc6, fuse mount is protected by
capable(CAP_SYS_ADMIN). After this version, it uses
ns_capable(CAP_SYS_ADMIN) to protect. Before the 4.16 kernel,
it was not allowed to mount fuse file systems without the
global CAP_SYS_ADMIN.
Fixes#3360
- Added a bunch of helpful options which help in speeding up the test and
providing useful output.
- Unexcluded passing tests and updated bugs. Excluded tests which were failing.
- Increased the batch size for java tests so that we can take advantage of
the shared JVMs.
The running time of the tests decreased from 3+ hours (I don't know the exact
running time because this test has always timed out after 3 hours) to 1 hour
15 minutes. We can reliably run this a CI kokoro job.
PiperOrigin-RevId: 324301503
This change:
- Ports the nginx benchmark.
- Switches the Httpd benchmark to use 'hey' as a client.
- Moves all parsers to their own package 'tools'.
Parsers are moved to their own package because 1) parsing output of a command
is often dependent on the format of the command (e.g. 'fio --json'), 2) to
enable easier reuse, and 3) clean up and simplify actual running benchmarks
(no TestParser functions and ugly sample output in benchmark files).
PiperOrigin-RevId: 324144165
9P2000.L is silent as to how readdir RPCs interact with directory mutation. The
most performant option is for Treaddir with offset=0 to restart iteration,
avoiding needing to walk+open+clunk a new directory fid between invocations of
getdents64(2), and the VFS2 gofer client assumes this is the case. Make this
actually true for the runsc fsgofer.
Fixes#3344, #3345, #3355
PiperOrigin-RevId: 324090384
- Unexported some passing tests. This will increase the testing surface and
will be especially helpful when this is enabled for vfs2.
- Run tool tests with -v (verbose output). We only print the output when a test
fails so this should not clutter the output.
- Run tool tests with "-no-rebuild" flag.
- Surround test name with appropriate regex, i.e. ^testname$. This will ensure
that only that test is run. Earlier running go_test:os would also run
go_test:os/exec, go_test:os/signal, go_test:os/user. This should help speed
up the tests as we do not run the same test multiple times anymore.
- Updated bugs.
Updates #3191
PiperOrigin-RevId: 324028878
Storing *testing.T on test helper structs is problematic when
subtests are used, because it is possible for nested tests to call
Fatal on parent test, which incorrect terminates the parent test.
For example
func TestOuter(t *testing.T) {
dut := NewDUT(t)
t.Run("first test", func(t *testing.T) {
dut.FallibleCall()
})
t.Run("second test", func(t *testing.T) {
dut.FallibleCall()
}
}
In the example above, assuming `FallibleCall` calls `t.Fatal` on the
`t` it holds, if `dut.FallibleCall` fails in "first test", it will
call `Fatal` on the parent `t`, quitting `TestOuter`. This is not a
behavior we want.
PiperOrigin-RevId: 323350241
Adds profiling with `runsc debug` or pprof to dockerutil. All
targets using dockerutil should now be able to use profiling.
In addition, modifies existing benchmarks to use profiling.
PiperOrigin-RevId: 323298634
... when it is possible.
The guitar gVisorKernel*Workflow-s runs test with the local execution_method.
In this case, blaze runs test cases locally without sandboxes. This means
that all tests run in the same network namespace. We have a few tests which
use hard-coded network ports and they can fail if one of these port will be
used by someone else or by another test cases.
PiperOrigin-RevId: 323137254
Sending UDP packets in a loop can be done in a separate goroutine. We
can't do this in ContainerAction because the container will terminate
early.
Locally, scripts/iptables_tests.sh runs ~40 seconds faster.
Previously, ICMP destination unreachable datagrams were ignored by TCP
endpoints. This caused connect to hang when an intermediate router
couldn't find a route to the host.
This manifested as a Kokoro error when Docker IPv6 was enabled. The Ruby
image test would try to install the sinatra gem and hang indefinitely
attempting to use an IPv6 address.
Fixes#3079.
Fixes a NAT bug that manifested as:
- A SYN was sent from gVisor to another host, unaffected by iptables.
- The corresponding SYN/ACK was NATted by a PREROUTING REDIRECT rule
despite being part of the existing connection.
- The socket that sent the SYN never received the SYN/ACK and thus a
connection could not be established.
We handle this (as Linux does) by tracking all connections, inserting a
no-op conntrack rule for new connections with no rules of their own.
Needed for istio support (#170).
Helps in fixing open syscall tests: AppendConcurrentWrite and AppendOnly.
We also now update the file size for seekable special files (regular files)
which we were not doing earlier.
Updates #2923
PiperOrigin-RevId: 322670843
DUT logs will include logs from the posix server and gVisor, which
provides a way to instrument the DUT during test failures.
PiperOrigin-RevId: 321816647
Ayush Ranjan
Nayana Bidari
Zach Koopmans
Jay Zhuang
Tamir Duberstein
Rahat Mahmood
Ghanan Gowripalan
Dean Deng
Kevin Krakauer
Andrei Vagin
gVisor bot
Jinmou Li
Nicolas Lacasse
Jamie Liu
Adin Scannell
Bhasker Hariharan
Mithun Iyer
Ting-Yu Wang
Fabricio Voznika
Ian Lewis
Julian Elischer
Craig Chi
Ridwan Sharif