NETLINK_KOBJECT_UEVENT sockets send udev-style messages for device events.
gVisor doesn't have any device events, so our sockets don't need to do anything
once created.
systemd's device manager needs to be able to create one of these sockets. It
also wants to install a BPF filter on the socket. Since we'll never send any
messages, the filter would never be invoked, thus we just fake it out.
Fixes#1117
Updates #1119
PiperOrigin-RevId: 278405893
Since we only supporting sending messages from the kernel, the peer is always
the kernel, simplifying handling.
There are currently no known users of SO_PASSCRED that would actually receive
messages from gVisor, but adding full support is barely more work than stubbing
out fake support.
Updates #1117Fixes#1119
PiperOrigin-RevId: 277981465
When execveat is called on an interpreter script, the symlink count for
resolving the script path should be separate from the count for resolving the
the corresponding interpreter. An ELOOP error should not occur if we do not hit
the symlink limit along any individual path, even if the total number of
symlinks encountered exceeds the limit.
Closes#574
PiperOrigin-RevId: 277358474
Set the snd/rcv buffer sizes so that the test is deterministic and runs in a
reasonable amount of time. It also ensures that we disable any auto-tuning of
the send/receive buffer which may happen.
PiperOrigin-RevId: 277337232
When an interpreter script is opened with O_CLOEXEC and the resulting fd is
passed into execveat, an ENOENT error should occur (the script would otherwise
be inaccessible to the interpreter). This matches the actual behavior of
Linux's execveat.
PiperOrigin-RevId: 277306680
container.startContainers() cannot be called twice in a test
(e.g. TestMultiContainerLoadSandbox) because the cleanup
function deletes the rootDir, together with information from
all other containers that may exist.
PiperOrigin-RevId: 276591806
Like (AF_INET, SOCK_RAW) sockets, AF_PACKET sockets require CAP_NET_RAW. With
runsc, you'll need to pass `--net-raw=true` to enable them.
Binding isn't supported yet.
PiperOrigin-RevId: 275909366
This change fixes several issues with the fsgofer host UDS support. Notably, it
adds support for SOCK_SEQPACKET and SOCK_DGRAM sockets [1]. It also fixes
unsafe use of unet.Socket, which could cause a panic if Socket.FD is called
when err != nil, and calls to Socket.FD with nothing to prevent the garbage
collector from destroying and closing the socket.
A set of tests is added to exercise host UDS access. This required extracting
most of the syscall test runner into a library that can be used by custom
tests.
Updates #235
Updates #1003
[1] N.B. SOCK_DGRAM sockets are likely not particularly useful, as a server can
only reply to a client that binds first. We don't allow bind, so these are
unlikely to be used.
PiperOrigin-RevId: 275558502
* Use mknod instead of open&close to create an empty file.
* Limit a number of files to (1<<16) instead of 100K.
In this case, a test set is (1, 8, 64, 512, 4K, 32K, 64K) instead of (1, 8, 64,
512, 4K, 32K, 98K). I think it is easier to compare results for 32K and 64K
than 32K and 98K. And results for 98K doesn't give us more information than for
54K.
PiperOrigin-RevId: 275552507
Netstack has its own stats, we use this to fill /proc/net/snmp.
Note that some metrics are not recorded in Netstack, which will be shown
as 0 in the proc file.
Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
Change-Id: Ie0089184507d16f49bc0057b4b0482094417ebe1
This proc file contains statistics according to [1].
[1] https://tools.ietf.org/html/rfc2013
Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
Change-Id: I9662132085edd8a7783d356ce4237d7ac0800d94
This allows for peeking at the length of the next message on a netlink socket
without pulling it off the socket's buffer/queue, allowing tools like 'ip' to
work.
This CL also fixes an issue where dump_done_errno was not included in the
NLMSG_DONE messages payload.
Issue #769
PiperOrigin-RevId: 274068637
The signalfd descriptors otherwise always show as available. This can lead
programs to spin, assuming they are looking to see what signals are pending.
Updates #139
PiperOrigin-RevId: 274017890
Adds two tests. One to make sure that $HOME is set when starting a container
via 'docker run' and one to make sure that $HOME is set for each container in a
multi-container sandbox.
Issue #701
PiperOrigin-RevId: 273395763
The behavior for sending and receiving local broadcast (255.255.255.255)
traffic is as follows:
Outgoing
--------
* A broadcast packet sent on a socket that is bound to an interface goes out
that interface
* A broadcast packet sent on an unbound socket follows the route table to
select the outgoing interface
+ if an explicit route entry exists for 255.255.255.255/32, use that one
+ else use the default route
* Broadcast packets are looped back and delivered following the rules for
incoming packets (see next). This is the same behavior as for multicast
packets, except that it cannot be disabled via sockopt.
Incoming
--------
* Sockets wishing to receive broadcast packets must bind to either INADDR_ANY
(0.0.0.0) or INADDR_BROADCAST (255.255.255.255). No other socket receives
broadcast packets.
* Broadcast packets are multiplexed to all sockets matching it. This is the
same behavior as for multicast packets.
* A socket can bind to 255.255.255.255:<port> and then receive its own
broadcast packets sent to 255.255.255.255:<port>
In addition, this change implicitly fixes an issue with multicast reception. If
two sockets want to receive a given multicast stream and one is bound to ANY
while the other is bound to the multicast address, only one of them will
receive the traffic.
PiperOrigin-RevId: 272792377
The input file descriptor is always a regular file, so sendfile can't lose any
data if it will not be able to write them to the output file descriptor.
Reported-by: syzbot+22d22330a35fa1c02155@syzkaller.appspotmail.com
PiperOrigin-RevId: 272730357
gVisor does not currently implement the functionality that would result in
AT_SECURE = 1, but Linux includes AT_SECURE = 0 in the normal case, so we
should do the same.
PiperOrigin-RevId: 272311488
Refactoring in 0036d1f7eb95bcc52977f15507f00dd07018e7e2 (v4.10) caused Linux to
start unconditionally zeroing the remainder of the last page in the
interpreter. Previously it did not due so if filesz == memsz, and *still* does
not do so when filesz == memsz for loading binaries, only interpreter.
This inconsistency is not worth replicating in gVisor, as it is arguably a bug,
but our tests must ensure we create interpreter ELFs compatible with this new
requirement.
PiperOrigin-RevId: 272266401
Kernel.cpuClockTicker increments kernel.cpuClock, which tasks use as a clock to
track their CPU usage. This improves latency in the syscall path by avoid
expensive monotonic clock calls on every syscall entry/exit.
However, this timer fires every 10ms. Thus, when all tasks are idle (i.e.,
blocked or stopped), this forces a sentry wakeup every 10ms, when we may
otherwise be able to sleep until the next app-relevant event. These wakeups
cause the sentry to utilize approximately 2% CPU when the application is
otherwise idle.
Updates to clock are not strictly necessary when the app is idle, as there are
no readers of cpuClock. This commit reduces idle CPU by disabling the timer
when tasks are completely idle, and computing its effects at the next wakeup.
Rather than disabling the timer as soon as the app goes idle, we wait until the
next tick, which provides a window for short sleeps to sleep and wakeup without
doing the (relatively) expensive work of disabling and enabling the timer.
PiperOrigin-RevId: 272265822
'docker exec' was getting CAP_NET_RAW even when --net-raw=false
because it was not filtered out from when copying container's
capabilities.
PiperOrigin-RevId: 272260451
Linux changed this behavior in 16e72e9b30986ee15f17fbb68189ca842c32af58
(v4.11). Previously, extra pages were always mapped RW. Now, those pages will
be executable if the segment specified PF_X. They still must be writeable.
PiperOrigin-RevId: 272256280
The test is checking the wrong poll_fd for POLLHUP. The only
reason it passed till now was because it was also checking
for POLLIN which was always true on the other fd from the
previous poll!
PiperOrigin-RevId: 270780401
Previously, when we set hostname:
$ strace hostname abc
...
sethostname("abc", 3) = -1 ENAMETOOLONG (File name too long)
...
According to man 2 sethostname:
"The len argument specifies the number of bytes in name. (Thus, name
does not require a terminating null byte.)"
We wrongly use the CopyStringIn() to check terminating zero byte in
the implementation of sethostname syscall.
To fix this, we use CopyInBytes() instead.
Fixes: #861
Reported-by: chenglang.hy <chenglang.hy@antfin.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
Adresses a deadlock with the rolled back change:
b6a5b950d2
Creating a session from an orphaned process group was causing a lock to be
acquired twice by a single goroutine. This behavior is addressed, and a test
(OrphanRegression) has been added to pty.cc.
Implemented the following ioctls:
- TIOCSCTTY - set controlling TTY
- TIOCNOTTY - remove controlling tty, maybe signal some other processes
- TIOCGPGRP - get foreground process group. Also enables tcgetpgrp().
- TIOCSPGRP - set foreground process group. Also enabled tcsetpgrp().
Next steps are to actually turn terminal-generated control characters (e.g. C^c)
into signals to the proper process groups, and to send SIGTTOU and SIGTTIN when
appropriate.
PiperOrigin-RevId: 270088599
Note that the exact semantics for these signalfds are slightly different from
Linux. These signalfds are bound to the process at creation time. Reads, polls,
etc. are all associated with signals directed at that task. In Linux, all
signalfd operations are associated with current, regardless of where the
signalfd originated.
In practice, this should not be an issue given how signalfds are used. In order
to fix this however, we will need to plumb the context through all the event
APIs. This gets complicated really quickly, because the waiter APIs are all
netstack-specific, and not generally exposed to the context. Probably not
worthwhile fixing immediately.
PiperOrigin-RevId: 269901749
- Fix ARG syntax in Dockerfiles.
- Fix curl commands in Dockerfiles.
- Fix some paths in proctor binaries.
- Check error from Walk in search helper.
PiperOrigin-RevId: 269641686
* Use multi-stage builds in Dockerfiles.
* Combine all proctor binaries into a single binary.
* Change the TestRunner interface to reduce code duplication.
PiperOrigin-RevId: 269462101
absl flags are more modern and we can easily depend on them directly.
The repo now successfully builds with --incompatible_load_cc_rules_from_bzl.
PiperOrigin-RevId: 269387081
This also allows the tee(2) implementation to be enabled, since dup can now be
properly supported via WriteTo.
Note that this change necessitated some minor restructoring with the
fs.FileOperations splice methods. If the *fs.File is passed through directly,
then only public API methods are accessible, which will deadlock immediately
since the locking is already done by fs.Splice. Instead, we pass through an
abstract io.Reader or io.Writer, which elide locks and use the underlying
fs.FileOperations directly.
PiperOrigin-RevId: 268805207
A recent Kokoro change pointed to go_tests.cfg (in line with the
other configurations), which unfortunately broke the presubmits.
This change also enabled the KVM tests, which were still using a
remote execution strategy.
This fixes both of these issues and allows presubmits to pass.
One additional test was caught with this case, which seems to
have been broken. It's unclear why this was not being caught.
PiperOrigin-RevId: 268166291
This is done because the root container for CRI is the infrastructure (pause)
container and always gets a low oom_score_adj. We do this to ensure that only
the oom_score_adj of user containers is used to calculated the sandbox
oom_score_adj.
Implemented in runsc rather than the containerd shim as it's a bit cleaner to
implement here (in the shim it would require overwriting the oomScoreAdj and
re-writing out the config.json again). This processing is Kubernetes(CRI)
specific but we are currently only supporting CRI for multi-container support
anyway.
PiperOrigin-RevId: 267507706
TestNoDuplicates is racy as it tries to read the /proc file system
while the test is running. But it's possible that from the time a
directory entries are read and each entry processed something could
change and in some cases the entry being processed could have been
deleted. In such cases we should not fail the test but just
ignore the error and move on.
PiperOrigin-RevId: 267483094
- Most AIO tests call io_setup(nr_events = 128). sizeof(struct io_event)
(128*32 = 4096). However, the actual size of the mapping created by
io_setup() is determined by:
(from fs/aio.c:ioctx_alloc())
/*
* We keep track of the number of available ringbuffer slots, to prevent
* overflow (reqs_available), and we also use percpu counters for this.
*
* So since up to half the slots might be on other cpu's percpu counters
* and unavailable, double nr_events so userspace sees what they
* expected: additionally, we move req_batch slots to/from percpu
* counters at a time, so make sure that isn't 0:
*/
nr_events = max(nr_events, num_possible_cpus() * 4);
nr_events *= 2;
(from fs/aio.c:aio_setup_ring())
/* Compensate for the ring buffer's head/tail overlap entry */
nr_events += 2; /* 1 is required, 2 for good luck */
size = sizeof(struct aio_ring);
size += sizeof(struct io_event) * nr_events;
nr_pages = PFN_UP(size);
When we mremap() only the first page of a multi-page AIO ring buffer
mapping, fs/aio.c:aio_ring_mremap() updates struct kioctx::mmap_base -
but struct kioctx::mmap_size is untouched, so sys_io_destroy() =>
kill_ioctx() vm_unmaps() the mremapped page, plus some number of pages
after it. Just get the actual size of the mapping from /proc/self/maps.
- Delete test case MremapOver; while it is correct that Linux will not
complain if you overwrite the AIO ring buffer with another mapping, it
won't actually work in the sense that AIO events will not be written to
the new mapping, because Linux stores the struct pages of the ring
buffer in struct kioctx::ring_pages and writes to those through kmap()
rather than using userspace addresses.
- Don't munmap() after mremap(MREMAP_FIXED) returns EFAULT; see new
comment in factored-out test case MremapExpansion.
PiperOrigin-RevId: 267482903
The simple test script has gotten out of control. Shard this script into
different pieces and attempt to impose order on overall test structure. This
change helps lay some of the foundations for future improvements.
* The runsc/test directories are moved into just test/.
* The runsc/test/testutil package is split into logical pieces.
* The scripts/ directory contains new top-level targets.
* Each test is now responsible for building targets it requires.
* The install functionality is moved into `runsc` itself for simplicity.
* The existing kokoro run_tests.sh file now just calls all (can be split).
After this change is merged, I will create multiple distinct workflows for
Kokoro, one for each of the scripts currently targeted by `run_tests.sh` today,
which should dramatically reduce the time-to-run for the Kokoro tests, and
provides a better foundation for further improvements to the infrastructure.
PiperOrigin-RevId: 267081397
Ioctl was returning just the buffer size from epsocket.endpoint
and it was not considering data from epsocket.SocketOperations
that was read from the endpoint, but not yet sent to the caller.
PiperOrigin-RevId: 266485461
When abstract unix domain socket paths are displayed in
/proc/net/unix, Linux historically emitted null bytes as padding at
the end of the path. Newer versions of Linux (v4.9,
e7947ea770d0de434d38a0f823e660d3fd4bebb5) display these as '@'
characters.
Update proc_net_unix test to handle both version of the padding.
PiperOrigin-RevId: 266230200
Using "go run ..." in the ENTRYPOINT causes the go compiler to run each time
the container is started. We can just compile the binary once as part of the
image.
PiperOrigin-RevId: 266212462
The flake had the call to futex_unlock_pi() returning EINVAL with the
FUTEX_OWNER_DIED set. In this case, userspace has to clean up stale
state. So instead of calling FUTEX_UNLOCK_PI outright, we'll use the
advised atomic compare_exchange as advised in the man page.
PiperOrigin-RevId: 265163920
In cl/264434674 and cl/264498919, we stop running test cases
in parallel to not overload test hosts. But now tests requires
more time to run, so we need to increase a default number of
shards or a default test timeout. Let's start with increasing
the number of shards and see how it will works.
PiperOrigin-RevId: 264917055
For SOCK_STREAM type unix socket, we shall return ECONNRESET if peer is
closed with data not read.
We explictly set a flag when closing one end, to differentiate from
just shutdown (where zero shall be returned).
Fixes: #735
Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
Previously, recvmsg() on a unix stream socket with its peer closed will
never return, with goroutine call trace like this:
...
2 in gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).block
at pkg/sentry/kernel/task_block.go:124
3 in gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).BlockWithDeadline
at pkg/sentry/kernel/task_block.go:69
4 in gvisor.dev/gvisor/pkg/sentry/socket/unix.(*SocketOperations).RecvMsg
at pkg/sentry/socket/unix/unix.go:612
5 in gvisor.dev/gvisor/pkg/sentry/syscalls/linux.recvFrom
at pkg/sentry/syscalls/linux/sys_socket.go:885
6 in gvisor.dev/gvisor/pkg/sentry/syscalls/linux.RecvFrom
at pkg/sentry/syscalls/linux/sys_socket.go:910
...
The issue is caused by that ErrClosedForReceive returned by
unix/transport.queue is turned into nil in
unix.(*EndpointReader).ReadToBlocks():
err.ToError()
As a result, in unix.(*SocketOperations).RecvMsg():
n == 0 and err == nil
We shall differentiate it from another case - no data to read where
ErrWouldBlock shall be returned; and return 0 immediately.
Fixes: #734
Reported-by: chenglang.hy <chenglang.hy@antfin.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
This fixes the issue of not being able to bind to either a multicast or
broadcast address as well as to send and receive data from it. The way to solve
this is to treat these addresses similar to the ANY address and register their
transport endpoint ID with the global stack's demuxer rather than the NIC's.
That way there is no need to require an endpoint with that multicast or
broadcast address. The stack's demuxer is in fact the only correct one to use,
because neither broadcast- nor multicast-bound sockets care which NIC a
packet was received on (for multicast a join is still needed to receive packets
on a NIC).
I also took the liberty of refactoring udp_test.go to consolidate a lot of
duplicate code and make it easier to create repetitive tests that test the same
feature for a variety of packet and socket types. For this purpose I created a
"flowType" that represents two things: 1) the type of packet being sent or
received and 2) the type of socket used for the test. E.g., a "multicastV4in6"
flow represents a V4-mapped multicast packet run through a V6-dual socket.
This allows writing significantly simpler tests. A nice example is testTTL().
PiperOrigin-RevId: 264766909
test/syscalls/linux/proc_net_tcp.cc:252: Failure
Value of: connect(client->get(), &addr, addrlen)
Expected: not -1 (success)
Actual: -1 (of type int), with errno PosixError(errno=4 Interrupted system call)
PiperOrigin-RevId: 264743815
We need real sharding, and will let Bazel handle the
parallelization. That is coming soon. Until then, remove
this call to t.Parallel() so that we can run the tests without
eating all CPU.
PiperOrigin-RevId: 264498919
The test is long running (175128 ms or so) which causes timeouts.
The test simply makes sure that private futexes can acquire
locks concurrently. Dropping current threads and increasing the
number of locks each thread tests the same concurrency concerns
but drops execution time to ~1411 ms.
PiperOrigin-RevId: 264476144
bazel runs a few instances of syscall_test_runner in parallel
and then syscall_test_runner runs test cases in parallel. It might
be a reason why we see that test hosts are overloaded and sandboxes
start slowly. It should be better to control how many tests are
running in parallel from one place, so let's try to disable this
feature in syscall_test_runner.
PiperOrigin-RevId: 264434674
SendMsg before this change would copy all the data over into a
new slice even if the underlying socket could only accept a
small amount of data. This is really inefficient with non-blocking
sockets and under high throughput where large writes could get
ErrWouldBlock or if there was say a timeout associated with the sendmsg()
syscall.
With this change we delay copying bytes in till they are needed and only
copy what can be potentially sent/held in the socket buffer. Reducing
the need to repeatedly copy data over.
Also a minor fix to change state FIN-WAIT-1 when shutdown(..., SHUT_WR) is called
instead of when we transmit the actual FIN. Otherwise the socket could remain in
CONNECTED state even though the user has called shutdown() on the socket.
Updates #627
PiperOrigin-RevId: 263430505
The new version has a change in behavior when using a custom platform:
* Old behavior: rules that don't require a toolchain used host_platform, no
matter what execution platforms are specified.
* New behavior: rules that don't require a toolchain use standard platform
resolution that starts with execution platforms.
As part of this change, we cannot use the "extra_exectution_platforms" flag
provided by the default bazelrc. I got rid of the default bazelrc file, and
made our custom .bazelrc as minimal as possible.
PiperOrigin-RevId: 263176802
Now if a process sends an unsupported netlink requests,
an error is returned from the send system call.
The linux kernel works differently in this case. It returns errors in the
nlmsgerr netlink message.
Reported-by: syzbot+571d99510c6f935202da@syzkaller.appspotmail.com
PiperOrigin-RevId: 262690453
Using the path_test.go file built by the Golang
devs as a base, tests have been created to verify
the functionality of common.Search().
A mock file system is created and fake test files
are generated to see if they get picked up by
common.Search().
Also included in this CL is a bug fix for
proctor-nodejs that was discovered using this test.
proctor-nodejs used to allow multiple "-" in its
test name filter. The regex has been updated to
prevent this.
PiperOrigin-RevId: 262647263
This can happen because endpoint.Close() closes the accept channel first and
then drains/resets any accepted but not delivered connections. But there can be
connections that are connected but not delivered to the channel as the channel
was full. But closing the channel can cause these writes to fail with a write to
a closed channel.
The correct solution is to abort any connections in SYN-RCVD state and
drain/abort all completed connections before closing the accept channel.
PiperOrigin-RevId: 261951132
After the refactoring of the proctor binaries, the Dockerfiles for each
language must be altered to copy the common folder into their image.
Additionally, Java has been changed to use the pre-built version of
JDK-11 from Ubuntu, instead of building it from the source. This allows
for a smaller image and faster test execution within the container.
PiperOrigin-RevId: 261805158
This change adds functionality for running more languages using
the runtimes test suite. It divides the languages into separate
test functions, which each call the helper testLang function in the
runtimes_test.go file. This allows them to be run individually
or as a group.
PiperOrigin-RevId: 261791935
(Don't worry, this is mostly tests.)
Implemented the following ioctls:
- TIOCSCTTY - set controlling TTY
- TIOCNOTTY - remove controlling tty, maybe signal some other processes
- TIOCGPGRP - get foreground process group. Also enables tcgetpgrp().
- TIOCSPGRP - set foreground process group. Also enabled tcsetpgrp().
Next steps are to actually turn terminal-generated control characters (e.g. C^c)
into signals to the proper process groups, and to send SIGTTOU and SIGTTIN when
appropriate.
PiperOrigin-RevId: 261387276
This change removes the filepath.Walk() function from
proctor- go, php, and nodejs. The filepath.Walk() is
now defined in common.go in Search(). Each proctor binary
passes root directory and testFilter arguments to Search().
proctor-python.go no longer uses filepath.Walk() to search
for tests. There is a built-in list test function within
python's language test suite so that is being used instead.
PiperOrigin-RevId: 261242897
proctor-go had a bug where it would incorrectly identify
a tool test as a disk test. Instead of searching for the
test on disk as the identification method, we now check if
the test name ends in ".go". If the test ends in ".go" it
is run as a disk test, otherwise the test is run as a tool test.
Python tests need to be run from within the directory they exist.
Functionality to split the test name from it's parent directory
has been added and a cmd.Dir argument has been set.
PiperOrigin-RevId: 261021693
Shared code among proctor-*.go files has been refactored
into common/common.go. The common package is imported in
each proctor binary and a struct is created to implement
the testRunner interface defined in common.go. This allows
for the proctor binaries to be updated without having to
copy/paste the same code across all files. There are no
usage or functionality changes.
PiperOrigin-RevId: 260967080
The checksum was not being reset before being re-calculated and sent out.
This caused the sent checksum to always be `0x0800`.
Fixes#605.
PiperOrigin-RevId: 260965059
This doesn't currently pass on gVisor.
While I'm here, fix a bug where connecting to the v6-mapped v4 address doesn't
work in gVisor.
PiperOrigin-RevId: 260923961
This fixes a bug introduced in cl/251934850 that caused
connect-accept-close-connect races to result in the second connect call
failiing when it should have succeeded.
PiperOrigin-RevId: 259584525
It is now correctly initialized to the top of the signal stack.
Previously it was initialized to the address of 'stack.ss_sp' on
the main thread stack.
PiperOrigin-RevId: 258248363
iptables also relies on IPPROTO_RAW in a way. It opens such a socket to
manipulate the kernel's tables, but it doesn't actually use any of the
functionality. Blegh.
PiperOrigin-RevId: 257903078
Adds support to set/get the TCP_MAXSEG value but does not
really change the segment sizes emitted by netstack or
alter the MSS advertised by the endpoint. This is currently
being added only to unblock iperf3 on gVisor. Plumbing
this correctly requires a bit more work which will come
in separate CLs.
PiperOrigin-RevId: 257859112
A userspace process (CPL=3) can access an i/o port if the bit corresponding to
the port is set to 0 in the I/O permission bitmap.
Configure the I/O permission bitmap address beyond the last valid byte in the
TSS so access to all i/o ports is blocked.
Signed-off-by: Liu Hua <sdu.liu@huawei.com>
Change-Id: I3df76980c3735491db768f7210e71703f86bb989
PiperOrigin-RevId: 257336518
The error set in the loop in createAt was being masked
by other errors declared with ":=". This allowed an
ErrResolveViaReadlink error to escape, which can cause
a sentry panic.
Added test case which repros without the fix.
PiperOrigin-RevId: 257061767
Get/Set pipe size and ioctl support were missing from
overlayfs. It required moving the pipe.Sizer interface
to fs so that overlay could get access.
Fixes#318
PiperOrigin-RevId: 255511125
Addresses obvious typos, in the documentation only.
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/gvisor/pull/443 from Pixep:fix/documentation-spelling 4d0688164eafaf0b3010e5f4824b35d1e7176d65
PiperOrigin-RevId: 255477779
The code was wrongly assuming that only read access was
required from the lower overlay when checking for permissions.
This allowed non-writable files to be writable in the overlay.
Fixes#316
PiperOrigin-RevId: 255263686
If we have a symlink whose target does not exist, creating the symlink (either
via 'creat' or 'open' with O_CREAT flag) should create the target of the
symlink. Previously, gVisor would error with EEXIST in this case
PiperOrigin-RevId: 255232944
Makes CLOCK_BOOTTIME available with
* clock_gettime
* timerfd_create
* clock_gettime vDSO
CLOCK_BOOTTIME is implemented as an alias to CLOCK_MONOTONIC.
CLOCK_MONOTONIC already keeps track of time across save
and restore. This is the closest possible behavior to Linux
CLOCK_BOOTIME, as there is no concept of suspend/resume.
Updates google/gvisor#218
For files with O_APPEND, a file write operation gets a file size and uses it as
offset to call an inode write operation. This means that all other operations
which can change a file size should be blocked while the write operation doesn't
complete.
PiperOrigin-RevId: 254873771
The previous number was for the arm architecture.
Also change the statx tests to force them to run on gVisor, which would have
caught this issue.
PiperOrigin-RevId: 254846831
Tests run at HEAD (35719d52):
```
$ bazel test $(bazel query 'filter(".*getdents.*", //test/syscalls:all)')
<snip>
//test/syscalls:getdents_test_native PASSED in 0.3s
//test/syscalls:getdents_test_runsc_ptrace PASSED in 4.9s
//test/syscalls:getdents_test_runsc_ptrace_overlay PASSED in 4.7s
//test/syscalls:getdents_test_runsc_ptrace_shared PASSED in 5.2s
//test/syscalls:getdents_test_runsc_kvm FAILED in 4.0s
```
Tests run at ab6774ce~1 (6f933a93):
```
$ bazel test $(bazel query 'filter(".*getdents.*", //test/syscalls:all)')
//test/syscalls:getdents_test_native PASSED in 0.2s
//test/syscalls:getdents_test_runsc_kvm FAILED in 4.2s
/usr/local/google/home/brb/.cache/bazel/_bazel_brb/967240a6aae7d353a221d73f4375e038/execroot/__main__/bazel-out/k8-fastbuild/testlogs/test/syscalls/getdents_test_runsc_kvm/test.log
//test/syscalls:getdents_test_runsc_ptrace FAILED in 5.3s
/usr/local/google/home/brb/.cache/bazel/_bazel_brb/967240a6aae7d353a221d73f4375e038/execroot/__main__/bazel-out/k8-fastbuild/testlogs/test/syscalls/getdents_test_runsc_ptrace/test.log
//test/syscalls:getdents_test_runsc_ptrace_overlay FAILED in 4.9s
/usr/local/google/home/brb/.cache/bazel/_bazel_brb/967240a6aae7d353a221d73f4375e038/execroot/__main__/bazel-out/k8-fastbuild/testlogs/test/syscalls/getdents_test_runsc_ptrace_overlay/test.log
//test/syscalls:getdents_test_runsc_ptrace_shared FAILED in 5.2s
/usr/local/google/home/brb/.cache/bazel/_bazel_brb/967240a6aae7d353a221d73f4375e038/execroot/__main__/bazel-out/k8-fastbuild/testlogs/test/syscalls/getdents_test_runsc_ptrace_shared/test.log
```
(I think all runsc_kvm tests are broken on my machine -- I'll rerun them
if you can point me at the documentation to set it up)
There will be a deadloop when we use getdents to read /proc/{pid}/task
of an exited process
Like this:
Process A is running
Process B: open /proc/{pid of A}/task
Process A exits
Process B: getdents /proc/{pid of A}/task
Then, process B will fall into deadloop, and return "." and ".."
in loops and never ends.
This patch returns ENOENT when use getdents to read /proc/{pid}/task
if the process is just exited.
Signed-off-by: chris.zn <chris.zn@antfin.com>
We don't have the plumbing for btime yet, so that field is left off. The
returned mask indicates that btime is absent.
Fixes#343
PiperOrigin-RevId: 254575752
Today we have the logic split in two places between endpoint Read() and the
worker goroutine which actually sends a zero window. This change makes it so
that when a zero window ACK is sent we set a flag in the endpoint which can be
read by the endpoint to decide if it should notify the worker to send a
nonZeroWindow update.
The worker now does not do the check again but instead sends an ACK and flips
the flag right away.
Similarly today when SO_RECVBUF is set the SetSockOpt call has logic
to decide if a zero window update is required. Rather than do that we move
the logic to the worker goroutine and it can check the zeroWindow flag
and send an update if required.
PiperOrigin-RevId: 254505447
Bump up the threshold on number of SIGALRMs received by worker
threads from 50 to 200. Even with the new threshold we still
expect that the majority of SIGALRMs are received by the
thread group leader.
PiperOrigin-RevId: 254289787
Inode ids are only stable across Save/Restore if we have an open FD on the
inode. All tests that compare inode ids must therefor hold an FD open.
PiperOrigin-RevId: 254086603
Test was calling Size() inside read and write loops. Size()
makes 2 syscalls to return the pipe size, making the test
do a lot more work than it should.
PiperOrigin-RevId: 253824690
The deadlock can occur when both ends of a connected Unix socket which has
FIOASYNC enabled on at least one end are closed at the same time. One end
notifies that it is closing, calling (*waiter.Queue).Notify which takes
waiter.Queue.mu (as a read lock) and then calls (*FileAsync).Callback, which
takes FileAsync.mu. The other end tries to unregister for notifications by
calling (*FileAsync).Unregister, which takes FileAsync.mu and calls
(*waiter.Queue).EventUnregister which takes waiter.Queue.mu.
This is fixed by moving the calls to waiter.Waitable.EventRegister and
waiter.Waitable.EventUnregister outside of the protection of any mutex used
in (*FileAsync).Callback.
The new test is related, but does not cover this particular situation.
Also fix a data race on FileAsync.e.Callback. (*FileAsync).Callback checked
FileAsync.e.Callback under the protection of FileAsync.mu, but the waiter
calling (*FileAsync).Callback could not and did not. This is fixed by making
FileAsync.e.Callback immutable before passing it to the waiter for the first
time.
Fixes#346
PiperOrigin-RevId: 253138340
This CL also cleans up the error returned for setting congestion
control which was incorrectly returning EINVAL instead of ENOENT.
PiperOrigin-RevId: 252889093
For sendfile(2), we propagate a TCP error through the system call layer.
This should be eaten if there is a partial result. This change also adds
a test to ensure that there is no panic in this case, for both TCP sockets
and unix domain sockets.
PiperOrigin-RevId: 252746192
Changes netstack to confirm to current linux behaviour where if the backlog is
full then we drop the SYN and do not send a SYN-ACK. Similarly we allow upto
backlog connections to be in SYN-RCVD state as long as the backlog is not full.
We also now drop a SYN if syn cookies are in use and the backlog for the
listening endpoint is full.
Added new tests to confirm the behaviour.
Also reverted the change to increase the backlog in TcpPortReuseMultiThread
syscall test.
Fixes#236
PiperOrigin-RevId: 252500462
We still only advertise a single NUMA node, and ignore mempolicy
accordingly, but mbind() at least now succeeds and has effects reflected
by get_mempolicy().
Also fix handling of nodemasks: round sizes to unsigned long (as
documented and done by Linux), and zero trailing bits when copying them
out.
PiperOrigin-RevId: 251950859
This is necessary for implementing network diagnostic interfaces like
/proc/net/{tcp,udp,unix} and sock_diag(7).
For pass-through endpoints such as hostinet, we obtain the socket
state from the backend. For netstack, we add explicit tracking of TCP
states.
PiperOrigin-RevId: 251934850
This is required to make the shutdown visible to peers outside the
sandbox.
The readClosed / writeClosed fields were dropped, as they were
preventing a shutdown socket from reading the remainder of queued bytes.
The host syscalls will return the appropriate errors for shutdown.
The control message tests have been split out of socket_unix.cc to make
the (few) remaining tests accessible to testing inherited host UDS,
which don't support sending control messages.
Updates #273
PiperOrigin-RevId: 251763060
Multicast packets are special in that their destination address does not
identify a specific interface. When sending out such a packet the multicast
address is the remote address, but for incoming packets it is the local
address. Hence, when looping a multicast packet, the route needs to be
tweaked to reflect this.
PiperOrigin-RevId: 251739298
We don't actually support core dumps, but some applications want to
get/set dumpability, which still has an effect in procfs.
Lack of support for set-uid binaries or fs creds simplifies things a
bit.
As-is, processes started via CreateProcess (i.e., init and sentryctl
exec) have normal dumpability. I'm a bit torn on whether sentryctl exec
tasks should be dumpable, but at least since they have no parent normal
UID/GID checks should protect them.
PiperOrigin-RevId: 251712714
VmData is the size of private data segments.
It has the same meaning as in Linux.
Change-Id: Iebf1ae85940a810524a6cde9c2e767d4233ddb2a
PiperOrigin-RevId: 250593739
After bf959931ddb88c4e4366e96dd22e68fa0db9527c ("wait/ptrace: assume
__WALL if the child is traced") (Linux 4.7), tracees are always eligible
for waiting, regardless of type.
PiperOrigin-RevId: 250399527
The previous commit adds WNOTHREAD support to waitid, so we may as well
complete the upstream change.
Linux added WCLONE, WALL, WNOTHREAD support to waitid(2) in
91c4e8ea8f05916df0c8a6f383508ac7c9e10dba ("wait: allow sys_waitid() to
accept __WNOTHREAD/__WCLONE/__WALL"). i.e., Linux 4.7.
PiperOrigin-RevId: 249560587
Change-Id: Iff177b0848a3f7bae6cb5592e44500c5a942fbeb
Pipe internals are made more efficient by avoiding garbage collection.
A pool is now used that can be shared by all pipes, and buffers are
chained via an intrusive list. The documentation for pipe structures
and methods is also simplified and clarified.
The pipe tests are now parameterized, so that they are run on all
different variants (named pipes, small buffers, default buffers).
The pipe buffer sizes are exposed by fcntl, which is now supported
by this change. A size change test has been added to the suite.
These new tests uncovered a bug regarding the semantics of open
named pipes with O_NONBLOCK, which is also fixed by this CL. This
fix also addresses the lack of the O_LARGEFILE flag for named pipes.
PiperOrigin-RevId: 249375888
Change-Id: I48e61e9c868aedb0cadda2dff33f09a560dee773
* A segment with filesz == 0, memsz > 0 should be an anonymous only
mapping. We were failing to load such an ELF.
* Anonymous pages are always mapped RW, regardless of the segment
protections.
PiperOrigin-RevId: 249355239
Change-Id: I251e5c0ce8848cf8420c3aadf337b0d77b1ad991
This does not actually implement an efficient splice or sendfile. Rather, it
adds a generic plumbing to the file internals so that this can be added. All
file implementations use the stub fileutil.NoSplice implementation, which
causes sendfile and splice to fall back to an internal copy.
A basic splice system call interface is added, along with a test.
PiperOrigin-RevId: 249335960
Change-Id: Ic5568be2af0a505c19e7aec66d5af2480ab0939b
* Creation of files, directories (and other fs objects) in a directory
should always update ctime.
* Same for removal.
* atime should not be updated on lookup, only readdir.
I've also renamed some misleading functions that update mtime and ctime.
PiperOrigin-RevId: 249115063
Change-Id: I30fa275fa7db96d01aa759ed64628c18bb3a7dc7
There is a lot of redundancy that we can simplify in the stat_times
test. This will make it easier to add new tests. However, the
simplification reveals that cached uattrs on goferfs don't properly
update ctime on rename.
PiperOrigin-RevId: 248773425
Change-Id: I52662728e1e9920981555881f9a85f9ce04041cf
The issue with duplicate /proc/sys entries seems to have been fixed in:
PiperOrigin-RevId 229305982
Git hash dc8450b567Fixesgoogle/gvisor#125
PiperOrigin-RevId: 248571903
Change-Id: I76ff3b525c93dafb92da6e5cf56e440187f14579
Some behavior was broken due to the difficulty of running automated raw
socket tests.
Change-Id: I152ca53916bb24a0208f2dc1c4f5bc87f4724ff6
PiperOrigin-RevId: 246747067
bazel has a lot of dependencies and users don't want to install them
just to build gvisor.
These changes allows to run bazel in a docker container.
A bazel cache is on the local file system (~/.cache/bazel), so
incremental builds should be fast event after recreating a bazel
container.
Here is an example how to build runsc:
make BAZEL_OPTIONS="build runsc:runsc" bazel
Change-Id: I8c0a6d0c30e835892377fb6dd5f4af7a0052d12a
PiperOrigin-RevId: 246570877
The test also times out when GCE machine has 2 CPUs. I cannot
repro it locally with a 2 CPU cgroup though. Let's skip the
test when there are 2 CPUs to stop the flakiness and retest it
once the fix is available.
PiperOrigin-RevId: 246523363
Change-Id: I9d9d922a5be3aa7bc91dff5a1807ca99f3f4a4f9
Fixed a small logic error that broke proper accounting of MultiPortEndpoints.
PiperOrigin-RevId: 246502126
Change-Id: I1a7d6ea134f811612e545676212899a3707bc2c2
This requires two changes:
1) Support for more than one socket to join a given multicast group.
2) Duplicate delivery of incoming multicast packets to all sockets listening
for it.
In addition, I tweaked the code (and added a test) to disallow duplicates
IP_ADD_MEMBERSHIP calls for the same group and NIC. This is how Linux does
it.
PiperOrigin-RevId: 246437315
Change-Id: Icad8300b4a8c3f501d9b4cd283bd3beabef88b72
Test times out when it runs on a single core. Skip until the
bug in the Go runtime is fixed.
PiperOrigin-RevId: 245866466
Change-Id: Ic3e72131c27136d58b71f6b11acc78abf55895d4
Based on the guidelines at
https://opensource.google.com/docs/releasing/authors/.
1. $ rg -l "Google LLC" | xargs sed -i 's/Google LLC.*/The gVisor Authors./'
2. Manual fixup of "Google Inc" references.
3. Add AUTHORS file. Authors may request to be added to this file.
4. Point netstack AUTHORS to gVisor AUTHORS. Drop CONTRIBUTORS.
Fixes#209
PiperOrigin-RevId: 245823212
Change-Id: I64530b24ad021a7d683137459cafc510f5ee1de9
Previously, createAt was eating all errors from FindInode except for EACCES and
proceeding with the creation. This is incorrect, as FindInode can return many
other errors (like ENAMETOOLONG) that should stop creation.
This CL changes createAt to return all errors encountered except for ENOENT,
which we can ignore because we are about to create the thing.
PiperOrigin-RevId: 245773222
Change-Id: I1b317021de70f0550fb865506f6d8147d4aebc56
Maximum filename length is filesystem-dependent, and obtained via
statfs::f_namelen. This limit is usually 255 bytes (NAME_MAX), but not
always. For example, VFAT supports filenames of up to 255... UCS-2
characters, which Linux conservatively takes to mean UTF-8-encoded
bytes: fs/fat/inode.c:fat_statfs(), FAT_LFN_LEN * NLS_MAX_CHARSET_SIZE.
As a result, Linux's VFS does not enforce NAME_MAX:
$ rg --maxdepth=1 '\WNAME_MAX\W' fs/ include/linux/
fs/libfs.c
38: buf->f_namelen = NAME_MAX;
64: if (dentry->d_name.len > NAME_MAX)
include/linux/relay.h
74: char base_filename[NAME_MAX]; /* saved base filename */
include/linux/fscrypt.h
149: * filenames up to NAME_MAX bytes, since base64 encoding expands the length.
include/linux/exportfs.h
176: * understanding that it is already pointing to a a %NAME_MAX+1 sized
Remove this check from core VFS, and add it to ramfs (and by extension
tmpfs), where it is actually applicable:
mm/shmem.c:shmem_dir_inode_operations.lookup == simple_lookup *does*
enforce NAME_MAX.
PiperOrigin-RevId: 245324748
Change-Id: I17567c4324bfd60e31746a5270096e75db963fac
TCP tests and the implementation will come in followup CLs.
Updates google/gvisor#206
Updates google/gvisor#207
PiperOrigin-RevId: 245121470
Change-Id: Ib50b62724d3ba0cbfb1374e1f908798431ee2b21
For a symbol link to some directory, eg.
`/tmp/symlink -> /tmp/dir`
`fstatat("/tmp/symlink")` should return symbol link data, but
`fstatat("/tmp/symlink/")` (symlink with trailing slash) should return
directory data it points following linux behaviour.
Currently fstatat() a symlink with trailing slash will get "not a
directory" error which is wrong.
Signed-off-by: Wei Zhang <zhangwei198900@gmail.com>
Change-Id: I63469b1fb89d083d1c1255d32d52864606fbd7e2
PiperOrigin-RevId: 244783916
Support shutdown on only the read side of an endpoint. Reads performed
after a call to Shutdown with only the ShutdownRead flag will return
ErrClosedForReceive without data.
Break out the shutdown(2) with SHUT_RD syscall test into to two tests.
The first tests that no packets are sent when shutting down the read
side of a socket. The second tests that, after shutting down the read
side of a socket, unread data can still be read, or an EOF if there is
no more data to read.
Change-Id: I9d7c0a06937909cbb466b7591544a4bcaebb11ce
PiperOrigin-RevId: 244459430
The MSG_TRUNC flag is set in the msghdr when a message is truncated.
Fixesgoogle/gvisor#200
PiperOrigin-RevId: 244440486
Change-Id: I03c7d5e7f5935c0c6b8d69b012db1780ac5b8456
Inode ids are only guaranteed to be stable across save/restore if the file is
held open. This CL fixes a simple stat test to allow it to compare symlink and
target by inode id, as long as the link target is held open.
PiperOrigin-RevId: 244238343
Change-Id: I74c5115915b1cc032a4c16515a056a480f218f00
The existing logic attempting to do this is incorrect. Unary ^ has
higher precedence than &^, so mask always has UnblockableSignals
cleared, allowing dequeueSignalLocked to dequeue unblockable signals
(which allows userspace to ignore them).
Switch the logic so that unblockable signals are always masked.
PiperOrigin-RevId: 244058487
Change-Id: Ib19630ac04068a1fbfb9dc4a8eab1ccbdb21edc3
FD limit and file size limit is read from the host, instead
of using hard-coded defaults, given that they effect the sandbox
process. Also limit the direct cache to use no more than half
if the available FDs.
PiperOrigin-RevId: 244050323
Change-Id: I787ad0fdf07c49d589e51aebfeae477324fe26e6
Even superuser cannot raise RLIMIT_NOFILE above /proc/sys/fs/nr_open, so
start the test by lowering the limits before raising.
Change-Id: Ied6021c64178a6cb9098088a1a3384db523a226f
PiperOrigin-RevId: 242965249
From a recent test failure:
"State:\tD (disk sleep)\n"
"disk sleep" does not match \w+. We need to allow spaces.
PiperOrigin-RevId: 242762469
Change-Id: Ic8d05a16669412a72c1e76b498373e5b22fe64c4
Kevin Krakauer
Andrei Vagin
Michael Pratt
Nicolas Lacasse
Brad Burlage
Dean Deng
Bhasker Hariharan
Fabricio Voznika
Haibo
gVisor bot
Jianfeng Tan
Ian Lewis
Adin Scannell
Ian Gudger
Chris Kuiper
Jamie Liu
Rahat Mahmood
Zach Koopmans
Brett Landau
Samantha Sample
Austin Kiekintveld
Tamir Duberstein
Adrien Leravat
Neel Natu
Kevin
Liu Hua
Adrien Leravat
chris.zn
Googler
chris.zn
Wei Zhang
Ben Burkert
Shiva Prasanth