Commit Graph

1599 Commits

Author SHA1 Message Date
Rahat Mahmood 6cfc76798b Document source and versioning of the TCPInfo struct.
PiperOrigin-RevId: 263637194
2019-08-15 14:05:59 -07:00
Tamir Duberstein fe74bba2bd Don't dereference errors passed to panic()
These errors are always pointers; there's no sense in dereferencing them
in the panic call. Changed one false positive for clarity.

PiperOrigin-RevId: 263611579
2019-08-15 11:58:16 -07:00
Tamir Duberstein 816a9211e9 netstack: move resumption logic into *_state.go
13a98df rearranged some of this code in a way that broke compilation of
the netstack-only export at github.com/google/netstack because
*_state.go files are not included in that export.

This commit moves resumption logic back into *_state.go, fixing the
compilation breakage.

PiperOrigin-RevId: 263601629
2019-08-15 11:13:46 -07:00
Tamir Duberstein d81d94ac4c Replace uinptr with int64 when returning lengths
This is in accordance with newer parts of the standard library.

PiperOrigin-RevId: 263449916
2019-08-14 16:05:56 -07:00
Tamir Duberstein 69d1414a32 Add tcpip.AddressWithPrefix.String
PiperOrigin-RevId: 263436592
2019-08-14 15:02:14 -07:00
Bhasker Hariharan 570fb1db6b Improve SendMsg performance.
SendMsg before this change would copy all the data over into a
new slice even if the underlying socket could only accept a
small amount of data. This is really inefficient with non-blocking
sockets and under high throughput where large writes could get
ErrWouldBlock or if there was say a timeout associated with the sendmsg()
syscall.

With this change we delay copying bytes in till they are needed and only
copy what can be potentially sent/held in the socket buffer. Reducing
the need to repeatedly copy data over.

Also a minor fix to change state FIN-WAIT-1 when shutdown(..., SHUT_WR) is called
instead of when we transmit the actual FIN. Otherwise the socket could remain in
CONNECTED state even though the user has called shutdown() on the socket.

Updates #627

PiperOrigin-RevId: 263430505
2019-08-14 14:34:27 -07:00
Jamie Liu cee044c2ab Add vfs.DynamicBytesFileDescriptionImpl.
This replaces fs/proc/seqfile for vfs2-based filesystems.

PiperOrigin-RevId: 263254647
2019-08-13 17:54:24 -07:00
Fabricio Voznika 0e907c4298 Fix file mode check in pipeOperations
PiperOrigin-RevId: 263203441
2019-08-13 13:33:33 -07:00
Ian Gudger 072d941e32 Add note to name logging mentioning trace logging should be enabled to debug.
PiperOrigin-RevId: 263194584
2019-08-13 12:49:18 -07:00
Fabricio Voznika c386f046c1 Fix file mode check in fsgofer Attach
PiperOrigin-RevId: 263189654
2019-08-13 12:23:02 -07:00
Ian Gudger 99bf75a6dc gonet: Replace NewPacketConn with DialUDP.
This better matches the standard library and allows creating connected
PacketConns.

PiperOrigin-RevId: 263187462
2019-08-13 12:11:09 -07:00
Andrei Vagin 8d97b22aa8 tests: print stack traces if test failed by timeout
PiperOrigin-RevId: 263184083
2019-08-13 11:56:29 -07:00
Nicolas Lacasse fa3d0e6f63 Bump Bazel to v0.28.0
The new version has a change in behavior when using a custom platform:
* Old behavior: rules that don't require a toolchain used host_platform, no
  matter what execution platforms are specified.
* New behavior: rules that don't require a toolchain use standard platform
  resolution that starts with execution platforms.

As part of this change, we cannot use the "extra_exectution_platforms" flag
provided by the default bazelrc. I got rid of the default bazelrc file, and
made our custom .bazelrc as minimal as possible.

PiperOrigin-RevId: 263176802
2019-08-13 11:21:55 -07:00
Nicolas Lacasse 9769a8eaa4 Handle ENOSPC with a partial write.
Similar to the EPIPE case, we can return the number of bytes written before
ENOSPC was encountered. If the app tries to write more, we can return ENOSPC on
the next write.

PiperOrigin-RevId: 263041648
2019-08-12 17:41:33 -07:00
Rahat Mahmood 691c2f8173 Compute size of struct tcp_info instead of hardcoding it.
PiperOrigin-RevId: 263040624
2019-08-12 17:34:38 -07:00
Ian Gudger eac690e358 Fix netstack build error on non-AMD64.
This stub had the wrong function signature.

PiperOrigin-RevId: 262992682
2019-08-12 13:31:16 -07:00
Andrei Vagin af90e68623 netlink: return an error in nlmsgerr
Now if a process sends an unsupported netlink requests,
an error is returned from the send system call.

The linux kernel works differently in this case. It returns errors in the
nlmsgerr netlink message.

Reported-by: syzbot+571d99510c6f935202da@syzkaller.appspotmail.com
PiperOrigin-RevId: 262690453
2019-08-09 22:34:54 -07:00
Brett Landau f2762e8c60 Create tests for common.Search().
Using the path_test.go file built by the Golang
devs as a base, tests have been created to verify
the functionality of common.Search().

A mock file system is created and fake test files
are generated to see if they get picked up by
common.Search().

Also included in this CL is a bug fix for
proctor-nodejs that was discovered using this test.
proctor-nodejs used to allow multiple "-" in its
test name filter. The regex has been updated to
prevent this.

PiperOrigin-RevId: 262647263
2019-08-09 15:35:58 -07:00
Bhasker Hariharan 5a38eb120a Add congestion control states to sender.
This change just introduces different congestion control states and
ensures the sender.state is updated to reflect the current state
of the connection.

It is not used for any decisions yet but this is required before
algorithms like Eiffel/PRR can be implemented.

Fixes #394

PiperOrigin-RevId: 262638292
2019-08-09 14:50:30 -07:00
Haibo Xu 1c9da886e7 Add initial ptrace stub and syscall support for arm64.
Signed-off-by: Haibo Xu <haibo.xu@arm.com>
Change-Id: I1dbd23bb240cca71d0cc30fc75ca5be28cb4c37c
PiperOrigin-RevId: 262619519
2019-08-09 13:18:11 -07:00
Ayush Ranjan c8961a6cbd ext: Move to pkg/sentry/fsimpl.
fsimpl is the keeper of all filesystem implementations in VFS2.

PiperOrigin-RevId: 262617869
2019-08-09 13:08:28 -07:00
Ayush Ranjan 690308111c ext: Benchmark tests.
Added benchmark tests which emulate memfs benchmarks.

Stat benchmarks
BenchmarkVFS2Ext4fsStat/1-12      	10000000	       145 ns/op
BenchmarkVFS2Ext4fsStat/2-12      	10000000	       170 ns/op
BenchmarkVFS2Ext4fsStat/3-12      	10000000	       202 ns/op
BenchmarkVFS2Ext4fsStat/8-12      	 3000000	       374 ns/op
BenchmarkVFS2Ext4fsStat/64-12     	  500000	      2159 ns/op
BenchmarkVFS2Ext4fsStat/100-12    	  300000	      3459 ns/op

BenchmarkVFS1TmpfsStat/1-12       	 5000000	       348 ns/op
BenchmarkVFS1TmpfsStat/2-12       	 3000000	       487 ns/op
BenchmarkVFS1TmpfsStat/3-12       	 2000000	       655 ns/op
BenchmarkVFS1TmpfsStat/8-12       	 1000000	      1365 ns/op
BenchmarkVFS1TmpfsStat/64-12      	  200000	      9565 ns/op
BenchmarkVFS1TmpfsStat/100-12     	  100000	     15158 ns/op

BenchmarkVFS2MemfsStat/1-12       	10000000	       133 ns/op
BenchmarkVFS2MemfsStat/2-12       	10000000	       155 ns/op
BenchmarkVFS2MemfsStat/3-12       	10000000	       182 ns/op
BenchmarkVFS2MemfsStat/8-12       	 5000000	       310 ns/op
BenchmarkVFS2MemfsStat/64-12      	 1000000	      1659 ns/op
BenchmarkVFS2MemfsStat/100-12     	  500000	      2787 ns/op

Mount Stat benchmarks
BenchmarkVFS2ExtfsMountStat/1-12  	 5000000	       245 ns/op
BenchmarkVFS2ExtfsMountStat/2-12  	 5000000	       266 ns/op
BenchmarkVFS2ExtfsMountStat/3-12  	 5000000	       304 ns/op
BenchmarkVFS2ExtfsMountStat/8-12  	 3000000	       456 ns/op
BenchmarkVFS2ExtfsMountStat/64-12 	  500000	      2308 ns/op
BenchmarkVFS2ExtfsMountStat/100-12   300000	      3482 ns/op

BenchmarkVFS1TmpfsMountStat/1-12  	 3000000	       488 ns/op
BenchmarkVFS1TmpfsMountStat/2-12  	 2000000	       658 ns/op
BenchmarkVFS1TmpfsMountStat/3-12  	 2000000	       806 ns/op
BenchmarkVFS1TmpfsMountStat/8-12  	 1000000	      1514 ns/op
BenchmarkVFS1TmpfsMountStat/64-12 	  100000	     10037 ns/op
BenchmarkVFS1TmpfsMountStat/100-12        100000	     15280 ns/op

BenchmarkVFS2MemfsMountStat/1-12           	10000000	       212 ns/op
BenchmarkVFS2MemfsMountStat/2-12           	 5000000	       232 ns/op
BenchmarkVFS2MemfsMountStat/3-12           	 5000000	       264 ns/op
BenchmarkVFS2MemfsMountStat/8-12           	 3000000	       390 ns/op
BenchmarkVFS2MemfsMountStat/64-12          	 1000000	      1813 ns/op
BenchmarkVFS2MemfsMountStat/100-12         	  500000	      2812 ns/op

PiperOrigin-RevId: 262477158
2019-08-08 18:45:37 -07:00
Rahat Mahmood 7bfad8ebb6 Return a well-defined socket address type from socket funtions.
Previously we were representing socket addresses as an interface{},
which allowed any type which could be binary.Marshal()ed to be used as
a socket address. This is fine when the address is passed to userspace
via the linux ABI, but is problematic when used from within the sentry
such as by networking procfs files.

PiperOrigin-RevId: 262460640
2019-08-08 16:50:33 -07:00
Rahat Mahmood 13a98df49e netstack: Don't start endpoint goroutines too soon on restore.
Endpoint protocol goroutines were previously started as part of
loading the endpoint. This is potentially too soon, as resources used
by these goroutine may not have been loaded. Protocol goroutines may
perform meaningful work as soon as they're started (ex: incoming
connect) which can cause them to indirectly access resources that
haven't been loaded yet.

This CL defers resuming all protocol goroutines until the end of
restore.

PiperOrigin-RevId: 262409429
2019-08-08 12:33:11 -07:00
gVisor bot 2e45d1696e Merge pull request #653 from xiaobo55x:dev
PiperOrigin-RevId: 262402929
2019-08-08 11:58:14 -07:00
Jamie Liu 06102af65a memfs fixes.
- Unexport Filesystem/Dentry/Inode.

- Support SEEK_CUR in directoryFD.Seek().

- Hold Filesystem.mu before touching directoryFD.off in
directoryFD.Seek().

- Remove deleted Dentries from their parent directory.childLists.

- Remove invalid FIXMEs.

PiperOrigin-RevId: 262400633
2019-08-08 11:46:38 -07:00
Ayush Ranjan 08cd5e1d36 ext: Seek unit tests.
PiperOrigin-RevId: 262264674
2019-08-07 19:13:41 -07:00
Ayush Ranjan 40d6d8c15b ext: StatAt unit tests.
PiperOrigin-RevId: 262249166
2019-08-07 17:21:00 -07:00
Ayush Ranjan 3b368cabf9 ext: Read unit tests.
PiperOrigin-RevId: 262242410
2019-08-07 16:44:10 -07:00
Ayush Ranjan ad67e5a7a0 ext: IterDirent unit tests.
PiperOrigin-RevId: 262226761
2019-08-07 15:24:33 -07:00
Ayush Ranjan 1c9781a4ed ext: vfs.FileDescriptionImpl and vfs.FilesystemImpl implementations.
- This also gets rid of pipes for now because pipe does not have vfs2 specific
  support yet.
- Added file path resolution logic.
- Fixes testing infrastructure.
- Does not include unit tests yet.

PiperOrigin-RevId: 262213950
2019-08-07 14:23:42 -07:00
Fabricio Voznika 79cc4397fd Set gofer's OOM score adjustment
Updates #512

PiperOrigin-RevId: 262195448
2019-08-07 12:55:06 -07:00
Tamir Duberstein 67a3f4039d Set target address in ARP Reply
PiperOrigin-RevId: 262163794
2019-08-07 10:27:43 -07:00
Fabricio Voznika e70eafc9e5 Make loading container in a sandbox more robust
PiperOrigin-RevId: 262071646
2019-08-06 23:26:46 -07:00
Bhasker Hariharan dfbc0b0a4c Fix for a panic due to writing to a closed accept channel.
This can happen because endpoint.Close() closes the accept channel first and
then drains/resets any accepted but not delivered connections. But there can be
connections that are connected but not delivered to the channel as the channel
was full. But closing the channel can cause these writes to fail with a write to
a closed channel.

The correct solution is to abort any connections in SYN-RCVD state and
drain/abort all completed connections before closing the accept channel.

PiperOrigin-RevId: 261951132
2019-08-06 11:01:27 -07:00
Michael Pratt 704f9610f3 Require pread/pwrite for splice file offsets
If there is an offset, the file must support pread/pwrite. See
fs/splice.c:do_splice.

PiperOrigin-RevId: 261944932
2019-08-06 10:35:28 -07:00
Samantha Sample fb996668e4 Alter Dockerfiles to include common.go and use a prebuilt JDK.
After the refactoring of the proctor binaries, the Dockerfiles for each
language must be altered to copy the common folder into their image.
Additionally, Java has been changed to use the pre-built version of
JDK-11 from Ubuntu, instead of building it from the source.  This allows
for a smaller image and faster test execution within the container.

PiperOrigin-RevId: 261805158
2019-08-05 17:20:05 -07:00
Samantha Sample 23e740433d Expand runtimes test suite to include Go, Java, PHP, and Python.
This change adds functionality for running more languages using
the runtimes test suite. It divides the languages into separate
test functions, which each call the helper testLang function in the
runtimes_test.go file. This allows them to be run individually
or as a group.

PiperOrigin-RevId: 261791935
2019-08-05 16:11:18 -07:00
Haibo Xu 83fdb7739e Change syscall.EPOLLET to unix.EPOLLET
syscall.EPOLLET has been defined with different values on amd64 and
arm64(-0x80000000 on amd64, and 0x80000000 on arm64), while unix.EPOLLET
has been unified this value to 0x80000000(golang/go#5328). ref #63

Signed-off-by: Haibo Xu <haibo.xu@arm.com>
Change-Id: Id97d075c4e79d86a2ea3227ffbef02d8b00ffbb8
2019-08-05 23:10:08 +00:00
Fabricio Voznika 960a5e5536 Remove stale TODO
This was done in commit 04cbb13ce9

PiperOrigin-RevId: 261414748
2019-08-02 16:35:05 -07:00
Kevin Krakauer 810cc07aab Plumbing for iptables sockopts.
PiperOrigin-RevId: 261413396
2019-08-02 16:26:48 -07:00
Kevin Krakauer b6a5b950d2 Job control: controlling TTYs and foreground process groups.
(Don't worry, this is mostly tests.)

Implemented the following ioctls:
- TIOCSCTTY - set controlling TTY
- TIOCNOTTY - remove controlling tty, maybe signal some other processes
- TIOCGPGRP - get foreground process group. Also enables tcgetpgrp().
- TIOCSPGRP - set foreground process group. Also enabled tcsetpgrp().

Next steps are to actually turn terminal-generated control characters (e.g. C^c)
into signals to the proper process groups, and to send SIGTTOU and SIGTTIN when
appropriate.

PiperOrigin-RevId: 261387276
2019-08-02 14:05:48 -07:00
Fabricio Voznika b461be88a8 Stops container if gofer is killed
Each gofer now has a goroutine that polls on the FDs used
to communicate with the sandbox. The respective gofer is
destroyed if any of the FDs is closed.

Closes #601

PiperOrigin-RevId: 261383725
2019-08-02 13:47:55 -07:00
Rahat Mahmood 2906dffcdb Automated rollback of changelist 261191548
PiperOrigin-RevId: 261373749
2019-08-02 12:52:40 -07:00
Nicolas Lacasse aaaefdf9ca Remove kernel.mounts.
We can get the mount namespace from the CreateProcessArgs in all cases where we
need it. This also gets rid of kernel.Destroy method, since the only thing it
was doing was DecRefing the mounts.

Removing the need to call kernel.SetRootMountNamespace also allowed for some
more simplifications in the container fs setup code.

PiperOrigin-RevId: 261357060
2019-08-02 11:23:11 -07:00
Brett Landau 6a1ac34077 Refactor ListTests() to common.Search().
This change removes the filepath.Walk() function from
proctor- go, php, and nodejs. The filepath.Walk() is
now defined in common.go in Search(). Each proctor binary
passes root directory and testFilter arguments to Search().

proctor-python.go no longer uses filepath.Walk() to search
for tests. There is a built-in list test function within
python's language test suite so that is being used instead.

PiperOrigin-RevId: 261242897
2019-08-01 18:53:35 -07:00
Ian Lewis 3eff0531ad Set sandbox oom_score_adj
Set /proc/self/oom_score_adj based on oomScoreAdj specified in the OCI bundle.
When new containers are added to the sandbox oom_score_adj for the sandbox and
all other gofers are adjusted so that oom_score_adj is equal to the lowest
oom_score_adj of all containers in the sandbox.

Fixes #512

PiperOrigin-RevId: 261242725
2019-08-01 18:49:21 -07:00
Nicolas Lacasse bad43772a1 Drop reference on fs.Inode if Mount goes wrong.
PiperOrigin-RevId: 261203674
2019-08-01 14:57:49 -07:00
Nicolas Lacasse f2b25aeac7 tmpfs and ramfs Dirs should drop references on children in Release().
This is the source of many warnings like:
AtomicRefCount 0x7f5ff84e3500 owned by "fs.Inode" garbage collected with ref count of 1 (want 0)

PiperOrigin-RevId: 261197093
2019-08-01 14:25:14 -07:00
Rahat Mahmood 79511e8a50 Implement getsockopt(TCP_INFO).
Export some readily-available fields for TCP_INFO and stub out the rest.

PiperOrigin-RevId: 261191548
2019-08-01 13:58:48 -07:00