Commit Graph

3039 Commits

Author SHA1 Message Date
Eyal Soha d5776be3fb Improve and update packetimpact README.md
PiperOrigin-RevId: 308328860
2020-04-24 14:43:02 -07:00
Adin Scannell c60613475c Standardize all Docker images.
This change moves all Docker images to a standard location, and abstracts the
build process so that they can be maintained in an automated fashion. This also
allows the images to be architecture-independent.

All images will now be referred to by the test framework via the canonical
`gvisor.dev/images/<name>`, where `<name>` is a function of the path within the
source tree.

In a subsequent change, continuous integration will be added so that the images
will always be correct and available locally.

In the end, using `bazel` for Docker containers is simply not possible. Given
that we already have the need to use `make` with the base container (for
Docker), we extend this approach to get more flexibility.

This change also adds a self-documenting and powerful Makefile that is intended
to replace the collection of scripts in scripts. Canonical (self-documenting)
targets can be added here for targets that understand which images need to be
loaded and/or built.

PiperOrigin-RevId: 308322438
2020-04-24 14:11:42 -07:00
Dean Deng f13f26d17d Port SCM Rights to VFS2.
Fixes #1477.

PiperOrigin-RevId: 308317511
2020-04-24 13:46:58 -07:00
Andrei Vagin f87964e829 kokoro: save all files from test.outputs/
If a test fails by timeout, bazel doesn't generate outputs.zip.

PiperOrigin-RevId: 308307815
2020-04-24 12:52:49 -07:00
Ghanan Gowripalan 1ceee04529 Do not copy tcpip.CancellableTimer
A CancellableTimer's AfterFunc timer instance creates a closure over the
CancellableTimer's address. This closure makes a CancellableTimer unsafe
to copy.

No behaviour change, existing tests pass.

PiperOrigin-RevId: 308306664
2020-04-24 12:46:56 -07:00
Dean Deng 632b104aff Plumb context.Context into kernfs.Inode.Open().
PiperOrigin-RevId: 308304793
2020-04-24 12:37:49 -07:00
Fabricio Voznika 2cc0fd42f4 Fixes for procfs
- Return ENOENT for /proc/[pid]/task if task is zoombied or terminated
- Allow directory to be Seek() to the end
- Construct synthetic files for /proc/[pid]/ns/*
- Changed GenericDirectoryFD.Init to not register with FileDescription,
  otherwise other implementation cannot change behavior.

Updates #1195,1193

PiperOrigin-RevId: 308294649
2020-04-24 11:45:19 -07:00
Dean Deng 1b88c63b3e Move hostfs mount to Kernel struct.
This is needed to set up host fds passed through a Unix socket. Note that
the host package depends on kernel, so we cannot set up the hostfs mount
directly in Kernel.Init as we do for sockfs and pipefs.

Also, adjust sockfs to make its setup look more like hostfs's and pipefs's.

PiperOrigin-RevId: 308274053
2020-04-24 10:03:43 -07:00
Dean Deng 40a712c57c Refactor syscall.Fstat calls in hostfs.
Just call syscall.Fstat directly each time mode/file owner are needed. This
feels more natural than using i.getPermissions().

PiperOrigin-RevId: 308257405
2020-04-24 08:20:28 -07:00
gVisor bot 21a54adebc Merge pull request #2509 from prattmic:benchmark_docs
PiperOrigin-RevId: 308215754
2020-04-24 01:41:29 -07:00
Eyal Soha 79542417fe Fix Layer merge and add unit tests
mergo was improperly merging nil and empty strings

PiperOrigin-RevId: 308170862
2020-04-23 18:24:31 -07:00
Rahat Mahmood f01f2132d8 Enable automated marshalling for mempolicy syscalls.
PiperOrigin-RevId: 308170679
2020-04-23 18:20:21 -07:00
Nicolas Lacasse 696feaf10c Port devpts to VFS2.
PiperOrigin-RevId: 308164359
2020-04-23 17:34:29 -07:00
Kevin Krakauer eccae0f77d Remove View.First() and View.RemoveFirst()
These methods let users eaily break the VectorisedView abstraction, and
allowed netstack to slip into pseudo-enforcement of the "all headers are
in the first View" invariant. Removing them and replacing with PullUp(n)
breaks this reliance and will make it easier to add iptables support and
rework network buffer management.

The new View.PullUp(n) method is low cost in the common case, when when
all the headers fit in the first View.

PiperOrigin-RevId: 308163542
2020-04-23 17:28:49 -07:00
Rahat Mahmood 93dd471461 Enable automated marshalling for epoll events.
Ensure we use the correct architecture-specific defintion of epoll
event, and use go-marshal for serialization.

PiperOrigin-RevId: 308145677
2020-04-23 15:49:05 -07:00
Jamie Liu 5042ea7e2c Add vfs.MkdirOptions.ForSyntheticMountpoint.
PiperOrigin-RevId: 308143529
2020-04-23 15:37:10 -07:00
Eyal Soha cc5de905e6 Fix test output so that filenames have the correct path.
Tested:
  Intentionally introduce an error and then run:
  blaze test --test_output=streamed //third_party/gvisor/test/packetimpact/tests:tcp_outside_the_window_linux_test
PiperOrigin-RevId: 308114194
2020-04-23 13:18:59 -07:00
Adin Scannell 2e8c35b506 Add basic GitHub labeler workflow.
This is the first automated GitHub actions workflow, and it simply applies
labels to pull request in a best-effort fashion.

PiperOrigin-RevId: 308112191
2020-04-23 13:02:17 -07:00
gVisor bot ded5c963ae Merge pull request #1819 from lubinszARM:pr_signal_2
PiperOrigin-RevId: 308100771
2020-04-23 12:01:38 -07:00
Adin Scannell 1481499fe2 Simplify Docker test infrastructure.
This change adds a layer of abstraction around the internal Docker APIs,
and eliminates all direct dependencies on Dockerfiles in the infrastructure.

A subsequent change will automated the generation of local images (with
efficient caching). Note that this change drops the use of bazel container
rules, as that experiment does not seem to be viable.

PiperOrigin-RevId: 308095430
2020-04-23 11:33:30 -07:00
Jamie Liu e0c67014cb Factor fsimpl/gofer.host{Preadv,Pwritev} out of fsimpl/gofer.
Also fix returning EOF when 0 bytes are read.

PiperOrigin-RevId: 308089875
2020-04-23 11:08:24 -07:00
Fabricio Voznika 7d1b7daf7e Disable nogo because it breaks Go 1.13
Even though the default build option is to use 1.14, we want to be
want to keep the ability to target different Go versions for testing
and in case the new release has bugs.

PiperOrigin-RevId: 308078876
2020-04-23 10:20:52 -07:00
Eyal Soha a2925a079f Run failing packetimpact test and expect failure.
This will make it easier to notice if a code change causes an existing test to
pass.

PiperOrigin-RevId: 308057978
2020-04-23 08:36:19 -07:00
Nicolas Lacasse e69a871c7b Move user home detection to its own library.
PiperOrigin-RevId: 307977689
2020-04-22 22:18:21 -07:00
Andrei Vagin 0c586946ea Specify a memory file in platform.New().
PiperOrigin-RevId: 307941984
2020-04-22 17:50:10 -07:00
Andrei Vagin 37f863f628 tcp: handle listen after shutdown properly
Right now, sentry panics in this case:
panic: close of nil channel

goroutine 67 [running]:
pkg/tcpip/transport/tcp/tcp.(*endpoint).listen(0xc0000ce000, 0x9, 0x0)
        pkg/tcpip/transport/tcp/endpoint.go:2208 +0x170
pkg/tcpip/transport/tcp/tcp.(*endpoint).Listen(0xc0000ce000, 0x9, 0xc0003a1ad0)
        pkg/tcpip/transport/tcp/endpoint.go:2179 +0x50

Fixes #2468

PiperOrigin-RevId: 307896725
2020-04-22 14:17:11 -07:00
Adin Scannell c31641150d Add GitHub pull request template.
This just provides some sane reminders and ticks a box on the GitHub UI. This
change also cleans up the issue template, as there is already an automatic
link to the repository's security disclosure policy.

PiperOrigin-RevId: 307868833
2020-04-22 12:12:49 -07:00
Adin Scannell a27d6329df Remove unnecessary kokoro configurations.
PiperOrigin-RevId: 307841689
2020-04-22 10:15:10 -07:00
Eyal Soha 6d23673e10 Add comments about deepcopy in Layer.incoming()
PiperOrigin-RevId: 307812340
2020-04-22 07:28:39 -07:00
Dean Deng 5e3596a6b8 Fix set/getsockopt in vfs2 override.
Updates #1476.

PiperOrigin-RevId: 307726055
2020-04-21 19:03:20 -07:00
Dean Deng 80d0a95819 Update gofer.filesystem.BoundEndpointAt() to allow path resolution.
Even though BoundEndpointAt is not yet implemented for gofer fs, allow path
resolution errors to be returned so that we can jump to tmpfs, where it is
implemented.

Updates #1476.

PiperOrigin-RevId: 307718335
2020-04-21 18:00:00 -07:00
gVisor bot 0e013d8b00 Don't ignore override if it is longer than layerStates
PiperOrigin-RevId: 307708653
2020-04-21 16:55:28 -07:00
Fabricio Voznika 37e01fd2ea Misc VFS2 fixes
- Fix defer operation ordering in kernfs.Filesystem.AccessAt()
- Add AT_NULL entry in proc/pid/auvx
- Fix line padding in /proc/pid/maps
- Fix linux_dirent serialization for getdents(2)
- Remove file creation flags from vfs.FileDescription.statusFlags()

Updates #1193, #1035

PiperOrigin-RevId: 307704159
2020-04-21 16:31:53 -07:00
gVisor bot eba086642e Merge pull request #1818 from lubinszARM:pr_signal_1
PiperOrigin-RevId: 307680200
2020-04-21 14:26:08 -07:00
Adin Scannell 89822a4461 Move to GitHub's new issue templates.
This allows us to specify a richer configuration for the issue template, that
effectively moves a lot of the "metadata" from the template itself to the main
issue page.

PiperOrigin-RevId: 307666509
2020-04-21 13:14:09 -07:00
Michael Pratt a471105367 benchmarks: use absolute bazel target
bazel run :benchmarks only works from the benchmarks directory.
bazel run //benchmarks works from anywhere in the workspace.

Also fix help commands, which should be a multiline code section.
2020-04-21 15:52:42 -04:00
Jamie Liu 9b5e305e05 Remove filesystem structure from vfs.Dentry.
This change:

- Drastically simplifies the synchronization model: filesystem structure is
  both implementation-defined and implementation-synchronized.

- Allows implementations of vfs.DentryImpl to use implementation-specific
  dentry types, reducing casts during path traversal.

- Doesn't require dentries representing non-directory files to waste space on a
  map of children.

- Allows dentry revalidation and mount lookup to be correctly ordered (fixed
  FIXME in fsimpl/gofer/filesystem.go).

- Removes the need to have two separate maps in gofer.dentry
  (dentry.vfsd.children and dentry.negativeChildren) for positive and negative
  lookups respectively.

//pkg/sentry/fsimpl/tmpfs/benchmark_test.go:
name                        old time/op  new time/op  delta
VFS2TmpfsStat/1-112          172ns ± 4%   165ns ± 3%   -4.08%  (p=0.002 n=9+9)
VFS2TmpfsStat/2-112          199ns ± 3%   195ns ±10%     ~     (p=0.132 n=8+9)
VFS2TmpfsStat/3-112          230ns ± 2%   216ns ± 2%   -6.15%  (p=0.000 n=8+8)
VFS2TmpfsStat/8-112          390ns ± 2%   358ns ± 4%   -8.33%  (p=0.000 n=9+8)
VFS2TmpfsStat/64-112        2.20µs ± 3%  2.01µs ± 3%   -8.48%  (p=0.000 n=10+8)
VFS2TmpfsStat/100-112       3.42µs ± 9%  3.08µs ± 2%   -9.82%  (p=0.000 n=9+8)
VFS2TmpfsMountStat/1-112     278ns ± 1%   286ns ±15%     ~     (p=0.712 n=8+10)
VFS2TmpfsMountStat/2-112     311ns ± 4%   298ns ± 2%   -4.27%  (p=0.000 n=9+8)
VFS2TmpfsMountStat/3-112     339ns ± 3%   330ns ± 9%     ~     (p=0.070 n=8+9)
VFS2TmpfsMountStat/8-112     503ns ± 3%   466ns ± 3%   -7.38%  (p=0.000 n=8+8)
VFS2TmpfsMountStat/64-112   2.53µs ±16%  2.17µs ± 7%  -14.19%  (p=0.000 n=10+9)
VFS2TmpfsMountStat/100-112  3.60µs ± 4%  3.30µs ± 8%   -8.33%  (p=0.001 n=8+9)

Updates #1035

PiperOrigin-RevId: 307655892
2020-04-21 12:18:07 -07:00
Kevin Krakauer 639c8dd808 Restore euid upon test finish
PiperOrigin-RevId: 307638329
2020-04-21 10:58:00 -07:00
gVisor bot 8b72623e6a Internal change.
PiperOrigin-RevId: 307622320
2020-04-21 09:44:00 -07:00
Dave Bailey 7c0f3bc857 Sentry metrics updates.
Sentry metrics with nanoseconds units are labeled as such, and non-cumulative
sentry metrics are supported.

PiperOrigin-RevId: 307621080
2020-04-21 09:36:43 -07:00
gVisor bot 120d3b50f4 Automated rollback of changelist 307477185
PiperOrigin-RevId: 307598974
2020-04-21 07:16:30 -07:00
Adin Scannell c615aafa21 Add internal nogo analysis & checkescape tool.
See tools/nogo/README.md.

The checkescape tool is able to perform recursive escape analysis, using the
actual generated binary to confirm the results produced by the compiler itself.

As an initial use case, this replaces the manual escape analysis tests used for
go_marshal, and validates that the CopyIn and CopyOut paths will not require
any allocation or stack splits.

Updates #2243

PiperOrigin-RevId: 307532986
2020-04-20 20:58:20 -07:00
Ghanan Gowripalan 782041509f Prevent race when reassigning CancellableTimer
Capture a timer's locker for each instance of a CancellableTimer so that
reassigning a tcpip.CancellableTimer does not cause a data race.

Reassigning a tcpip.CancellableTimer updates its underlying locker. When
a timer fires, it does a read of the timer's locker variable to lock it.
This read of the locker was not synchronized so a race existed where one
goroutine may reassign the timer (updating the locker) and another
handles the timer firing (attempts to lock the timer's locker).

Test: tcpip_test.TestCancellableTimerReassignment
PiperOrigin-RevId: 307499822
2020-04-20 16:32:44 -07:00
Adin Scannell 1a597e01be Add a functional vm_test for root_test.
This change renames the tools/images directory to tools/vm for clarity, and
adds a functional vm_test. Sharding is also added to the same test, and some
documentation added around key flags & variables to describe how they work.

Subsequent changes will add vm_tests for other cases, such as the runtime tests.

PiperOrigin-RevId: 307492245
2020-04-20 15:48:27 -07:00
gVisor bot 07b1b4cc98 Merge pull request #2313 from kevinGC:firstn
PiperOrigin-RevId: 307477185
2020-04-20 14:31:12 -07:00
Ting-Yu Wang 470633d7e9 Fix release.sh. git commands need to be run in git repo.
PiperOrigin-RevId: 307458938
2020-04-20 12:58:32 -07:00
gVisor bot 763d16f912 Merge pull request #2060 from xiaobo55x:rseq
PiperOrigin-RevId: 307453436
2020-04-20 12:31:22 -07:00
Adin Scannell 9ba3086d9d Move runtime_tests.sh to align with other scripts.
PiperOrigin-RevId: 307435879
2020-04-20 11:08:59 -07:00
Jamie Liu e72ce8cce4 Change lingering uses of "memfs" in fsimpl/tmpfs to "tmpfs".
PiperOrigin-RevId: 307422746
2020-04-20 10:10:35 -07:00
Dean Deng 1a940f2b6c Resolve issue with file mode for host fds.
Instead of plumbing error through kernfs.Inode.Mode, panic if err != nil.
The errors that can result from an fstat syscall all indicate that something
is fundamentally wrong, and panicking should be acceptable.

PiperOrigin-RevId: 307406847
2020-04-20 08:51:47 -07:00