Commit Graph

5444 Commits

Author SHA1 Message Date
Rahat Mahmood 5960674c8f Document how to handle build failures from go-marshal verbosity.
With debugging enabled, go-marshal can generate too much output for
bazel under default configurations, which can cause builds to
fail. The limit defaults to 1 MB.

PiperOrigin-RevId: 372030402
2021-05-04 17:40:08 -07:00
Kevin Krakauer 689b369f57 tcp_socket_test: replace tcp_wmem with SO_SNDBUF
PiperOrigin-RevId: 372022596
2021-05-04 16:50:52 -07:00
Mithun Iyer 682415b6d0 Use cmp.Diff for tcpip.Error comparison
PiperOrigin-RevId: 372021039
2021-05-04 16:45:42 -07:00
Rahat Mahmood e00bd82816 Remove uses of the binary package from the rest of the sentry.
PiperOrigin-RevId: 372020696
2021-05-04 16:41:08 -07:00
Dean Deng d496c285aa Add TODOs to old reference counting utility.
PiperOrigin-RevId: 372012795
2021-05-04 15:58:43 -07:00
Fabricio Voznika 95df852bf2 Make Mount.Type optional for bind mounts
According to the OCI spec Mount.Type is an optional field and it
defaults to "bind" when any of "bind" or "rbind" is included in
Mount.Options.

Also fix the shim to remove bind/rbind from options when mount is
converted from bind to tmpfs inside the Sentry.

Fixes #2330
Fixes #3274

PiperOrigin-RevId: 371996891
2021-05-04 14:36:06 -07:00
Dean Deng dd3875eabe Increase error margin for memory accounting test.
PiperOrigin-RevId: 371963265
2021-05-04 11:55:41 -07:00
gVisor bot 2f53933752 Merge pull request #5903 from zchee:safecopy/fix-args
PiperOrigin-RevId: 371829568
2021-05-03 19:56:12 -07:00
Ghanan Gowripalan 279f9fcee7 Implement standard clock safely
Previously, tcpip.StdClock depended on linking with the unexposed method
time.now to implement tcpip.Clock using the time package. This change
updates the standard clock to not require manually linking to this
unexported method and use publicly documented functions from the time
package.

PiperOrigin-RevId: 371805101
2021-05-03 16:45:48 -07:00
Ghanan Gowripalan f0b3298db0 Convey GSO capabilities through GSOEndpoint
...as all GSO capable endpoints must implement GSOEndpoint.

PiperOrigin-RevId: 371804175
2021-05-03 16:40:47 -07:00
Ting-Yu Wang 4218ba6fb4 netstack: Add a test for mixed Push/Consume
Not really designed to be used this way, but it works and it's been relied
upon. Add a test.

PiperOrigin-RevId: 371802756
2021-05-03 16:33:14 -07:00
Dean Deng 1d92396aaa Temporarily disable atime/mtime check in utimensat test.
PiperOrigin-RevId: 371776583
2021-05-03 14:22:29 -07:00
Fabricio Voznika 1947c87342 Fix deadlock in /proc/[pid]/fd/[num]
In order to resolve path names, fsSymlink.Readlink() may need to reenter
kernfs. Change the code so that kernfs.Inode.Readlink() is called without
locks and document the new contract.

PiperOrigin-RevId: 371770222
2021-05-03 13:50:37 -07:00
Ayush Ranjan 7cafac9f42 [perf] Check caching on IncRef'd dentries before the others.
When a child is added to a parent (directory) dentry, both child and parent are
queued for checkCachingLocked(). Make sure that the parent is queued first
because the parent gained a ref and so could be removed from the LRU cache hence
making space for the new child. This could prevent an LRU cache eviction.

In practice, this did seem to help. ~800 RPCs were reduced while building
//absl/... (ABSL build benchmark). Evictions hurt in 2 ways - create renameMu
contention and destroy a possibly useful dentry which will have to be re-walked
and re-opened later.

Follow up fix for #5859.

PiperOrigin-RevId: 371509392
2021-05-01 12:37:49 -07:00
Andrei Vagin cde8e8b7a9 kvm: prefault a root table page before switching into a user address space
The root table physical page has to be mapped to not fault in iret or sysret
after switching into a user address space.  sysret and iret are in the upper
half that is global and so page tables of lower levels are already mapped.

Fixes #5742

PiperOrigin-RevId: 371458644
2021-04-30 23:07:15 -07:00
Kevin Krakauer 6fb8c01bb4 Fix //test/syscalls:tcp_socket_test_native
The data written was larger than the write buffer, and nobody was reading the
other end.

PiperOrigin-RevId: 371436084
2021-04-30 17:58:13 -07:00
Ghanan Gowripalan eb2b39f702 Comment ip package in a single place
Fixes the below linting error:
```
  From Golint:
  > Package ip has package comment defined in multiple places:
  >   duplicate_address_detection.go
  >   generic_multicast_protocol.go
```

PiperOrigin-RevId: 371430486
2021-04-30 17:06:26 -07:00
Chong Cai ea89cd38a1 Do not return content if verity translate fails
If verification fails for translating mmapped memory, the content should
not be returned. This is not an issue for panic mode, but for error mode
we should return empty content along with the error.

PiperOrigin-RevId: 371393519
2021-04-30 13:26:51 -07:00
Fabricio Voznika c958c5a4f1 Fix up TODOs in the code
PiperOrigin-RevId: 371231148
2021-04-29 16:42:45 -07:00
Ayush Ranjan 9ff0d382d6 [perf] Remove unnecessary existence checks in doCreateAt().
Originally we were making a WalkGetAttrOne RPC to confirm that a file does not
exist on the remote filesystem - when there was no cached information about the
existence of a dentry at that position.
This change avoids making that RPC and speculatively makes the
mkdir/mknod/linkat/symlink RPC. They will fail with EEXIST if a file exists at
that position as we want.

However the error ordering is important. Existence check comes before
writability check. So we make the existence check when the writability check
fails and give it precedence.

This change saves ~76,000 RPCs while building //absl/... (ABSL build benchmark).
That is 10% of all RPCs made while running that workload.

PiperOrigin-RevId: 371225633
2021-04-29 16:12:08 -07:00
Jing Chen eefa00f4ae Implement epoll_pwait2.
PiperOrigin-RevId: 371216407
2021-04-29 15:22:09 -07:00
Chong Cai 669523f7d2 Remove outdated TODOs in verity
PiperOrigin-RevId: 371198372
2021-04-29 14:02:24 -07:00
Ting-Yu Wang a41c5fe217 netstack: Rename pkt.Data().TrimFront() to DeleteFront(), and ...
... it may now invalidate backing slice references

This is currently safe because TrimFront() in VectorisedView only shrinks the
view. This may not hold under the a different buffer implementation.
Reordering method calls order to allow this.

PiperOrigin-RevId: 371167610
2021-04-29 11:43:26 -07:00
Fabricio Voznika 2e442f9081 Remove ResolvingPath.Restart
PiperOrigin-RevId: 371163405
2021-04-29 11:26:04 -07:00
Michael Pratt c03b380c2c Automated rollback of changelist 370733869
PiperOrigin-RevId: 371131985
2021-04-29 09:10:10 -07:00
Fabricio Voznika 26adb3c474 Automated rollback of changelist 369686285
PiperOrigin-RevId: 371015541
2021-04-28 17:02:33 -07:00
Ian Lewis 39fdf0b950 Use containerd v2 config format in docs
Fixes #5170

PiperOrigin-RevId: 371007691
2021-04-28 16:28:46 -07:00
Fabricio Voznika e1149179f3 Remove side effect from test cases
Restore capabilities for tests that need to change them.

PiperOrigin-RevId: 371007047
2021-04-28 16:22:24 -07:00
Fabricio Voznika 704728d38f Disable test that is always skipped
PiperOrigin-RevId: 370989166
2021-04-28 14:44:06 -07:00
Rahat Mahmood f54d87b9ec Remove uses of the binary package from networking code.
Co-Author: ayushranjan
PiperOrigin-RevId: 370785009
2021-04-27 16:20:01 -07:00
Jing Chen 9ec49aabd3 Fix SyscallInfo for epoll_pwait in strace.
PiperOrigin-RevId: 370733869
2021-04-27 12:08:19 -07:00
Koichi Shiraishi cfa02cc126
Fix typo of arg name on assembly files
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-04-28 02:24:24 +09:00
Nayana Bidari 5b207fe783 Remove metrics: fallback, vsyscallCount and partialResult
The newly added Weirdness metric with fields should be used instead of them.

Simple query for weirdness metric: http://shortn/_DGNk0z2Up6

PiperOrigin-RevId: 370578132
2021-04-26 17:37:29 -07:00
gVisor bot 509f5aa57f Merge pull request #5898 from zhlhahaha:2126-1
PiperOrigin-RevId: 370569000
2021-04-26 16:44:04 -07:00
Adin Scannell 36fdc6c9ef Handle tmpfs with 5 fields in /proc/mounts parsing.
PiperOrigin-RevId: 370565903
2021-04-26 16:27:45 -07:00
Adin Scannell 5b7b7daa42 nogo: enable bazel workers and other optimizations.
This is a suite of changes intended to dramatically speed up nogo speed.

First, there are minor changes that help efficiency significantly.

* Gob-based encoding is used internally, and JSON only used for the final
  set of findings. This is done to preserve the existing format (which is
  consumed by external tooling), and to facilitate manual debugging.

* Unnecessary regex compilation is elided in the configuration, and care is
  taken for merges to prevent redundant entries. I'm not sure quite sure how,
  but it turns out that this was consumed a significant amount of time,
  presumably compiling the same regexes over and over again.

Second, this change enables bazel workers for nogo analyzers.

Workers enable persistent processes instead of creating and tearing down a
sandbox every invocation. A library is introduced to abstraction these details,
and allow the tools to still be written using standard flags, etc.

The key here is that these binaries and the core of nogo become aware of
caches with worker.Cache. This allows us to save significant time loading the
same set of files and findings over and over again. These caches are keyed by
the digests that are provided by bazel, and are capped in overall size.

Note that the worker package attempts to capture output during each run, but
tools are no longer permitted to write to stdout. This necessitated dropping
some spurious output from checklocks.

PiperOrigin-RevId: 370505732
2021-04-26 11:42:49 -07:00
Howard Zhang b1f4998873 fix brace misplaced error
Signed-off-by: Howard Zhang <howard.zhang@arm.com>
2021-04-26 14:22:41 +08:00
Chong Cai bf64560681 Add verity tests for stat, deleted/renamed file
PiperOrigin-RevId: 370219558
2021-04-24 00:00:55 -07:00
Adin Scannell f373f67048 Improve nogo action cache-ability.
Presently, the standard library facts are not serialized in a deterministic
order. This means that they have the possibility to change on each iteration,
requiring a large scale re-analysis of all downstream actions, which includes
all packages.

Improve cache-ability of nogo actions by improving the determinism of the both
facts and findings. Internally, default facts should be serialized as a sorted
list for this reason already.

PiperOrigin-RevId: 370188259
2021-04-23 17:33:32 -07:00
Andrei Vagin 80cd26c2f4 hostinet: parse the timeval structure from a SO_TIMESTAMP control message
PiperOrigin-RevId: 370181621
2021-04-23 16:45:03 -07:00
Ayush Ranjan 915b813798 [op] Remove Build job from GitHub actions.
We already have blocking nogo tests which show all findings. This job was
building all nogo targets, and posting all the findings to GitHub as a check
run. Building nogo takes a while so we actually end up wasting a lot of time
doing redundant work.

This is aligned with our goal of moving away from GitHub actions to BuildKite
only.

PiperOrigin-RevId: 370134875
2021-04-23 12:33:48 -07:00
Fabricio Voznika 9e4aa04ad1 Remove side effect from mount tests
Dropping CAP_SYS_ADMIN and not restoring it causes
other tests to be skipped.

PiperOrigin-RevId: 370002644
2021-04-22 20:06:37 -07:00
Fabricio Voznika d1859fe179 Add mlock syscall test
PiperOrigin-RevId: 369993733
2021-04-22 18:38:15 -07:00
Bhasker Hariharan 2739cf4628 Fix AF_UNIX listen() w/ zero backlog.
In https://github.com/google/gvisor/commit/f075522849fa a check to increase zero
to a minimum backlog length was removed from sys_socket.go to bring it in parity
with linux and then in tcp/endpoint.go we bump backlog by 1. But this broke
calling listen on a AF_UNIX socket w/ a zero backlog as in linux it does allow 1
connection even with a zero backlog.

This was caught by a php runtime test socket_abstract_path.phpt.

PiperOrigin-RevId: 369974744
2021-04-22 16:34:00 -07:00
Nayana Bidari 0a6eaed50b Add weirdness sentry metric.
Weirdness metric contains fields to track the number of clock fallback,
partial result and vsyscalls. This metric will avoid the overhead of
having three different metrics (fallbackMetric, partialResultMetric,
vsyscallCount).

PiperOrigin-RevId: 369970218
2021-04-22 16:07:15 -07:00
Rahat Mahmood d93907110e Also report mount options through /proc/<pid>/mounts.
PiperOrigin-RevId: 369967629
2021-04-22 15:52:24 -07:00
Chong Cai dbfdb31e8a Add verity tests for modified file/Merkle file
PiperOrigin-RevId: 369909691
2021-04-22 11:12:56 -07:00
Ghanan Gowripalan 47bc115158 Only carry GSO options in the packet buffer
With this change, GSO options no longer needs to be passed around as
a function argument in the write path.

This change is done in preparation for a later change that defers
segmentation, and may change GSO options for a packet as it flows
down the stack.

Updates #170.

PiperOrigin-RevId: 369774872
2021-04-21 18:09:27 -07:00
gVisor bot 6f9db949d8 Merge pull request #5737 from dqminh:tsc-scaling
PiperOrigin-RevId: 369758655
2021-04-21 16:27:27 -07:00
Rahat Mahmood e3a5da8ce6 Stub the custom "job" controller required by some workloads.
PiperOrigin-RevId: 369724358
2021-04-21 13:36:50 -07:00