Commit Graph

4926 Commits

Author SHA1 Message Date
Fabricio Voznika 12ac31ed04 fsgofer optimizations
- Skip chown call in case owner change is not needed
- Skip filepath.Clean() calls when joining paths
- Pass unix.Stat_t by value to reduce runtime.duffcopy calls.
  This change allows for better inlining in localFile.walk().

                                Change            Baseline    Improvement
BenchmarkWalkOne-6           	 2912 ns/op       3082 ns/op     5.5%
BenchmarkCreate-6            	15915 ns/op      19126 ns/op    16.8%
BenchmarkCreateDiffOwner-6	18795 ns/op      19741 ns/op     4.8%

PiperOrigin-RevId: 347667833
2020-12-15 12:23:55 -08:00
gVisor bot 5843a5007c Merge pull request #4722 from zhlhahaha:2010
PiperOrigin-RevId: 347660920
2020-12-15 11:54:04 -08:00
Nayana Bidari b15acae9a6 Fix error code for connect in raw sockets.
PiperOrigin-RevId: 347650354
2020-12-15 11:07:48 -08:00
Adin Scannell 4e963c99ce Cleanup GitHub actions workflows.
Also, drop the pull_request template, since this has not proved to be
helpful, and just results in a commit message the includes the list.

PiperOrigin-RevId: 347636507
2020-12-15 10:05:47 -08:00
Ting-Yu Wang 25ebddbddf Fix a data race in packetEPs
packetEPs may get into a state that `len < cap`, casuing append() modifying the
original slice storage.

Reported-by: syzbot+978dd0e9c2600ab7a76b@syzkaller.appspotmail.com
PiperOrigin-RevId: 347634351
2020-12-15 09:55:40 -08:00
Fabricio Voznika b2a6973348 Update containerd/cgroups
PiperOrigin-RevId: 347532687
2020-12-14 21:18:06 -08:00
Ayush Ranjan a1c56bc227 [netstack] Update raw socket and hostinet control message parsing.
There are surprisingly few syscall tests that run with hostinet. For example
running the following command only returns two results:
`bazel query test/syscalls:all | grep hostnet`

I think as a result, as our control messages evolved, hostinet was left
behind. Update it to support all control messages netstack supports.

This change also updates sentry's control message parsing logic to make it up to
date with all the control messages we support.

PiperOrigin-RevId: 347508892
2020-12-14 18:00:55 -08:00
Nayana Bidari 2e191cb3f7 Move SO_LINGER option to socketops.
PiperOrigin-RevId: 347437786
2020-12-14 12:03:27 -08:00
Dean Deng 65e4ed8fbe Do not check for reference leaks after saving.
We should not assert that all resources are dropped after saving.

PiperOrigin-RevId: 347420131
2020-12-14 10:47:01 -08:00
Nayana Bidari ab593661ef Move SO_ERROR and SO_OOBINLINE option to socketops.
SO_OOBINLINE option is set/get as boolean value, which is the same as linux.
As we currently do not support disabling this option, we always return it as
true.

PiperOrigin-RevId: 347413905
2020-12-14 10:22:32 -08:00
Toshi Kikuchi 08d36b6c63 Reduce the memory overhead in IP fragment management
- Deep-copy pkt.Data and hold it instead of shallow-copy (vv.Clone).
  This allows the pkt's backing array, which includes the header portion,
  to be freed.
- Remove fragHeap. The fragments are now held in holes struct instead.
- Stop reserving the initial capacity of holes slice.

PiperOrigin-RevId: 347198744
2020-12-12 15:32:03 -08:00
Bruno Dal Bo 4aef908c92 Introduce IPv6 extension header serialization facilities
Adds IPv6 extension header serializer and Hop by Hop options serializer.
Add RouterAlert option serializer and use it in MLD.

Fixed #4996

Startblock:
  has LGTM from marinaciocea
  and then
  add reviewer ghanan
PiperOrigin-RevId: 347174537
2020-12-12 09:07:44 -08:00
Zach Koopmans 4b697aae55 Fix long running bazel build jobs.
- Skip the bazel clean command on the last run of the benchmark.
- Use --test.benchtime=1ns to force running the benchmark once
(https://github.com/golang/go/issues/32051)

PiperOrigin-RevId: 347124606
2020-12-11 20:56:09 -08:00
gVisor bot be5922fbd2 Internal change.
PiperOrigin-RevId: 347091372
2020-12-11 16:06:01 -08:00
Dean Deng 1e92732eb1 Make fixes to vfs2 leak checking.
PiperOrigin-RevId: 347089828
2020-12-11 15:55:41 -08:00
Dean Deng 80379894d3 Add runsc symbolize command.
This command takes instruction pointers from stdin and converts them into their
corresponding file names and line/column numbers in the runsc source code. The
inputs are not interpreted as actual addresses, but as synthetic values that are
exposed through /sys/kernel/debug/kcov. One can extract coverage information
from kcov and translate those values into locations in the source code by
running symbolize on the same runsc binary.

This will allow us to generate syzkaller coverage reports.

PiperOrigin-RevId: 347089624
2020-12-11 15:43:22 -08:00
Nayana Bidari d45420b152 Fix panic when IPv4 address is used in sendmsg for IPv6 sockets
We do not rely on error for getsockopt options(which have boolean values)
anymore. This will cause issue in sendmsg where we used to return error
for IPV6_V6Only option. Fix the panic by returning error (for sockets other
than TCP and UDP) if the address does not match the type(AF_INET/AF_INET6) of
the socket.

PiperOrigin-RevId: 347063838
2020-12-11 13:31:28 -08:00
Adin Scannell 5bdc167d17 Fix run and sudo targets.
These are not passing arguments properly. This breaks the current
pre-command for BuildKite.

PiperOrigin-RevId: 347062729
2020-12-11 13:25:44 -08:00
Zach Koopmans 305a456551 Adjust requests to be constant equal to b.N in network tests.
For "hey", requests >= concurrency. b.N can be set by the
--test.benchtime={b.N}x. The previous setting of b.N * c
can be surprisingly slow for larger c.

Set the requests to max(b.N, c) and log to the user if it is c.

PiperOrigin-RevId: 347053675
2020-12-11 12:37:28 -08:00
Adin Scannell 4cba3904f4 Remove existing nogo exceptions.
PiperOrigin-RevId: 347047550
2020-12-11 12:06:49 -08:00
Adin Scannell 76c2f21cec Add local hooks for BuildKite.
PiperOrigin-RevId: 347044353
2020-12-11 11:52:43 -08:00
Zach Koopmans 3c673caf86 Fix parser to include iterations.
PiperOrigin-RevId: 347038652
2020-12-11 11:26:42 -08:00
Ayush Ranjan af4afdc0e0 [netstack] Decouple tcpip.ControlMessages from the IP control messges.
tcpip.ControlMessages can not contain Linux specific structures which makes it
painful to convert back and forth from Linux to tcpip back to Linux when passing
around control messages in hostinet and raw sockets.

Now we convert to the Linux version of the control message as soon as we are
out of tcpip.

PiperOrigin-RevId: 347027065
2020-12-11 10:33:58 -08:00
Adin Scannell e0cde3fb87 Ensure individual steps timeout in case of infinite hang.
Also, add a basic release test.

PiperOrigin-RevId: 347016796
2020-12-11 09:45:24 -08:00
Jing Chen 73eccab91e Make semctl IPC_INFO cmd return the index of highest used entry.
PiperOrigin-RevId: 346973338
2020-12-11 04:22:04 -08:00
Ian Lewis 0eb4acad37 Fix website build
Fix 'run' function call so that parameters are passed properly to the function.

PiperOrigin-RevId: 346929952
2020-12-10 21:50:03 -08:00
Chong Cai e7279936e8 Change merkle root file name to avoid collision
PiperOrigin-RevId: 346923826
2020-12-10 20:53:00 -08:00
Bhasker Hariharan bcb97a3bb7 Disable host reassembly for fragments.
fdbased endpoint was enabling fragment reassembly on the host AF_PACKET socket
to ensure that fragments are delivered inorder to the right dispatcher. But this
prevents fragments from being delivered to gvisor at all and makes testing of
gvisor's fragment reassembly code impossible.

The potential impact from this is minimal since IP Fragmentation is not really
that prevelant and in cases where we do get fragments we may deliver the
fragment out of order to the TCP layer as multiple network dispatchers may
process the fragments and deliver a reassembled fragment after the next packet
has been delivered to the TCP endpoint. While not desirable I believe the impact
from this is minimal due to low prevalence of fragmentation.

Also removed PktType and Hatype fields when binding the socket as these are not
used when binding. Its just confusing to have them specified.

See: https://man7.org/linux/man-pages/man7/packet.7.html
"Fields used for binding are
       sll_family (should be AF_PACKET), sll_protocol, and sll_ifindex."

Fixes #5055

PiperOrigin-RevId: 346919439
2020-12-10 20:08:59 -08:00
Ian Lewis 28c3260d74 Add simple guidelines & instructions for blog contribution
Modeled after knative's blog guidelines.
https://github.com/knative/docs/blob/master/blog/README.md

PiperOrigin-RevId: 346905713
2020-12-10 18:06:46 -08:00
Dean Deng dbc86593db Fix typo in go template error messages.
PiperOrigin-RevId: 346878344
2020-12-10 15:31:24 -08:00
Jamie Liu ed0c7be614 Proposal for runtime.DedicateOSThread().
Updates #2184

PiperOrigin-RevId: 346875966
2020-12-10 15:19:10 -08:00
Ghanan Gowripalan 53a95ad0df Use specified source address for IGMP/MLD packets
This change also considers interfaces and network endpoints enabled up
up to the point all work to disable them are complete. This was needed
so that protocols can perform shutdown work while being disabled (e.g.
sending a packet which requires the endpoint to be enabled to obtain a
source address).

Bug #4682, #4861
Fixes #4888

Startblock:
  has LGTM from peterjohnston
  and then
  add reviewer brunodalbo
PiperOrigin-RevId: 346869702
2020-12-10 14:50:20 -08:00
Tamir Duberstein 01126d47fc Log window size
Without this change, this test produces quite opaque errors as valid TCP
segments are not logged; what we see in the end is any random traffic that
happened *after* the final TCP segment.

PiperOrigin-RevId: 346864169
2020-12-10 14:23:10 -08:00
Tamir Duberstein 7fa0a57fc9 Uncomment payload generation
PiperOrigin-RevId: 346852543
2020-12-10 13:27:43 -08:00
Tamir Duberstein 205015687b Correctly align non-matching layers
PiperOrigin-RevId: 346840424
2020-12-10 12:27:46 -08:00
Tamir Duberstein a177bfdbc4 Correctly print non-matching layers
PiperOrigin-RevId: 346818310
2020-12-10 10:52:13 -08:00
Adin Scannell 65a2242db4 Tweak aarch64 support.
A few images were broken with respect to aarch64. We should now
be able to run push-all-images with ARCH=aarch64 as part of the
regular continuous integration builds, and add aarch64 smoke tests
(via user emulation for now) to the regular test suite (future).

PiperOrigin-RevId: 346685462
2020-12-09 18:51:17 -08:00
Bhasker Hariharan 92ca72ecb7 Add support for IP_RECVORIGDSTADDR IP option.
Fixes #5004

PiperOrigin-RevId: 346643745
2020-12-09 15:58:53 -08:00
Zach Koopmans b4af9d4572 Add network benchmarks jobs
Add httpd, nginx, node, and ruby benchmarks to continuous jobs.

PiperOrigin-RevId: 346629115
2020-12-09 15:57:58 -08:00
Jamie Liu aaf4901c8c Add //pkg/sync:generic_atomicptrmap.
AtomicPtrMap is a generic concurrent map from arbitrary keys to arbitrary
pointer values.

Benchmarks:
name                                                            time/op
StoreDelete/RWMutexMap-12                                        335ns ± 1%
StoreDelete/SyncMap-12                                           705ns ± 3%
StoreDelete/AtomicPtrMap-12                                      287ns ± 4%
StoreDelete/AtomicPtrMapSharded-12                               289ns ± 1%
LoadOrStoreDelete/RWMutexMap-12                                  342ns ± 2%
LoadOrStoreDelete/SyncMap-12                                     662ns ± 2%
LoadOrStoreDelete/AtomicPtrMap-12                                290ns ± 7%
LoadOrStoreDelete/AtomicPtrMapSharded-12                         293ns ± 2%
LookupPositive/RWMutexMap-12                                     101ns ±26%
LookupPositive/SyncMap-12                                        202ns ± 2%
LookupPositive/AtomicPtrMap-12                                  71.1ns ± 2%
LookupPositive/AtomicPtrMapSharded-12                           73.2ns ± 1%
LookupNegative/RWMutexMap-12                                     119ns ± 1%
LookupNegative/SyncMap-12                                        154ns ± 1%
LookupNegative/AtomicPtrMap-12                                  84.7ns ± 3%
LookupNegative/AtomicPtrMapSharded-12                           86.8ns ± 1%
Concurrent/FixedKeys_1PercentWrites_RWMutexMap-12               1.32µs ± 2%
Concurrent/FixedKeys_1PercentWrites_SyncMap-12                  52.7ns ±10%
Concurrent/FixedKeys_1PercentWrites_AtomicPtrMap-12             31.8ns ±20%
Concurrent/FixedKeys_1PercentWrites_AtomicPtrMapSharded-12      24.0ns ±15%
Concurrent/FixedKeys_10PercentWrites_RWMutexMap-12               860ns ± 3%
Concurrent/FixedKeys_10PercentWrites_SyncMap-12                 68.8ns ±20%
Concurrent/FixedKeys_10PercentWrites_AtomicPtrMap-12            98.6ns ± 7%
Concurrent/FixedKeys_10PercentWrites_AtomicPtrMapSharded-12     42.0ns ±25%
Concurrent/FixedKeys_50PercentWrites_RWMutexMap-12              1.17µs ± 3%
Concurrent/FixedKeys_50PercentWrites_SyncMap-12                  136ns ±34%
Concurrent/FixedKeys_50PercentWrites_AtomicPtrMap-12             286ns ± 3%
Concurrent/FixedKeys_50PercentWrites_AtomicPtrMapSharded-12      115ns ±35%
Concurrent/ChangingKeys_1PercentWrites_RWMutexMap-12            1.27µs ± 2%
Concurrent/ChangingKeys_1PercentWrites_SyncMap-12               5.01µs ± 3%
Concurrent/ChangingKeys_1PercentWrites_AtomicPtrMap-12          38.1ns ± 3%
Concurrent/ChangingKeys_1PercentWrites_AtomicPtrMapSharded-12   22.6ns ± 2%
Concurrent/ChangingKeys_10PercentWrites_RWMutexMap-12           1.08µs ± 2%
Concurrent/ChangingKeys_10PercentWrites_SyncMap-12              5.97µs ± 1%
Concurrent/ChangingKeys_10PercentWrites_AtomicPtrMap-12          390ns ± 2%
Concurrent/ChangingKeys_10PercentWrites_AtomicPtrMapSharded-12  93.6ns ± 1%
Concurrent/ChangingKeys_50PercentWrites_RWMutexMap-12           1.77µs ± 2%
Concurrent/ChangingKeys_50PercentWrites_SyncMap-12              8.07µs ± 2%
Concurrent/ChangingKeys_50PercentWrites_AtomicPtrMap-12         1.61µs ± 2%
Concurrent/ChangingKeys_50PercentWrites_AtomicPtrMapSharded-12   386ns ± 1%

Updates #231

PiperOrigin-RevId: 346614776
2020-12-09 15:57:02 -08:00
Zach Koopmans 992769c774 Add tensorflow, ffmpeg, and redis jobs.
PiperOrigin-RevId: 346603153
2020-12-09 15:56:08 -08:00
Ayush Ranjan 45619ded38 [netstack] Make tcpip.Error savable.
Earlier we could not save tcpip.Error objects in structs because upon restore
the constant's address changes in netstack's error translation map and
translating the error would panic because the map is based on the address of the
tcpip.Error instead of the error itself.

Now I made that translations map use the error message as key instead of the
address. Added relevant synchronization mechanisms to protect the structure
and initialize it upon restore.

PiperOrigin-RevId: 346590485
2020-12-09 15:55:12 -08:00
Ghanan Gowripalan 50189b0d6f Do not perform IGMP/MLD on loopback interfaces
The loopback interface will never have any neighbouring nodes so
advertising its interest in multicast groups is unnecessary.

Bug #4682, #4861

Startblock:
  has LGTM from asfez
  and then
  add reviewer tamird
PiperOrigin-RevId: 346587604
2020-12-09 15:54:18 -08:00
Adin Scannell a855a814d6 Refactor the Makefile to avoid recursive Make.
Recursive make is difficult to follow and debug. Drop this by using
internal functions, which, while difficult, are easier than trying to
following recursive invokations.

Further simplify the Makefile by collapsing the image bits and removing
the tools/vm directory, which is effectively unused.

Fixes #4952

PiperOrigin-RevId: 346569133
2020-12-09 15:53:23 -08:00
Bruno Dal Bo f6cb96bd57 Cap UDP payload size to length informed in UDP header
startblock:
  has LGTM from peterjohnston
  and then
  add reviewer ghanan,tamird
PiperOrigin-RevId: 346565589
2020-12-09 15:52:28 -08:00
Andrei Vagin 658f874b94 Prepare for supporting cross compilation.
PiperOrigin-RevId: 346496532
2020-12-09 15:51:33 -08:00
Zeling Feng 96d14de0fa export MountTempDirectory
PiperOrigin-RevId: 346487763
2020-12-09 15:50:35 -08:00
Adin Scannell 1a778397db Advance RevId
This is an empty commit that advances the RevId below, to fix a
repository synchronization issue.

PiperOrigin-RevId: 346465449
2020-12-09 14:04:38 -08:00
Tamir Duberstein cc522a9cfb Run tcpdump as root
Prior to this change tcpdump would fail to create its output file
because the destination directory was owned by root. This would later
cause killall to fail, as tcpdump was not running. Check exit code of
tcpdump/tshark to produce better error messages should this regress.

PiperOrigin-RevId: 346353911
2020-12-08 10:20:38 -08:00
Tamir Duberstein 5ea6419478 Consolidate sniffer program logic
Avoid action at a distance where both `snifferArgs` and `snifferProg`
must stay in sync.

PiperOrigin-RevId: 346341231
2020-12-08 09:24:02 -08:00