Commit Graph

30 Commits

Author SHA1 Message Date
Jamie Liu f051ec6463 Add gohacks.Slice/StringHeader.
See https://github.com/golang/go/issues/19367 for rationale. Note that the
upstream decision arrived at in that thread, while useful for some of our use
cases, doesn't account for all of our SliceHeader use cases (we often use
SliceHeader to extract pointers from slices in a way that avoids bounds
checking and/or handles nil slices correctly) and also doesn't exist yet.

PiperOrigin-RevId: 358071574
2021-02-17 17:41:10 -08:00
Michael Pratt 665c966084 Internal change.
PiperOrigin-RevId: 354327882
2021-01-28 09:31:11 -08:00
Adin Scannell 4cba3904f4 Remove existing nogo exceptions.
PiperOrigin-RevId: 347047550
2020-12-11 12:06:49 -08:00
Jamie Liu aaf4901c8c Add //pkg/sync:generic_atomicptrmap.
AtomicPtrMap is a generic concurrent map from arbitrary keys to arbitrary
pointer values.

Benchmarks:
name                                                            time/op
StoreDelete/RWMutexMap-12                                        335ns ± 1%
StoreDelete/SyncMap-12                                           705ns ± 3%
StoreDelete/AtomicPtrMap-12                                      287ns ± 4%
StoreDelete/AtomicPtrMapSharded-12                               289ns ± 1%
LoadOrStoreDelete/RWMutexMap-12                                  342ns ± 2%
LoadOrStoreDelete/SyncMap-12                                     662ns ± 2%
LoadOrStoreDelete/AtomicPtrMap-12                                290ns ± 7%
LoadOrStoreDelete/AtomicPtrMapSharded-12                         293ns ± 2%
LookupPositive/RWMutexMap-12                                     101ns ±26%
LookupPositive/SyncMap-12                                        202ns ± 2%
LookupPositive/AtomicPtrMap-12                                  71.1ns ± 2%
LookupPositive/AtomicPtrMapSharded-12                           73.2ns ± 1%
LookupNegative/RWMutexMap-12                                     119ns ± 1%
LookupNegative/SyncMap-12                                        154ns ± 1%
LookupNegative/AtomicPtrMap-12                                  84.7ns ± 3%
LookupNegative/AtomicPtrMapSharded-12                           86.8ns ± 1%
Concurrent/FixedKeys_1PercentWrites_RWMutexMap-12               1.32µs ± 2%
Concurrent/FixedKeys_1PercentWrites_SyncMap-12                  52.7ns ±10%
Concurrent/FixedKeys_1PercentWrites_AtomicPtrMap-12             31.8ns ±20%
Concurrent/FixedKeys_1PercentWrites_AtomicPtrMapSharded-12      24.0ns ±15%
Concurrent/FixedKeys_10PercentWrites_RWMutexMap-12               860ns ± 3%
Concurrent/FixedKeys_10PercentWrites_SyncMap-12                 68.8ns ±20%
Concurrent/FixedKeys_10PercentWrites_AtomicPtrMap-12            98.6ns ± 7%
Concurrent/FixedKeys_10PercentWrites_AtomicPtrMapSharded-12     42.0ns ±25%
Concurrent/FixedKeys_50PercentWrites_RWMutexMap-12              1.17µs ± 3%
Concurrent/FixedKeys_50PercentWrites_SyncMap-12                  136ns ±34%
Concurrent/FixedKeys_50PercentWrites_AtomicPtrMap-12             286ns ± 3%
Concurrent/FixedKeys_50PercentWrites_AtomicPtrMapSharded-12      115ns ±35%
Concurrent/ChangingKeys_1PercentWrites_RWMutexMap-12            1.27µs ± 2%
Concurrent/ChangingKeys_1PercentWrites_SyncMap-12               5.01µs ± 3%
Concurrent/ChangingKeys_1PercentWrites_AtomicPtrMap-12          38.1ns ± 3%
Concurrent/ChangingKeys_1PercentWrites_AtomicPtrMapSharded-12   22.6ns ± 2%
Concurrent/ChangingKeys_10PercentWrites_RWMutexMap-12           1.08µs ± 2%
Concurrent/ChangingKeys_10PercentWrites_SyncMap-12              5.97µs ± 1%
Concurrent/ChangingKeys_10PercentWrites_AtomicPtrMap-12          390ns ± 2%
Concurrent/ChangingKeys_10PercentWrites_AtomicPtrMapSharded-12  93.6ns ± 1%
Concurrent/ChangingKeys_50PercentWrites_RWMutexMap-12           1.77µs ± 2%
Concurrent/ChangingKeys_50PercentWrites_SyncMap-12              8.07µs ± 2%
Concurrent/ChangingKeys_50PercentWrites_AtomicPtrMap-12         1.61µs ± 2%
Concurrent/ChangingKeys_50PercentWrites_AtomicPtrMapSharded-12   386ns ± 1%

Updates #231

PiperOrigin-RevId: 346614776
2020-12-09 15:57:02 -08:00
Jamie Liu 8a45c81616 Allow use of SeqAtomic with pointer-containing types.
Per runtime.memmove, pointers are always copied atomically, as this is required
by the GC. (Also, the init() safety check doesn't work because it gets renamed
to <prefix>init() by template instantiation.)

PiperOrigin-RevId: 345800302
2020-12-04 19:07:17 -08:00
Michael Pratt 6708c8c97e Require sync.RWMutex to lock and unlock from the same goroutine
This is the RWMutex equivalent to the preceding sync.Mutex CL.

Updates #4804

PiperOrigin-RevId: 345681051
2020-12-04 08:20:08 -08:00
Jamie Liu ed8bdf461b Consolidate most synchronization primitive linknames in the sync package.
PiperOrigin-RevId: 345359823
2020-12-02 19:08:32 -08:00
Adin Scannell 3deb5d0c04 Fix link against runtime.goyield.
This function does not exist in Go 1.13. We need to add an adaptor
to build against Go 1.13, which is the default Ubuntu version.

PiperOrigin-RevId: 343929132
2020-11-23 14:30:23 -08:00
Michael Pratt 3454d57219 Require sync.Mutex to lock and unlock from the same goroutine
We would like to track locks ordering to detect ordering violations. Detecting
violations is much simpler if mutexes must be unlocked by the same goroutine
that locked them.

Thus, as a first step to tracking lock ordering, add this lock/unlock
requirement to gVisor's sync.Mutex. This is more strict than the Go standard
library's sync.Mutex, but initial testing indicates only a single lock that is
used across goroutines. The new sync.CrossGoroutineMutex relaxes the
requirement (but will not provide lock order checking).

Due to the additional overhead, enforcement is only enabled with the
"checklocks" build tag. Build with this tag using:

bazel build --define=gotags=checklocks ...

From my spot-checking, this has no changed inlining properties when disabled.

Updates #4804

PiperOrigin-RevId: 343370200
2020-11-19 14:29:34 -08:00
Jamie Liu 5c66011200 Update Go version constraint on sync/spin_unsafe.go.
PiperOrigin-RevId: 329801584
2020-09-02 15:37:26 -07:00
Jamie Liu 9bd0164237 Improve sync.SeqCount performance.
- Make sync.SeqCountEpoch not a struct. This allows sync.SeqCount.BeginRead()
  to be inlined.

- Mark sync.SeqAtomicLoad<T> nosplit to mitigate the Go compiler's refusal to
  inline it. (Best I could get was "cost 92 exceeds budget 80".)

- Use runtime-guided spinning in SeqCount.BeginRead().

Benchmarks:
name                               old time/op  new time/op   delta
pkg:pkg/sync/sync goos:linux goarch:amd64
SeqCountWriteUncontended-12        8.24ns ± 0%  11.40ns ± 0%  +38.35%  (p=0.000 n=10+10)
SeqCountReadUncontended-12         0.33ns ± 0%   0.14ns ± 3%  -57.77%  (p=0.000 n=7+8)
pkg:pkg/sync/seqatomictest/seqatomic goos:linux goarch:amd64
SeqAtomicLoadIntUncontended-12     0.64ns ± 1%   0.41ns ± 1%  -36.40%  (p=0.000 n=10+8)
SeqAtomicTryLoadIntUncontended-12  0.18ns ± 4%   0.18ns ± 1%     ~     (p=0.206 n=10+8)
AtomicValueLoadIntUncontended-12   0.27ns ± 3%   0.27ns ± 0%   -1.77%  (p=0.000 n=10+8)

(atomic.Value.Load is, of course, inlined. We would expect an uncontended
inline SeqAtomicLoad<int> to perform identically to SeqAtomicTryLoad<int>.) The
"regression" in BenchmarkSeqCountWriteUncontended, despite this CL changing
nothing in that path, is attributed to microarchitectural subtlety; the
benchmark loop is unchanged except for its address:

Before this CL:
  :0                    0x4e62d1                48ffc2                  INCQ DX
  :0                    0x4e62d4                48399110010000          CMPQ DX, 0x110(CX)
  :0                    0x4e62db                7e26                    JLE 0x4e6303
  :0                    0x4e62dd                90                      NOPL
  :0                    0x4e62de                bb01000000              MOVL $0x1, BX
  :0                    0x4e62e3                f00fc118                LOCK XADDL BX, 0(AX)
  :0                    0x4e62e7                ffc3                    INCL BX
  :0                    0x4e62e9                0fbae300                BTL $0x0, BX
  :0                    0x4e62ed                733a                    JAE 0x4e6329
  :0                    0x4e62ef                90                      NOPL
  :0                    0x4e62f0                bb01000000              MOVL $0x1, BX
  :0                    0x4e62f5                f00fc118                LOCK XADDL BX, 0(AX)
  :0                    0x4e62f9                ffc3                    INCL BX
  :0                    0x4e62fb                0fbae300                BTL $0x0, BX
  :0                    0x4e62ff                73d0                    JAE 0x4e62d1

After this CL:
  :0                    0x4e6361                48ffc2                  INCQ DX
  :0                    0x4e6364                48399110010000          CMPQ DX, 0x110(CX)
  :0                    0x4e636b                7e26                    JLE 0x4e6393
  :0                    0x4e636d                90                      NOPL
  :0                    0x4e636e                bb01000000              MOVL $0x1, BX
  :0                    0x4e6373                f00fc118                LOCK XADDL BX, 0(AX)
  :0                    0x4e6377                ffc3                    INCL BX
  :0                    0x4e6379                0fbae300                BTL $0x0, BX
  :0                    0x4e637d                733a                    JAE 0x4e63b9
  :0                    0x4e637f                90                      NOPL
  :0                    0x4e6380                bb01000000              MOVL $0x1, BX
  :0                    0x4e6385                f00fc118                LOCK XADDL BX, 0(AX)
  :0                    0x4e6389                ffc3                    INCL BX
  :0                    0x4e638b                0fbae300                BTL $0x0, BX
  :0                    0x4e638f                73d0                    JAE 0x4e6361

PiperOrigin-RevId: 329754148
2020-09-02 11:37:31 -07:00
Michael Pratt ab6c474210 Bump build constraints to 1.17
This enables pre-release testing with 1.16. The intention is to replace these
with a nogo check before the next release.

PiperOrigin-RevId: 328193911
2020-08-24 12:58:39 -07:00
Ghanan Gowripalan 9c32fd3f4d Do not copy sleep.Waker
sleep.Waker's fields are modified as values.

PiperOrigin-RevId: 320873451
2020-07-12 17:22:08 -07:00
Michael Pratt 65569cfca0 Update Go version build tags
None of the dependencies have changed in 1.15. It may be possible to simplify
some of the wrappers in rawfile following 1.13, but that can come in a later
change.

PiperOrigin-RevId: 313863264
2020-05-29 15:44:07 -07:00
Rahat Mahmood 49d2cf287d Remove implicit dependencies for leaf packages.
These packages don't actually use go_stateify or go_marshal, but end
up implicitly dependent on the respective packages due to our build
rules.

These unnecessary dependencies make them unusuable in certain contexts
due to circular dependency.

PiperOrigin-RevId: 312595738
2020-05-20 18:35:32 -07:00
Jamie Liu 30388ff591 Rename files in //pkg/sync to better reflect what they contain.
PiperOrigin-RevId: 304447031
2020-04-02 11:57:06 -07:00
Bhasker Hariharan d04adebaab Fix data-race in endpoint.Readiness
PiperOrigin-RevId: 302924789
2020-03-25 10:55:22 -07:00
Adin Scannell d29e59af9f Standardize on tools directory.
PiperOrigin-RevId: 291745021
2020-01-27 12:21:00 -08:00
Ian Gudger 6a59e7f510 Rename DowngradableRWMutex to RWmutex.
Also renames TMutex to Mutex.

These custom mutexes aren't any worse than the standard library versions (same
code), so having both seems redundant.

PiperOrigin-RevId: 290873587
2020-01-21 19:36:12 -08:00
Ian Gudger d0e75f2bef Add trylock support to DowngradableRWMutex.
Updates #231

PiperOrigin-RevId: 290868875
2020-01-21 19:11:18 -08:00
Ian Gudger 1effdc091b TMutex based on sync.Mutex.
Updates #231

PiperOrigin-RevId: 290854399
2020-01-21 18:49:28 -08:00
Ian Gudger 27500d529f New sync package.
* Rename syncutil to sync.
* Add aliases to sync types.
* Replace existing usage of standard library sync package.

This will make it easier to swap out synchronization primitives. For example,
this will allow us to use primitives from github.com/sasha-s/go-deadlock to
check for lock ordering violations.

Updates #1472

PiperOrigin-RevId: 289033387
2020-01-09 22:02:24 -08:00
Michael Pratt fe1369ac98 Move package sync to third_party
PiperOrigin-RevId: 231889261
Change-Id: I482f1df055bcedf4edb9fe3fe9b8e9c80085f1a0
2019-01-31 17:49:14 -08:00
Michael Pratt 2a0c69b19f Remove license comments
Nothing reads them and they can simply get stale.

Generated with:
$ sed -i "s/licenses(\(.*\)).*/licenses(\1)/" **/BUILD

PiperOrigin-RevId: 231818945
Change-Id: Ibc3f9838546b7e94f13f217060d31f4ada9d4bf0
2019-01-31 11:12:53 -08:00
Michael Pratt 03c1eb78b5 Reference upstream licenses
Include copyright notices and the referenced LICENSE file.

PiperOrigin-RevId: 222171321
Change-Id: I0cc0b167ca51b536d1087bf1c4742fdf1430bc2a
2018-11-20 14:05:16 -08:00
Adin Scannell 75cd70ecc9 Track paths and provide a rename hook.
This change also adds extensive testing to the p9 package via mocks. The sanity
checks and type checks are moved from the gofer into the core package, where
they can be more easily validated.

PiperOrigin-RevId: 218296768
Change-Id: I4fc3c326e7bf1e0e140a454cbacbcc6fd617ab55
2018-10-23 00:20:15 -07:00
Ian Gudger 8fce67af24 Use correct company name in copyright header
PiperOrigin-RevId: 217951017
Change-Id: Ie08bf6987f98467d07457bcf35b5f1ff6e43c035
2018-10-19 16:35:11 -07:00
Jamie Liu 8e729e0e1f Add //pkg/sync:generic_atomicptr.
PiperOrigin-RevId: 215620949
Change-Id: I519da4b44386d950443e5784fb8c48ff9a36c5d3
2018-10-03 13:52:15 -07:00
Zhaozhong Ni be7fcbc558 stateify: support explicit annotation mode; convert refs and stack packages.
We have been unnecessarily creating too many savable types implicitly.

PiperOrigin-RevId: 206334201
Change-Id: Idc5a3a14bfb7ee125c4f2bb2b1c53164e46f29a8
2018-07-27 10:17:21 -07:00
Googler d02b74a5dc Check in gVisor.
PiperOrigin-RevId: 194583126
Change-Id: Ica1d8821a90f74e7e745962d71801c598c652463
2018-04-28 01:44:26 -04:00