ring0.Save/LoadFloatingPoint() are only usable if the caller can ensure that Go
will not clobber floating point registers before/after calling them
respectively. Due to regabig in Go 1.17, this is no longer the case; regabig
(among other things) maintains a zeroed XMM15 during ABIInternal execution,
including by zeroing it after ABI0-to-ABIInternal transitions. In
ring0.sysenter/exception, this happens in
ring0.kernelSyscall/kernelException.abi0 respectively; in
ring0.CPU.SwitchToUser, this happens after returning from
ring0.sysret/iret.abi0. Delete these functions and do floating point save/load
in assembly.
While arm64 doesn't appear to be immediately affected (so this CL permits us to
resume usage of Go 1.17), its use of Save/LoadFloatingPoint() still seems to be
incorrect for the same fundamental reason (Go code can't sanely assume what
registers the Go compiler will or won't use) and should be fixed eventually.
PiperOrigin-RevId: 401895658
We install seccomp rules so that the SIGSYS signal is generated for
each mmap system call. Then our signal handler executes the real mmap
syscall and if a new regions is created, it maps it to the guest.
Signed-off-by: Andrei Vagin <avagin@google.com>
CR0.NE enables internal x87 floating point error reporting when set, else
enables PC style x87 error detection.
On AMD, the #MF exception isn't generated if CR0.NE isn't set.
PiperOrigin-RevId: 386340269
Go 1.17 adds a new register-based calling convention. While transparent for
most applications, the KVM platform needs special work in a few cases.
First of all, we need the actual address of some assembly functions, rather
than the address of a wrapper. See http://gvisor.dev/pr/5832 for complete
discussion of this.
More relevant to this CL is that ABI0-to-ABIInternal wrappers (i.e., calls from
assembly to Go) access the G via FS_BASE. The KVM quite fast-and-loose about
the Go environment, often calling into (nosplit) Go functions with
uninitialized FS_BASE.
That will no longer work in Go 1.17, so this CL changes the platform to
consistently restore FS_BASE before calling into Go code.
This CL does not affect arm64 code. Go 1.17 does not support the register-based
calling convention for arm64 (it will come in 1.18), but arm64 also does not
use a non-standard register like FS_BASE for TLS, so it may not require any
changes.
PiperOrigin-RevId: 384234305
XCR0 has to be synchronized with the host. We can call xsave from the host
context and then call xrstor from the guest context and vise versa. This means
we need to support the same set of FPU features in both contexts.
PiperOrigin-RevId: 378988281
The root table physical page has to be mapped to not fault in iret or sysret
after switching into a user address space. sysret and iret are in the upper
half that is global and so page tables of lower levels are already mapped.
Fixes#5742
PiperOrigin-RevId: 371458644
Split usermem package to help remove syserror dependency in go_marshal.
New hostarch package contains code not dependent on syserror.
PiperOrigin-RevId: 365651233
It is enough to invalidate the tlb of local vcpu in switch().
TLBI with inner-sharable will invalidate the tlb in other vcpu.
Arm64 hardware supports at least 256 pcid, so I think it's ok
to set the length of pcid pool to 128.
Signed-off-by: Robin Luk <lubin.lu@antgroup.com>
Implement basic lazy save and restore for FPSIMD registers, which only
restore FPSIMD state on el0_fpsimd_acc and save FPSIMD state in switch().
Signed-off-by: Robin Luk <lubin.lu@antgroup.com>
In order to improve the performance and stability, I reorg 2 modules slightly.
arch: no red zone on Arm64.
ring0: use stp instead of movd, and set RSV_REG_APP=R19.
Signed-off-by: Robin Luk <lubin.lu@antgroup.com>
This allows the package to serve as a general purpose ring0 support package, as
opposed to being bound to specific sentry platforms.
Updates #5039
PiperOrigin-RevId: 355220044
Jamie Liu
Andrei Vagin
Andrei Vagin
Michael Pratt
gVisor bot
Howard Zhang
Zach Koopmans
Robin Luk
Adin Scannell