Filippo Valsorda [Fri, 1 Nov 2019 23:00:33 +0000 (19:00 -0400)]
crypto/tls: refactor certificate and signature algorithm logic
This refactors a lot of the certificate support logic to make it cleaner
and reusable where possible. These changes will make the following CLs
much simpler.
In particular, the heavily overloaded pickSignatureAlgorithm is gone.
That function used to cover both signing and verifying side, would work
both for pre-signature_algorithms TLS 1.0/1.1 and TLS 1.2, and returned
sigalg, type and hash.
Now, TLS 1.0/1.1 and 1.2 are differentiated at the caller, as they have
effectively completely different logic. TLS 1.0/1.1 simply use
legacyTypeAndHashFromPublicKey as they employ a fixed hash function and
signature algorithm for each public key type. TLS 1.2 is instead routed
through selectSignatureScheme (on the signing side) or
isSupportedSignatureAlgorithm (on the verifying side) and
typeAndHashFromSignatureScheme, like TLS 1.3.
On the signing side, signatureSchemesForCertificate was already version
aware (for PKCS#1 v1.5 vs PSS support), so selectSignatureScheme just
had to learn the Section 7.4.1.4.1 defaults for a missing
signature_algorithms to replace pickSignatureAlgorithm.
On the verifying side, pickSignatureAlgorithm was also checking the
public key type, while isSupportedSignatureAlgorithm +
typeAndHashFromSignatureScheme are not, but that check was redundant
with the one in verifyHandshakeSignature.
There should be no major change in behavior so far. A few minor changes
came from the refactor: we now correctly require signature_algorithms in
TLS 1.3 when using a certificate; we won't use Ed25519 in TLS 1.2 if the
client didn't send signature_algorithms; and we don't send
ec_points_format in the ServerHello (a compatibility measure) if we are
not doing ECDHE anyway because there are no mutually supported curves.
The tests also got simpler because they test simpler functions. The
caller logic switching between TLS 1.0/1.1 and 1.2 is tested by the
transcript tests.
Dmitri Shuralyov [Mon, 4 Nov 2019 21:50:03 +0000 (16:50 -0500)]
go/doc: add NewFromFiles with support for classifying examples
This CL is based on work started by Joe Tsai in CL 94855.
It's rebased on top of the latest master branch, and
addresses various code review comments and findings
from attempting to use the original CL in practice.
The testing package documents a naming convention for examples
so that documentation tools can associate them with:
• a package (Example or Example_suffix)
• a function F (ExampleF or ExampleF_suffix)
• a type T (ExampleT or ExampleT_suffix)
• a method T.M (ExampleT_M or ExampleT_M_suffix)
This naming convention is in widespread use and enforced
via existing go vet checks.
This change adds first-class support for classifying examples
to go/doc, the package responsible for computing package
documentation from Go AST.
There isn't a way to supply test files to New that works well.
External test files may have a package name with "_test" suffix,
so ast.NewPackage may end up using the wrong package name if given
test files. A workaround is to add test files to *ast.Package.Files
after it is returned from ast.NewPackage:
pkg, _ := ast.NewPackage(fset, goFiles, ...)
for name, f := range testGoFiles {
pkg.Files[name] = f
}
p := doc.New(pkg, ...)
But that is not a good API.
After nearly 8 years, a new entry-point is added to the go/doc
package, the function NewFromFiles. It accepts a Go package in
the form of a list of parsed Go files (including _test.go files)
and an import path. The caller is responsible with filtering out
files based on build constraints, as was the case before with New.
NewFromFiles computes package documentation from .go files,
extracts examples from _test.go files and classifies them.
Examples fields are added to Package, Type, and Func. They are
documented to only be populated with examples found in _test.go
files provided to NewFromFiles.
The new behavior is:
1. NewFromFiles computes package documentation from provided
parsed .go files. It extracts examples from _test.go files.
2. It assigns each Example to corresponding Package, Type,
or Func.
3. It sets the Suffix field in each example to the suffix.
4. Malformed examples are skipped.
This change implements behavior that matches the current behavior
of existing godoc-like tools, and will enable them to rely on the
logic in go/doc instead of reimplementing it themselves.
Fixes #23864
Change-Id: Iae834f2ff92fbd1c93a9bb7c2bf47d619bee05cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/204830
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Than McIntosh [Mon, 11 Nov 2019 21:30:35 +0000 (16:30 -0500)]
cmd/link: disable a DWARF testpoint on Windows pending investigation
Disable a portion of the TestDWARF testpoint for Windows using
c-archive buildmode, pending investigation of the issue at hand, so as
to get the longtest builder unblocked.
Updates #35512.
Change-Id: Ib72d82ceaa674b9a51da220fb8e225231d5c3433
Reviewed-on: https://go-review.googlesource.com/c/go/+/206557
Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Jeremy Faller <jeremy@golang.org>
Jay Conrod [Mon, 11 Nov 2019 19:24:00 +0000 (14:24 -0500)]
cmd/go/internal/modcmd: skip modules with empty version strings
This CL restores behavior before CL 189797 and fixes a misleading
comment. modload.ListModules may return info without a version for the
main module and for modules replaced with local directories.
Fixes #35505
Change-Id: I5b4e68053a680ff897b072fdf6e7aa17b6e1ac34
Reviewed-on: https://go-review.googlesource.com/c/go/+/206538
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Bryan C. Mills [Mon, 11 Nov 2019 17:41:01 +0000 (12:41 -0500)]
cmd/go: in 'go build -o', allow the destination file to exist if it is empty
This allows the target of 'go build' to be a filename constructed
using ioutil.TempFile or similar, without racily deleting the file
before rebuilding it.
Updates #32407
Updates #28387
Change-Id: I4c5072830a02b93f0c4186b50bffa9de00257afe
Reviewed-on: https://go-review.googlesource.com/c/go/+/206477
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
This makes Ed25519 certificates work for CreateCRL(). This previously
failed (panic: crypto: requested hash function #0 is unavailable) because
the hash could not be skipped, but Ed25519 uses no hash.
A similar fix has been applied in a few other places when Ed25519 was added
when Ed25519 certificates were originally introduced, but was missed
here.
Than McIntosh [Fri, 8 Nov 2019 21:11:44 +0000 (16:11 -0500)]
cmd/go: remove -w workaround for -buildmode=plugin on Darwin
The problem causing the assert in #21647 are fixed at this point,
along with various other linker issues with plugin + Darwin. With
this in mind, remove the "-ldflags=-w" workaround for plugin mode on
Darwin and re-enable the appropriate tests misc/cgo/testplugin
Daniel Martí [Mon, 29 Jul 2019 03:16:14 +0000 (20:16 -0700)]
encoding/json: error when encoding a pointer cycle
Otherwise we'd panic with a stack overflow.
Most programs are in control of the data being encoded and can ensure
there are no cycles, but sometimes it's not that simple. For example,
running a user's html template with script tags can easily result in
crashes if the user can find a pointer cycle.
Adding the checks via a map to every ptrEncoder.encode call slowed down
the benchmarks below by a noticeable 13%. Instead, only start doing the
relatively expensive pointer cycle checks if we're many levels of
pointers deep in an encode state.
A threshold of 1000 is small enough to capture pointer cycles before
they're a problem (the goroutine stack limit is currently 1GB, and I
needed close to a million levels to reach it). Yet it's large enough
that reasonable uses of the json encoder only see a tiny 1% slow-down
due to the added ptrLevel field and check.
name old time/op new time/op delta
CodeEncoder-8 2.34ms ± 1% 2.37ms ± 0% +1.05% (p=0.000 n=10+10)
CodeMarshal-8 2.42ms ± 1% 2.44ms ± 0% +1.10% (p=0.000 n=10+10)
name old speed new speed delta
CodeEncoder-8 829MB/s ± 1% 820MB/s ± 0% -1.04% (p=0.000 n=10+10)
CodeMarshal-8 803MB/s ± 1% 795MB/s ± 0% -1.09% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
CodeEncoder-8 43.1kB ± 8% 42.5kB ±10% ~ (p=0.989 n=10+10)
CodeMarshal-8 1.99MB ± 0% 1.99MB ± 0% ~ (p=0.254 n=9+6)
name old allocs/op new allocs/op delta
CodeEncoder-8 0.00 0.00 ~ (all equal)
CodeMarshal-8 1.00 ± 0% 1.00 ± 0% ~ (all equal)
Finally, add a few tests to ensure that the code handles the edge cases
properly.
Fixes #10769.
Change-Id: I73d48e0cf6ea140127ea031f2dbae6e6a55e58b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/187920
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bjørn Erik Pedersen <bjorn.erik.pedersen@gmail.com> Reviewed-by: Andrew Bonventre <andybons@golang.org>
Michael Anthony Knyszek [Sat, 9 Nov 2019 00:21:02 +0000 (00:21 +0000)]
runtime: fix min/max logic in findScavengeCandidate
Before this CL, if max > min and max was unaligned to min, then the
function could return an unaligned (unaligned to min) region to
scavenge. On most platforms, this leads to some kind of crash.
Fix this by explicitly aligning max to the next multiple of min.
Fixes #35445.
Updates #35112.
Change-Id: I0af42d4a307b48a97e47ed152c619d77b0298291
Reviewed-on: https://go-review.googlesource.com/c/go/+/206277 Reviewed-by: Ian Lance Taylor <iant@golang.org>
By using a 'rotate then xor selected bits' instruction combined with
either a 'load and and' or a 'load and or' instruction we can
implement And8 and Or8 with far fewer instructions. Replacing
'compare and swap' with atomic instructions may also improve
performance when there is contention.
Cherry Zhang [Sun, 10 Nov 2019 18:18:06 +0000 (13:18 -0500)]
runtime: don't save G during VDSO if we're handling signal
On some platforms (currently ARM and ARM64), when calling into
VDSO we store the G to the gsignal stack, if there is one, so if
we receive a signal during VDSO we can find the G.
If we receive a signal during VDSO, and within the signal handler
we call nanotime again (e.g. when handling profiling signal),
we'll save/clear the G slot on the gsignal stack again, which
clobbers the original saved G. If we receive a second signal
during the same VDSO execution, we will fetch a nil G, which will
lead to bad things such as deadlock.
Don't save G if we're calling VDSO code from the gsignal stack.
Saving G is not necessary as we won't receive a nested signal.
Reason for revert: It was retracted by the author in a comment on the PR
but that doesn't get synced to Gerrit, and the Gerrit CL wasn't closed
when the PR was closed.
David Chase [Thu, 24 Oct 2019 17:48:17 +0000 (13:48 -0400)]
cmd/compile: add framework for logging optimizer (non)actions to LSP
This is intended to allow IDEs to note where the optimizer
was not able to improve users' code. There may be other
applications for this, for example in studying effectiveness
of optimizer changes more quickly than running benchmarks,
or in verifying that code changes did not accidentally disable
optimizations in performance-critical code.
Logging of nilcheck (bad) for amd64 is implemented as
proof-of-concept. In general, the intent is that optimizations
that didn't happen are what will be logged, because that is
believed to be what IDE users want.
Added flag -json=version,dest
Check that version=0. (Future compilers will support a
few recent versions, I hope that version is always <=3.)
Dest is expected to be one of:
/path (or \path in Windows)
will create directory /path and fill it w/ json files
file://path
will create directory path, intended either for
I:\dont\know\enough\about\windows\paths
trustme_I_know_what_I_am_doing_probably_testing
Not passing an absolute path name usually leads to
json splattered all over source directories,
or failure when those directories are not writeable.
If you want a foot-gun, you have to ask for it.
The JSON output is directed to subdirectories of dest,
where each subdirectory is net/url.PathEscape of the
package name, and each for each foo.go in the package,
net/url.PathEscape(foo).json is created. The first line
of foo.json contains version and context information,
and subsequent lines contains LSP-conforming JSON
describing the missing optimizations.
Agniva De Sarker [Sun, 10 Nov 2019 08:44:47 +0000 (14:14 +0530)]
time: change variable name to next
The variable now implies that the next tick always
returns the current time which is not always the case.
Change it to next to clarify that it returns
the time of the next tick which is more appropriate.
Michael Anthony Knyszek [Thu, 7 Nov 2019 22:42:38 +0000 (22:42 +0000)]
runtime: make the test addresses for pageAlloc smaller on 32-bit
This change makes the test addresses start at 1 GiB instead of 2 GiB to
support mips and mipsle, which only have 31-bit address spaces.
It also changes some tests to use smaller offsets for the chunk index to
avoid jumping too far ahead in the address space to support 31-bit
address spaces. The tests don't require such large jumps for what
they're testing anyway.
Fazlul Shahriar [Sat, 9 Nov 2019 21:02:46 +0000 (16:02 -0500)]
cmd/go/internal/lockedfile, os: fix O_CREATE flag on Plan 9
os.OpenFile was assuming that a failed syscall.Open means the file does
not exist and it tries to create it. However, syscall.Open may have
failed for some other reason, such as failing to lock a os.ModeExclusive
file. We change os.OpenFile to only create the file if the error
indicates that the file doesn't exist.
Remove skip of TestTransform test, which was failing because sometimes
syscall.Open would fail due to the file being locked, but the
syscall.Create would succeed because the file is no longer locked. The
create was truncating the file.
Setting InsecureSkipVerify and VerifyPeerCertificate is the recommended
way to customize and override certificate validation.
However, there is boilerplate involved and it usually requires first
reimplementing the default validation strategy to then customize it.
Provide an example that does the same thing as the default as a starting
point.
Examples of where we directed users to do something similar are in
issues #35467, #31791, #28754, #21971, and #24151.
Fixes #31792
Change-Id: Id033e9fa3cac9dff1f7be05c72dfb34b4f973fd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/193620 Reviewed-by: Adam Langley <agl@golang.org>
Rhys Hiltner [Fri, 8 Nov 2019 18:30:24 +0000 (10:30 -0800)]
sync: yield to the waiter when unlocking a starving mutex
When we have already assigned the semaphore ticket to a specific
waiter, we want to get the waiter running as fast as possible since
no other G waiting on the semaphore can acquire it optimistically.
The net effect is that, when a sync.Mutex is contended, the code in
the critical section guarded by the Mutex gets a priority boost.
Fixes #33747
The original work was done in CL 200577 by Carlo Alberto Ferraris. The
change was reverted in CL 205817 because it broke the linux-arm64-packet
and solaris-amd64-oraclerel builders.
Tobias Klauser [Fri, 8 Nov 2019 08:25:51 +0000 (09:25 +0100)]
syscall: skip TestSyscallNoError on mips{,le}
On MIPS, Linux returns whether the syscall had an error in a separate
register (R7), not using a negative return value as on other
architectures. Thus, skip TestSyscallNoError as there is no error case
for syscall.RawSyscall which it could test against.
Also reformat the error output so the expected and gotten values are
aligned so they're easier to compare.
Fixes #35422
Change-Id: Ibc88f7c5382bb7ee8faf15ad4589ca1f9f017a06
Reviewed-on: https://go-review.googlesource.com/c/go/+/205898
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Ian Lance Taylor [Fri, 8 Nov 2019 23:28:51 +0000 (15:28 -0800)]
os/signal: use a larger channel buffer for all signals in TestSignal
Now that the runtime can send preemption signals, it is possible that
a channel that asks for all signals can see both SIGURG and SIGHUP
before reading either, in which case one of the signals will be dropped.
We have to use a larger buffer so that the test see the signal it expects.
Fixes #35466
Change-Id: I36271eae0661c421780c72292a5bcbd443ada987
Reviewed-on: https://go-review.googlesource.com/c/go/+/206257
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
David Chase [Fri, 8 Nov 2019 21:11:29 +0000 (16:11 -0500)]
runtime: copy some functions from math/bits to runtime/internal/sys
CL 201765 activated calls from the runtime to functions in math/bits.
When coverage and race detection were simultaneously enabled,
this caused a crash when the covered+race-checked code in
math/bits was called from the runtime before there was even a P.
PS Win for gdlv in helping sort this out.
TODO - next CL intrinsifies the new functions in
runtime/internal/sys
TODO/Would-be-nice - Ctz64 and TrailingZeros64 are the same
function; 386.s is intrinsified; clean all that up.
Fixes #35461.
Updates #35112.
Change-Id: I750a54dba493130ad3e68a06530ede7687d41e1d
Reviewed-on: https://go-review.googlesource.com/c/go/+/206199 Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Matthew Dempsky [Thu, 7 Nov 2019 23:54:59 +0000 (15:54 -0800)]
cmd/compile: don't apply -lang=go1.X restrictions to imported packages
Previously langSupported applied -lang as though it's a global
restriction, but it's actually a per-package restriction. This CL
fixes langSupported to take a *types.Pkg parameter to reflect this and
updates its callers accordingly.
This is relevant for signed shifts (added in Go 1.12), because they
can be inlined into a Go 1.11 package; and for overlapping interfaces
(added in Go 1.13), because they can be exported as part of the
package's API.
Today we require all Go packages to be compiled with the same
toolchain, and all uses of langSupported are for controlling
backwards-compatible features. So we can simply assume that since the
imported packages type-checked successfully, they must have been
compiled with an appropriate -lang setting.
In the future if we ever want to use langSupported to control
backwards-incompatible language changes, we might need to record the
-lang flag used for compiling a package in its export data.
Fixes #35437.
Fixes #35442.
Change-Id: Ifdf6a62ee80cd5fb4366cbf12933152506d1b36e
Reviewed-on: https://go-review.googlesource.com/c/go/+/205977 Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
Keith Randall [Mon, 23 Sep 2019 21:36:48 +0000 (14:36 -0700)]
runtime: fix line number for faulting instructions
Unlike function calls, when processing instructions that directly
fault we must not subtract 1 from the pc before looking up the
file/line information.
Since the file/line lookup unconditionally subtracts 1, add 1 to
the faulting instruction PCs to compensate.
Ariel Mashraki [Thu, 24 Oct 2019 11:36:57 +0000 (14:36 +0300)]
cmd/go/internal/modload: fail if no package was found in local module
Changing the Import function to return a PackageNotInModuleError if no
package was found in a local module. This replacing the vague message
"missing dot in first path element" you get today with much more friendly
one - "module was found, but does not contain package".
Fixes #35273
Change-Id: I6d726c17e6412258274b10f58f76621617d26e0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/203118 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Ian Lance Taylor [Fri, 8 Nov 2019 05:04:53 +0000 (21:04 -0800)]
runtime: add pipe/pipe2 on Solaris
This adds pipe/pipe2 on Solaris as they exist on other Unix systems.
They were not added previously because Solaris does not need them
for netpollBreak. They are added now in preparation for using pipes
in TestSignalM.
Updates #35276
Change-Id: I53dfdf077430153155f0a79715af98b0972a841c
Reviewed-on: https://go-review.googlesource.com/c/go/+/206077
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Emmanuel T Odeke [Sun, 10 Mar 2019 06:33:43 +0000 (22:33 -0800)]
net/http: support gzip, x-gzip Transfer-Encodings
Support "gzip" aka "x-gzip" as a transfer-encoding for
requests and responses as per RFC 7230 Section 3.3.1.
"gzip" and "x-gzip" are equivalents as requested by
RFC 7230 Section 4.2.3.
Transfer-Encoding is an on-fly property of the body
that can be applied by proxies, other servers and basically
any intermediary to transport the content e.g. across data centers
or backends/machine to machine that need compression.
For this change, "gzip" is both explicitly and implicitly combined
with transfer-encoding "chunked" in an ordering such as:
Transfer-Encoding: gzip, chunked
and NOT
Transfer-Encoding: chunked, gzip
Obviously the latter form is counter-intuitive for streaming.
Thus "chunked" is the last value to appear in that transfer-encoding header,
if explicitly included.
When parsing the response, the chunked body is concatenated as "chunked" does,
before finally being decompressed as "gzip".
A chunked and compressed body would typically look like this:
and then finally gunzip it
<FINAL_BODY> := gunzip(<FULL_BODY>)
If a "chunked" transfer-encoding is NOT applied but "gzip" is applied,
we implicitly assume that they requested using "chunked" at the end.
This is as per the recommendation of RFC 3.3.1. which explicitly says
that for:
* Request:
" If any transfer coding
other than chunked is applied to a request payload body, the sender
MUST apply chunked as the final transfer coding to ensure that the
message is properly framed."
* Response:
" If any transfer coding other than
chunked is applied to a response payload body, the sender MUST either
apply chunked as the final transfer coding or terminate the message
by closing the connection."
Martin Garton [Mon, 30 Sep 2019 09:27:38 +0000 (09:27 +0000)]
encoding/binary: add float support to fast path
This adds float type support to the main switch blocks in Read and
Write, instead of falling back to reflection. This gives a considerable
speedup for the float types:
Chris Stockton [Sun, 27 Oct 2019 15:55:53 +0000 (15:55 +0000)]
net: halve the allocs in ParseCIDR by sharing slice backing
Share a slice backing between the host address, network ip and mask.
Add tests to verify that each slice header has len==cap to prevent
introducing new behavior into Go programs. This has a small tradeoff
of allocating a larger slice backing when the address is invalid.
Earlier error detection of invalid prefix length helps balance this
cost and a new benchmark for ParseCIDR helps measure it.
This yields a ~22% speedup for all nil err cidr tests:
Ian Lance Taylor [Tue, 5 Nov 2019 04:06:19 +0000 (20:06 -0800)]
runtime: sleep a bit when waiting for running debug call goroutine
Without this CL, one of the TestDebugCall tests would fail 1% to 2% of
the time on the android-amd64-emu gomote. With this CL, I ran the
tests for 1000 iterations with no failures.
Fixes #32985
Change-Id: I541268a2a0c10d0cd7604f0b2dbd15c1d18e5730
Reviewed-on: https://go-review.googlesource.com/c/go/+/205248
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Michael Anthony Knyszek [Mon, 16 Sep 2019 21:23:24 +0000 (21:23 +0000)]
runtime: add per-p page allocation cache
This change adds a per-p free page cache which the page allocator may
allocate out of without a lock. The change also introduces a completely
lockless page allocator fast path.
Although the cache contains at most 64 pages (and usually less), the
vast majority (85%+) of page allocations are exactly 1 page in size.
Michael Anthony Knyszek [Wed, 18 Sep 2019 17:51:16 +0000 (17:51 +0000)]
runtime: add page cache and tests
This change adds a page cache structure which owns a chunk of free pages
at a given base address. It also adds code to allocate to this cache
from the page allocator. Finally, it adds tests for both.
Notably this change does not yet integrate the code into the runtime,
just into runtime tests.
Michael Anthony Knyszek [Wed, 18 Sep 2019 15:57:36 +0000 (15:57 +0000)]
runtime: add per-p mspan cache
This change adds a per-p mspan object cache similar to the sudog cache.
Unfortunately this cache can't quite operate like the sudog cache, since
it is used in contexts where write barriers are disallowed (i.e.
allocation codepaths), so rather than managing an array and a slice,
it's just an array and a length. A little bit more unsafe, but avoids
any write barriers.
The purpose of this change is to reduce the number of operations which
require the heap lock in allocation, paving the way for a lockless fast
path.
Michael Anthony Knyszek [Wed, 18 Sep 2019 15:44:11 +0000 (15:44 +0000)]
runtime: rearrange mheap_.alloc* into allocSpan
This change combines the functionality of allocSpanLocked, allocManual,
and alloc_m into a new method called allocSpan. While these methods'
abstraction boundaries are OK when the heap lock is held throughout,
they start to break down when we want finer-grained locking in the page
allocator.
allocSpan does just that, and only locks the heap when it absolutely has
to. Piggy-backing off of work in previous CLs to make more of span
initialization lockless, this change makes span initialization entirely
lockless as part of the reorganization.
Ultimately this change will enable us to add a lockless fast path to
allocSpan.
Michael Anthony Knyszek [Thu, 24 Oct 2019 22:15:14 +0000 (22:15 +0000)]
runtime: fix (*gcSweepBuf).block guarantees
Currently gcSweepBuf guarantees that push operations may be performed
concurrently with each other and that block operations may be performed
concurrently with push operations as well.
Unfortunately, this isn't quite true. The existing code allows push
operations to happen concurrently with each other, but block operations
may return blocks with nil entries. The way this can happen is if two
concurrent pushers grab a slot to push to, and the first one (the one
with the earlier slot in the buffer) doesn't quite write a span value
when the block is called. The existing code in block only checks if the
very last value in the block is nil, when really an arbitrary number of
the last few values in the block may or may not be nil.
Today, this case can't actually happen because when push operations
happen concurrently during a GC (which is the only time block is
called), they only ever happen during an allocation with the heap lock
held, effectively serializing them. A block operation may happen
concurrently with one of these pushes, but its callers will never see a
nil mspan. Outside of a GC, this isn't a problem because although push
operations from allocations can run concurrently with push operations
from sweeping, block operations will never run.
In essence, the real concurrency guarantees provided by gcSweepBuf are
that block operations may happen concurrently with push operations, but
that push operations may not be concurrent with each other if there are
any block operations.
To fix this, and to prepare for push operations happening without the
heap lock held in a future CL, we update the documentation for block to
correctly state that there may be nil entries in the returned slice.
While we're here, make the mspan writes into the buffer atomic to avoid
a block user racing on a nil check, and document that the user should
load mspan values from the returned slice atomically. Finally, we make
all callers of block adhere to the new rules.
We choose to allow nil values rather than filter them out because the
only caller of block is markrootSpans, and if it catches a nil entry,
then there wasn't anything to mark in there anyway since the span is
just being created.
Michael Anthony Knyszek [Wed, 18 Sep 2019 15:33:17 +0000 (15:33 +0000)]
runtime: make more page sweeper operations atomic
This change makes it so that allocation and free related page sweeper
metadata operations (e.g. pageInUse and pagesInUse) are atomic rather
than protected by the heap lock. This will help in reducing the length
of the critical path with the heap lock held in future changes.
Cherry Zhang [Fri, 8 Nov 2019 03:40:50 +0000 (22:40 -0500)]
cmd/internal/obj/arm64: make function epilogue async-signal safe
When the frame size is large, we generate
MOVD.P 0xf0(SP), LR
ADD $(framesize-0xf0), SP
This is problematic: after the first instruction, we have a
partial frame of size (framesize-0xf0). If we try to unwind the
stack at this point, we'll try to read the LR from the stack at
0(SP) (the new SP) as the frame size is not 0. But this slot does
not contain a valid LR.
Fix this by not changing SP in two instructions. Instead,
generate
MOVD (SP), LR
ADD $framesize, SP
This affects not only async preemption but also profiling. So we
change the generated instructions, instead of marking unsafe
point.
Change-Id: I4e78c62d50ffc4acff70ccfbfec16a5ccae17f24
Reviewed-on: https://go-review.googlesource.com/c/go/+/206057
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Mon, 28 Oct 2019 04:53:14 +0000 (00:53 -0400)]
runtime: add async preemption support on PPC64
This CL adds support of call injection and async preemption on
PPC64.
For the injected call to return to the preempted PC, we have to
clobber either LR or CTR. For reasons mentioned in previous CLs,
we choose CTR. Previous CLs have marked code sequences that use
CTR async-nonpreemtible.
Michael Anthony Knyszek [Wed, 18 Sep 2019 15:15:59 +0000 (15:15 +0000)]
runtime: remove unnecessary large parameter to mheap_.alloc
mheap_.alloc currently accepts both a spanClass and a "large" parameter
indicating whether the allocation is large. These are redundant, since
spanClass.sizeclass() == 0 is an equivalent way to determine this and is
already used in mheap_.alloc. There are no places in the runtime where
the size class could be non-zero and large == true.
Michael Anthony Knyszek [Thu, 7 Nov 2019 21:14:37 +0000 (21:14 +0000)]
runtime: define maximum supported physical page and huge page sizes
This change defines a maximum supported physical and huge page size in
the runtime based on the new page allocator's implementation, and uses
them where appropriate.
Furthemore, if the system exceeds the maximum supported huge page
size, we simply ignore it silently.
It also fixes a huge-page-related test which is only triggered by a
condition which is definitely wrong.
Finally, it adds a few TODOs related to code clean-up and supporting
larger huge page sizes.
Updates #35112.
Fixes #35431.
Change-Id: Ie4348afb6bf047cce2c1433576d1514720d8230f
Reviewed-on: https://go-review.googlesource.com/c/go/+/205937
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Bryan C. Mills [Fri, 8 Nov 2019 15:52:10 +0000 (10:52 -0500)]
cmd/go: delete flaky TestQEMUUserMode
If QEMU user-mode is actually a supported configuration, then per
http://golang.org/wiki/PortingPolicy it needs to have a builder
running tests for all packages, not just a simple “hello world”
program.
Michael Anthony Knyszek [Wed, 18 Sep 2019 15:03:50 +0000 (15:03 +0000)]
runtime: ensure heap memstats are updated atomically
For the most part, heap memstats are already updated atomically when
passed down to OS-level memory functions (e.g. sysMap). Elsewhere,
however, they're updated with the heap lock.
In order to facilitate holding the heap lock for less time during
allocation paths, this change more consistently makes the update of
these statistics atomic by calling mSysStat{Inc,Dec} appropriately
instead of simply adding or subtracting. It also ensures these values
are loaded atomically.
Furthermore, an undocumented but safe update condition for these
memstats is during STW, at which point using atomics is unnecessary.
This change also documents this condition in mstats.go.
Michael Anthony Knyszek [Wed, 18 Sep 2019 14:11:28 +0000 (14:11 +0000)]
runtime: remove useless heap_objects accounting
This change removes useless additional heap_objects accounting for large
objects. heap_objects is computed from scratch at ReadMemStats time
(which stops the world) by using nlargealloc and nlargefree, so mutating
heap_objects turns out to be pointless.
As a result, the "large" parameter on "mheap_.freeSpan" is no longer
necessary and so this change cleans that up too.
Michael Anthony Knyszek [Mon, 28 Oct 2019 19:17:21 +0000 (19:17 +0000)]
runtime: make allocNeedsZero lock-free
In preparation for a lockless fast path in the page allocator, this
change makes it so that checking if an allocation needs to be zeroed may
be done atomically.
Unfortunately, this means there is a CAS-loop to ensure monotonicity of
the zeroedBase value in heapArena. This CAS-loop exits if an allocator
acquiring memory further on in the arena wins or if it succeeds. The
CAS-loop should have a relatively small amount of contention because of
this monotonicity, though it would be ideal if we could just have
CAS-ers with the greatest value always win. The CAS-loop is unnecessary
in the steady-state, but should bring some start-up performance gains as
it's likely cheaper than the additional zeroing required, especially for
large allocations.
For very large allocations that span arenas, the CAS-loop should be
completely uncontended for most of the arenas it touches, it may only
encounter contention on the first and last arena.
Ian Lance Taylor [Fri, 8 Nov 2019 02:40:10 +0000 (02:40 +0000)]
Revert "math/cmplx: handle special cases"
This reverts CL 169501.
Reason for revert: The new tests fail at least on s390x and MIPS. This is likely a minor bug in the compiler or runtime. But this point in the release cycle is not the time to debug these details, which are unlikely to be new. Let's try again for 1.15.
Updates #29320
Fixes #35443
Change-Id: I2218b2083f8974b57d528e3742524393fc72b355
Reviewed-on: https://go-review.googlesource.com/c/go/+/206037
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Hana (Hyang-Ah) Kim [Sat, 2 Nov 2019 08:16:36 +0000 (17:16 +0900)]
runtime/pprof: correctly encode inlined functions in CPU profile
The pprof profile proto message expects inlined functions of a PC
to be encoded in one Location entry using multiple Line entries.
https://github.com/google/pprof/blob/5e96527/proto/profile.proto#L177-L184
runtime/pprof has encoded the symbolization information by creating
a Location for each PC found in the stack trace and including info
from all the frames expanded from the PC using runtime.CallersFrames.
This assumes inlined functions are represented as a single PC in the
stack trace. (https://go-review.googlesource.com/41256)
In the recent years, behavior around inlining and the traceback
changed significantly (e.g. https://golang.org/cl/152537,
https://golang.org/issue/29582, and many changes). Now the PCs
in the stack trace represent user frames even including inline
marks. As a result, the profile proto started to allocate a Location
entry for each user frame, lose the inline information (so pprof
presented incorrect results when inlined functions are involved),
and confuse the pprof tool with those PCs made up for inline marks.
This CL attempts to detect inlined call frames from the stack traces
of CPU profiles, and organize the Location information as intended.
Currently, runtime does not provide a reliable and convenient way to
detect inlined call frames and expand user frames from a given externally
recognizable PCs. So we use heuristics to recover the groups
- inlined call frames have nil Func field
- inlined call frames will have the same Entry point
- but must be careful with recursive functions that have the
same Entry point by definition, and non-Go functions that
may lack most of the fields of Frame.
The followup CL will address the issue with other profile types.
Change-Id: I0c9667ab016a3e898d648f31c3f82d84c15398db
Reviewed-on: https://go-review.googlesource.com/c/go/+/204636 Reviewed-by: Keith Randall <khr@golang.org>
Keith Randall [Mon, 21 Oct 2019 22:03:48 +0000 (15:03 -0700)]
doc: document new math.Fma function
This accidentally got committed - please review the whole paragraph
as if it was new.
Change-Id: I98e1db4670634c6e792d26201ce0cd329a6928b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/202579 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Matthew Dempsky [Thu, 7 Nov 2019 20:32:30 +0000 (12:32 -0800)]
cmd/compile: restore more missing -m=2 escape analysis details
This CL also restores analysis details for (1) expressions that are
directly heap allocated because of being too large for the stack or
non-constant in size, and (2) for assignments that we short circuit
because we flow their address to another escaping object.
No change to normal compilation behavior. Only adds additional Printfs
guarded by -m=2.
Updates #31489.
Change-Id: I43682195d389398d75ced2054e29d9907bb966e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/205917
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Cherry Zhang [Sun, 27 Oct 2019 02:54:28 +0000 (22:54 -0400)]
runtime: add async preemption support on MIPS and MIPS64
This CL adds support of call injection and async preemption on
MIPS and MIPS64.
Like ARM64, we need to clobber one register (REGTMP) for
returning from the injected call. Previous CLs have marked code
sequences that use REGTMP async-nonpreemtible.
It seems on MIPS/MIPS64, a CALL instruction is not "atomic" (!).
If a signal is delivered right at the CALL instruction, we may
see an updated LR with a not-yet-updated PC. In some cases this
may lead to failed stack unwinding. Don't preempt in this case.
Change-Id: I99437b2d05869ded5c0c8cb55265dbfc933aedab
Reviewed-on: https://go-review.googlesource.com/c/go/+/203720 Reviewed-by: Keith Randall <khr@golang.org>
Michael Anthony Knyszek [Mon, 28 Oct 2019 18:38:17 +0000 (18:38 +0000)]
runtime: compute whether a span needs zeroing in the new page allocator
This change adds the allocNeedZero method to mheap which uses the new
heapArena field zeroedBase to determine whether a new allocation needs
zeroing. The purpose of this work is to avoid zeroing memory that is
fresh from the OS in the context of the new allocator, where we no
longer have the concept of a free span to track this information.
The new field in heapArena, zeroedBase, is small, which runs counter to
the advice in the doc comment for heapArena. Since heapArenas are
already not a multiple of the system page size, this advice seems stale,
and we're OK with using an extra physical page for a heapArena. So, this
change also deletes the comment with that advice.
Cherry Zhang [Wed, 30 Oct 2019 00:42:00 +0000 (20:42 -0400)]
runtime: add async preemption support on S390X
This CL adds support of call injection and async preemption on
S390X.
Like ARM64, we need to clobber one register (REGTMP) for
returning from the injected call. Previous CLs have marked code
sequences that use REGTMP async-nonpreemtible.
Change-Id: I78adbc5fd70ca245da390f6266623385b45c9dfc
Reviewed-on: https://go-review.googlesource.com/c/go/+/204106 Reviewed-by: Keith Randall <khr@golang.org>
Cherry Zhang [Wed, 30 Oct 2019 00:40:26 +0000 (20:40 -0400)]
cmd/internal/obj/s390x: mark unsafe points
For async preemption, we will be using REGTMP as a temporary
register in injected call on S390X, which will clobber it. So any
code that uses REGTMP is not safe for async preemption.
In the assembler backend, we expand a Prog to multiple machine
instructions and use REGTMP as a temporary register if necessary.
These need to be marked unsafe. Unlike ARM64 and MIPS,
instructions on S390X are variable length so we don't use the
length as a condition. Instead, we set a bit on the Prog whenever
REGTMP is used.
Michael Anthony Knyszek [Thu, 17 Oct 2019 15:35:54 +0000 (15:35 +0000)]
runtime: make the scavenger self-paced
Currently the runtime background scavenger is paced externally,
controlled by a collection of variables which together describe a line
that we'd like to stay under.
However, the line to stay under is computed as a function of the number
of free and unscavenged huge pages in the heap at the end of the last
GC. Aside from this number being inaccurate (which is still acceptable),
the scavenging system also makes an order-of-magnitude assumption as to
how expensive scavenging a single page actually is.
This change simplifies the scavenger in preparation for making it
operate on bitmaps. It makes it so that the scavenger paces itself, by
measuring the amount of time it takes to scavenge a single page. The
scavenging methods on mheap already avoid breaking huge pages, so if we
scavenge a real huge page, then we'll have paced correctly, otherwise
we'll sleep for longer to avoid using more than scavengePercent wall
clock time.
Unfortunately, all this involves measuring time, which is quite tricky.
Currently we don't directly account for long process sleeps or OS-level
context switches (which is quite difficult to do in general), but we do
account for Go scheduler overhead and variations in it by maintaining an
EWMA of the ratio of time spent scavenging to the time spent sleeping.
This ratio, as well as the sleep time, are bounded in order to deal with
the aforementioned OS-related anomalies.
Cherry Zhang [Mon, 4 Nov 2019 23:11:57 +0000 (18:11 -0500)]
cmd/internal/obj/ppc64: handle MOVDU for SP delta
If a MOVDU instruction is used with an offset of SP, the
instruction changes SP therefore needs an SP delta, which is used
for generating the PC-SP table for stack unwinding. MOVDU is
frequently used for allocating the frame and saving the LR in the
same instruction, so this is particularly useful.
Cherry Zhang [Mon, 28 Oct 2019 04:50:39 +0000 (00:50 -0400)]
cmd/compile, cmd/internal/obj/ppc64: mark unsafe points
We'll use CTR as a scratch register for call injection. Mark code
sequences that use CTR as unsafe for async preemption. Currently
it is only used in LoweredZero and LoweredMove. It is unfortunate
that they are nonpreemptible. But I think it is still better than
using LR for call injection and marking all leaf functions
nonpreemptible.
Also mark the prologue of large frame functions nonpreemptible,
as we write below SP.
Cherry Zhang [Mon, 28 Oct 2019 04:49:13 +0000 (00:49 -0400)]
cmd/compile, cmd/internal/obj/ppc64: use LR for indirect calls
On PPC64, indirect calls can be made through LR or CTR. Currently
both are used. This CL changes it to always use LR.
For async preemption, to return from the injected call, we need
an indirect jump back to the PC we preeempted. This jump can be
made through LR or CTR. So we'll have to clobber either LR or CTR.
Currently, LR is used more frequently. In particular, for a leaf
function, LR is live throughout the function. We don't want to
make leaf functions nonpreemptible. So we choose CTR for the call
injection. For code sequences that use CTR, if it is ok to use
another register, change it to.
Plus, it is a call so it will clobber LR anyway. It doesn't need
to also clobber CTR (even without preemption).
Cherry Zhang [Sun, 27 Oct 2019 02:49:13 +0000 (22:49 -0400)]
cmd/internal/obj/mips: mark unsafe points
For async preemption, we will be using REGTMP as a temporary
register in injected call on MIPS, which will clobber it. So any
code that uses REGTMP is not safe for async preemption.
In the assembler backend, we expand a Prog to multiple machine
instructions and use REGTMP as a temporary register if necessary.
These need to be marked unsafe. In fact, most of the
multi-instruction Progs use REGTMP, so we mark all of them,
except ones that are whitelisted.
Cherry Zhang [Mon, 21 Oct 2019 18:07:50 +0000 (14:07 -0400)]
runtime: add async preemption support on ARM64
This CL adds support of call injection and async preemption on
ARM64.
There seems no way to return from the injected call without
clobbering *any* register. So we have to clobber one, which is
chosen to be REGTMP. Previous CLs have marked code sequences
that use REGTMP async-nonpreemtible.