crypto/rsa: refactor RSA-PSS signing and verification
Cleaned up for readability and consistency.
There is one tiny behavioral change: when PSSSaltLengthEqualsHash is
used and both hash and opts.Hash were set, hash.Size() was used for the
salt length instead of opts.Hash.Size(). That's clearly wrong because
opts.Hash is documented to override hash.
Joel Sing [Tue, 25 Feb 2020 15:48:24 +0000 (02:48 +1100)]
cmd/link: skip symbol references when looking for missing symbols
ErrorUnresolved attempts to find the missing symbol in another ABI,
in order to provide more friendly error messages. However, in doing so
it checks the same ABI and can find the symbol reference for the symbol
that it is currently reporting the unresolved error for. Avoid this by
ignoring SXREF symbols, which is the same behaviour used when linking
is performed.
fanzha02 [Thu, 19 Dec 2019 08:15:06 +0000 (08:15 +0000)]
cmd/asm: align an instruction or a function's address
Recently, the gVisor project needs an instruction's address
with 128 bytes alignment to fit the architecture requirement
for interrupt table.
This patch allows aligning an instruction's address to be
aligned to a specific value (2^n and in the range [8, 2048])
The main changes include:
1. Adds a new element in the FuncInfo structure defined in
cmd/internal/obj/link.go file to record the alignment
information.
2. Adds a new element in the Func structure defined in
cmd/internal/goobj/read.go file to read the alignment
information.
3. Adds the assembler support to align an intruction's offset
with a specific value (2^n and in the range [8, 2048]).
e.g. "PCALIGN $256" indicates that the next instruction should
be aligned to 256 bytes.
4. An instruction's alignment is relative to the start of the
function where this instruction is located, so the function's
address must be aligned to the same or coarser boundary.
Michael Munday [Wed, 1 Apr 2020 10:21:03 +0000 (03:21 -0700)]
cmd/compile: mark 'store multiple' as clobbering flags on s390x
Store multiple instructions can clobber flags on s390x when the
offset passed into the assembler is outside the range representable
with a signed 20 bit integer. This is because the assembler uses
the agfi instruction to implement the large offset. The assembler
could use a different sequence of instructions, but for now just
mark the instruction as 'clobberFlags' since this is risk free.
Noticed while investigating #38195.
No test yet since I'm not sure how to get this bug to trigger and
I haven't seen it affect real code.
Change-Id: I4a6ab96455a3ef8ffacb76ef0166b97eb40ff925
Reviewed-on: https://go-review.googlesource.com/c/go/+/226759
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Alex Brainman [Sat, 7 Mar 2020 00:08:06 +0000 (11:08 +1100)]
internal/syscall/windows: change WSAMsg.Name type
The problem was discovered while running
go test -a -short -gcflags=all=-d=checkptr -run=TestUDPConnSpecificMethods net
WSAMsg is type defined by Windows. And WSAMsg.Name could point to two
different structures for IPv4 and IPV6 sockets.
Currently WSAMsg.Name is declared as *syscall.RawSockaddrAny. But that
violates
(1) Conversion of a *T1 to Pointer to *T2.
rule of
https://golang.org/pkg/unsafe/#Pointer
When we convert *syscall.RawSockaddrInet4 into *syscall.RawSockaddrAny,
syscall.RawSockaddrInet4 and syscall.RawSockaddrAny do not share an
equivalent memory layout.
Same for *syscall.SockaddrInet6 into *syscall.RawSockaddrAny.
This CL changes WSAMsg.Name type to *syscall.Pointer. syscall.Pointer
length is 0, and that at least makes type checker happy.
After this change I was able to run
go test -a -short -gcflags=all=-d=checkptr std cmd
without type checker complaining.
Updates #34972
Change-Id: Ic5c2321c20abd805c687ee16ef6f643a2f8cd93f
Reviewed-on: https://go-review.googlesource.com/c/go/+/222457
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
We can initialize the runtime sigqueue packages on first use.
We don't require an explicit initialization step. So, remove it.
Change-Id: I484e02dc2c67395fd5584f35ecda2e28b37168df
Reviewed-on: https://go-review.googlesource.com/c/go/+/226540
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Keith Randall [Tue, 3 Mar 2020 18:07:32 +0000 (18:07 +0000)]
reflect: when Converting between float32s, don't lose signal NaNs
Trying this CL again, with a test that skips 387.
When converting from float32->float64->float32, any signal NaNs
get converted to quiet NaNs. Avoid that so using reflect.Value.Convert
between two float32 types keeps the signal bit of NaNs.
Skip the test on 387. I don't see any sane way of ensuring that a
float load + float store is faithful on that platform.
Bradford Lamson-Scribner [Sun, 29 Mar 2020 19:17:46 +0000 (13:17 -0600)]
cmd/compile: combine ssa.html columns with identical contents
Combine columns in ssa.html output if they are identical. There
can now be multiple titles per column which are all clickable to
expand and collapse their column. Give collapsed columns some
padding for better readability. Some of the work in this CL was
started by Josh Bleecher Snyder and mailed to me in order to
continue to completion.
Updates #37766
Change-Id: I313b0917dc1bafe1eb99d91798ea915e5bcfaae9
Reviewed-on: https://go-review.googlesource.com/c/go/+/226209 Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
cmd/compile, runtime: use more registers for amd64 write barrier calls
The compiler-inserted write barrier calls use a special ABI
for speed and to minimize the binary size impact.
runtime.gcWriteBarrier takes its args in DI and AX.
This change adds gcWriteBarrier wrapper functions,
varying only in the register used for the second argument.
(Allowing variation in the first argument doesn't offer improvements,
which is convenient, as it avoids quadratic API growth.)
This reduces the number of register copies.
The goals are reduced binary size via reduced register pressure/copies.
One downside to this change is that when the write barrier is on,
we may bounce through several different write barrier wrappers,
which is bad for the instruction cache.
Package runtime write barrier benchmarks for this change:
name old time/op new time/op delta
WriteBarrier-8 16.6ns ± 6% 15.6ns ± 6% -5.73% (p=0.000 n=97+99)
BulkWriteBarrier-8 4.37ns ± 7% 4.22ns ± 8% -3.45% (p=0.000 n=96+99)
However, I don't particularly trust these numbers.
I ran runtime.BenchmarkWriteBarrier multiple times as I rebased
this change, and noticed that the results have high variance
depending on the parent change, perhaps due to aligment.
This change was stress tested with GOGC=1 GODEBUG=gccheckmark=1 go test std.
In CL 187657, I refactored constant conversion logic without realizing
that conversions between int/float and complex types are allowed for
constants (assuming the constant values are representable by the
destination type), but are never allowed for non-constant expressions.
This CL expands convertop to take an extra srcConstant parameter to
indicate whether the source expression is a constant; and if so, to
allow any numeric-to-numeric conversion. (Conversions of values that
cannot be represented in the destination type are rejected by
evconst.)
Fixes #38117.
Change-Id: Id7077d749a14c8fd910be38da170fa5254819f2b
Reviewed-on: https://go-review.googlesource.com/c/go/+/226197
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Jay Conrod [Thu, 12 Mar 2020 22:56:40 +0000 (18:56 -0400)]
cmd/go: add support for GOPROXY fallback on unexpected errors
URLs in GOPROXY may now be separated with commas (,) or pipes (|). If
a request to a proxy fails with any error (including connection errors
and timeouts) and the proxy URL is followed by a pipe, the go command
will try the request with the next proxy in the list. If the proxy is
followed by a comma, the go command will only try the next proxy if
the error a 404 or 410 HTTP response.
The go command will determine how to connect to the checksum database
using the same logic. Before accessing the checksum database, the go
command sends a request to <proxyURL>/sumdb/<sumdb-name>/supported.
If a proxy responds with 404 or 410, or if any other error occurs and
the proxy URL in GOPROXY is followed by a pipe, the go command will
try the request with the next proxy. If all proxies respond with 404
or 410 or are configured to fall back on errors, the go command will
connect to the checksum database directly.
This CL does not change the default value or meaning of GOPROXY.
Fixes #37367
Change-Id: I35dd218823fe8cb9383e9ac7bbfec2cc8a358748
Reviewed-on: https://go-review.googlesource.com/c/go/+/226460
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Bryan C. Mills [Mon, 30 Mar 2020 19:43:08 +0000 (15:43 -0400)]
os/signal: make TestStop resilient to initially-blocked signals
For reasons unknown, SIGUSR1 appears to be blocked at process start
for tests on the android-arm-corellium and android-arm64-corellium
builders. (This has been observed before, too: see CL 203957.)
Make the test resilient to blocked signals by always calling Notify
and waiting for potential signal delivery after sending any signal
that is not known to be unblocked.
Also remove the initial SIGWINCH signal from testCancel. The behavior
of an unhandled SIGWINCH is already tested in TestStop, so we don't
need to re-test that same case: waiting for an unhandled signal takes
a comparatively long time (because we necessarily don't know when it
has been delivered), so this redundancy makes the overall test binary
needlessly slow, especially since it is called from both TestReset and
TestIgnore.
Since each signal is always unblocked while we have a notification
channel registered for it, we don't need to modify any other tests:
TestStop and testCancel are the only functions that send signals
without a registered channel.
Fixes #38165
Updates #33174
Updates #15661
Change-Id: I215880894e954b62166024085050d34323431b63
Reviewed-on: https://go-review.googlesource.com/c/go/+/226461
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Richard Miller [Tue, 31 Mar 2020 18:44:19 +0000 (19:44 +0100)]
runtime: skip gdb tests on Plan 9
There's no gdb on Plan 9.
Change-Id: Ibeb0fbd3c096a69181c19b1fb2bc6291612b6da3
Reviewed-on: https://go-review.googlesource.com/c/go/+/226657 Reviewed-by: David du Colombier <0intro@gmail.com>
Bryan C. Mills [Mon, 30 Mar 2020 18:04:08 +0000 (14:04 -0400)]
context: fix a flaky timeout in TestLayersTimeout
In CL 223019, I reduced the short timeout in the testLayers helper to
be even shorter than it was. That exposed a racy (time-dependent)
select later in the function, which failed in one of the slower
builders (android-386-emu).
Also streamline the test to make it easier to test with a very high -count flag:
- Run tests that sleep for shortDuration in parallel to reduce latency.
- Use shorter durations in examples to reduce test running time.
- Avoid mutating global state (in package math/rand) in testLayers.
After this change (but not before it),
'go test -run=TestLayersTimeout -count=100000 context' passes on my workstation.
Fixes #38161
Change-Id: Iaf4abe7ac308b2100d8828267cda9f4f8ae4be82
Reviewed-on: https://go-review.googlesource.com/c/go/+/226457 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Per RFC 8017, reject signatures which are not the same length as the RSA
modulus. This matches the behavior of SignPKCS1v15 which properly left pads
the signatures it generates to the size of the modulus.
Joel Sing [Mon, 30 Mar 2020 14:57:52 +0000 (01:57 +1100)]
cmd/asm,cmd/internal/obj/riscv: provide branch pseudo-instructions
Implement various branch pseudo-instructions for riscv64. These make it easier
to read/write assembly and will also make it easier for the compiler to generate
optimised code.
fanzha02 [Thu, 19 Dec 2019 08:15:06 +0000 (08:15 +0000)]
cmd/asm: align an instruction or a function's address
Recently, the gVisor project needs an instruction's address
with 128 bytes alignment and a function's start address with
2K bytes alignment to fit the architecture requirement for
interrupt table.
This patch allows aligning the address of an instruction to be
aligned to a specific value (2^n and not higher than 2048) and
the address of a function to be 2048 bytes.
The main changes include:
1. Adds ALIGN2048 flag to align a function's address with
2048 bytes.
e.g. "TEXT ·Add(SB),NOSPLIT|ALIGN2048" indicates that the
address of Add function should be aligned to 2048 bytes.
2. Adds a new element in the FuncInfo structure defined in
cmd/internal/obj/link.go file to record the alignment
information.
3. Adds a new element in the Func structure defined in
cmd/internal/goobj/read.go file to read the alignment
information.
4. Because go introduces a new object file format, also add
a new element in the FuncInfo structure defined in
cmd/internal/goobj2/funcinfo.go to record the alignment
information.
5. Adds the assembler support to align an intruction's offset
with a specific value (2^n and not higher than 2048).
e.g. "PCALIGN $256" indicates that the next instruction should
be aligned to 256 bytes.
This CL also adds a test.
Change-Id: I31cfa6fb5bc35dee2c44bf65913e90cddfcb492a
Reviewed-on: https://go-review.googlesource.com/c/go/+/212767 Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Bryan C. Mills [Mon, 30 Mar 2020 19:01:33 +0000 (15:01 -0400)]
os/signal: in TestStop, skip the final "unexpected signal" check for SIGUSR1 on Android
In CL 226138, I updated TestStop to have more uniform behavior for its signals.
However, that test seems to always fail for SIGUSR1 on the Android ARM builders.
I'm not sure what's special about Android for this particular case,
but let's skip the test to unbreak the builders while I investigate.
For #38165
Updates #33174
Change-Id: I35a70346cd9757a92acd505a020bf95e6871405c
Reviewed-on: https://go-review.googlesource.com/c/go/+/226458
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Giovanni Bajo [Sun, 29 Mar 2020 12:21:12 +0000 (14:21 +0200)]
cmd/compile: avoid zero extensions after 32-bit shifts
zeroUpper32Bits wasn't checking for shift-extension ops. This would not
check shifts that were marking as bounded by prove (normally, shifts
are wrapped in a sequence that ends with an ANDL, and zeroUpper32Bits
would see the ANDL).
This produces no changes on generated output right now, but will be
important once CL196679 lands because many shifts will be marked
as bounded, and lower will stop generating the masking code sequence
around them.
Change-Id: Iaea94acc5b60bb9a5021c9fb7e4a1e2e5244435e
Reviewed-on: https://go-review.googlesource.com/c/go/+/226338 Reviewed-by: Keith Randall <khr@golang.org>
Tobias Klauser [Sun, 29 Mar 2020 22:38:09 +0000 (00:38 +0200)]
cmd/cgo, misc/cgo: only cache anonymous struct typedefs with parent name
CL 181857 broke the translation of certain C types using cmd/cgo -godefs
because it stores each typedef, array and qualified type with their
parent type name in the translation cache.
Fix this by only considering the parent type for typedefs of anonymous
structs which is the only case where types might become ambiguous.
Updates #31891
Fixes #37479
Fixes #37621
Change-Id: I301a749ec89585789cb0d213593bb8b7341beb88
Reviewed-on: https://go-review.googlesource.com/c/go/+/226341
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Fri, 27 Mar 2020 19:04:09 +0000 (15:04 -0400)]
os/signal: rework test timeouts and concurrency
Use a uniform function (named “quiesce”) to wait for possible signals
in a way that gives the kernel many opportunities to deliver them.
Simplify channel usage and concurrency in stress tests.
Use (*testing.T).Deadline instead of parsing the deadline in TestMain.
In TestStop, sleep forever in a loop if we expect the test to die from
a signal. That should reduce the flakiness of TestNohup, since
TestStop will no longer spuriously pass when run as a subprocess of
TestNohup.
Since independent signals should not interfere, run the different
signals in TestStop in parallel when testing in short mode.
Since TestNohup runs TestStop as a subprocess, and TestStop needs to
wait many times for signals to quiesce, run its test subprocesses
concurrently and in short mode — reducing the latency of that test by
more than a factor of 2.
The above two changes reduce the running time of TestNohup on my
workstation to ~345ms, making it possible to run much larger counts of
the test in the same amount of wall time. If the test remains flaky
after this CL, we can spend all or part of that latency improvement on
a longer settle time.
Updates #33174
Change-Id: I09206f213d8c1888b50bf974f965221a5d482419
Reviewed-on: https://go-review.googlesource.com/c/go/+/226138
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Lynn Boger [Wed, 25 Mar 2020 17:47:43 +0000 (13:47 -0400)]
cmd/objdump: add support for -gnu option on Go objdump
This adds support for the -gnu option on Go objdump. When
this option is used, then output will include gnu
assembly in comments alongside the Go assembly.
The objdump test was updated to test this new option.
This option is supported for the arches found in
golang.org/x that provide the GNUsyntax function.
Michael Anthony Knyszek [Sat, 28 Mar 2020 16:11:15 +0000 (16:11 +0000)]
runtime: check the correct sanity condition in the page allocator
Currently there are a few sanity checks in the page allocator which
should fail immediately but because it's a check for a negative number
on a uint, it's actually dead-code.
If there's a bug in the page allocator which would cause the sanity
check to fail, this could cause memory corruption by returning an
invalid address (more precisely, one might either see a segfault, or
span overlap).
This change fixes these sanity checks to check the correct condition.
Fixes #38130.
Change-Id: Ia19786cece783d39f26df24dec8788833a6a3f21
Reviewed-on: https://go-review.googlesource.com/c/go/+/226297 Reviewed-by: Giovanni Bajo <rasky@develer.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Michał Łowicki [Sun, 29 Mar 2020 16:59:08 +0000 (17:59 +0100)]
doc: fix path to make.bash
Change-Id: I78c7197b8b93590470a782b492bba177a14d80ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/226340 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Daniel Theophanes [Wed, 18 Mar 2020 17:03:51 +0000 (10:03 -0700)]
database/sql: add test for Conn.Validator interface
This addresses comments made by Russ after
https://golang.org/cl/174122 was merged. It addes a test
for the connection validator and renames the interface to just
"Validator".
Giovanni Bajo [Sat, 21 Mar 2020 12:22:14 +0000 (13:22 +0100)]
doc: decrease prominence of GOROOT_BOOTSTRAP
Go build scripts on UNIX (make.bash, all.bash) have not required
GOROOT_BOOTSTRAP since August 2017 (CL 57753). Windows build scripts
have followed suit since CL 96455. Most people building Go will have
a Go toolchain in their PATH and will not need to specify a different
toolchain.
This CL removes the GOROOT_BOOTSTRAP mention from the contribution guide
(it was there for Windows only, but it's not required anymore). The guide
is meant to be light and clear for beginners and is not supposed to be
a reference, so there's not need to keep mentioning GOROOT_BOOTSTRAP.
Also update install-source.html to reflect the current status quo,
where using the PATH is probably the first and most used default, and
GOROOT_BOOTSTRAP is just an option.
Change-Id: Iab453e61b0c749c256aaaf81ea9b2ae58822cb89
Reviewed-on: https://go-review.googlesource.com/c/go/+/224717
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Michael Matloob [Thu, 27 Feb 2020 22:14:07 +0000 (17:14 -0500)]
cmd/go: avoid needing to manipulate ImportStack when constructing error
Simplify the printing of PackageErrors by pushing and popping packages
from the import stack when creating the error, rather than when printing
the error. In some cases, we don't have the same amount of information
to recreate the exact error, so we'll print the name of the package
the error is for, even when it's redundant. In the case of import cycle
errors, this change results in the addition of the position information
of the error.
This change supercedes CLs 220718 and 217106. It introduces a simpler
way to format errors.
Fixes #36173
Change-Id: Ie27011eb71f82e165ed4f9567bba6890a3849fc1
Reviewed-on: https://go-review.googlesource.com/c/go/+/224660
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Ian Lance Taylor [Thu, 26 Mar 2020 16:13:11 +0000 (09:13 -0700)]
runtime: lock mtxpoll in AIX netpollBreak
netpollBreak calls netpollwakeup, and netpollwakeup expects the mtxpoll
lock to be held, so that it has exclusive access to pendingUpdates.
Not acquiring the lock was a mistake in CL 171824. Fortunately it
rarely matters in practice.
Change-Id: I32962ec2575c846ef3d6a91a4d821b2ff02d983c
Reviewed-on: https://go-review.googlesource.com/c/go/+/225618 Reviewed-by: Michael Knyszek <mknyszek@google.com>
Keith Randall [Tue, 24 Mar 2020 20:39:44 +0000 (13:39 -0700)]
cmd/compile: convert 386 port to use addressing modes pass (take 2)
Retrying CL 222782, with a fix that will hopefully stop the random crashing.
The issue with the previous CL is that it does pointer arithmetic
in a way that may briefly generate an out-of-bounds pointer. If an
interrupt happens to occur in that state, the referenced object may
be collected incorrectly.
Suppose there was code that did s[x+c]. The previous CL had a rule
to the effect of ptr + (x + c) -> c + (ptr + x). But ptr+x is not
guaranteed to point to the same object as ptr. In contrast,
ptr+(x+c) is guaranteed to point to the same object as ptr, because
we would have already checked that x+c is in bounds.
For example, strconv.trim used to have this code:
MOVZX -0x1(BX)(DX*1), BP
CMPL $0x30, AL
After CL 222782, it had this code:
LEAL 0(BX)(DX*1), BP
CMPB $0x30, -0x1(BP)
An interrupt between those last two instructions could see BP pointing
outside the backing store of the slice involved.
It's really hard to actually demonstrate a bug. First, you need to
have an interrupt occur at exactly the right time. Then, there must
be no other pointers to the object in question. Since the interrupted
frame will be scanned conservatively, there can't even be a dead
pointer in another register or on the stack. (In the example above,
a bug can't happen because BX still holds the original pointer.)
Then, the object in question needs to be collected (or at least
scanned?) before the interrupted code continues.
This CL needs to handle load combining somewhat differently than CL 222782
because of the new restriction on arithmetic. That's the only real
difference (other than removing the bad rules) from that old CL.
This bug is also present in the amd64 rewrite rules, and we haven't
seen any crashing as a result. I will fix up that code similarly to
this one in a separate CL.
Michael Pratt [Thu, 26 Mar 2020 19:10:21 +0000 (15:10 -0400)]
runtime/pprof: increment fake overflow record PC
gentraceback generates PCs which are usually following the CALL
instruction. For those that aren't, it fixes up the PCs so that
functions processing the output can unconditionally decrement the PC.
runtime_expandInlineFrames does this unconditional decrement when
looking up the function. However, the fake stack frame generated for
overflow records fails to meet the contract, and decrementing the PC
results in a PC in the previous function. If that function contains
inlined call, runtime_expandInlineFrames will not short-circuit and will
panic trying to look up a PC that doesn't exist.
Note that the added test does not fail at HEAD. It will only fail (with
a panic) if the function preceeding lostProfileEvent contains inlined
function calls. At the moment (on linux/amd64), that is
runtime/pprof.addMaxRSS, which does not.
Fixes #38096
Change-Id: Iad0819f23c566011c920fd9a5b1254719228da0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/225661 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Bryan C. Mills [Fri, 27 Mar 2020 16:11:21 +0000 (12:11 -0400)]
net/http: remove arbitrary timeouts from TestIdentityResponse and TestTLSHandshakeTimeout
These hard-coded timeouts make the tests flaky on slow builders (such
as solaris-amd64-oraclerel), and make test failures harder to diagnose
anyway (by replacing dumps of the stuck goroutine stacks with failure
messages that do not describe the stuck goroutines). Eliminate them
and simplify the tests.
Andy Pan [Fri, 27 Mar 2020 03:21:17 +0000 (03:21 +0000)]
runtime: converge duplicate calls to netpollBreak into one
There might be some concurrent (maybe not concurrent, just sequential but in a short time window) and duplicate calls to `netpollBreak`, trying to wake up a net-poller. If one has called `netpollBreak` and that waking event hasn't been received by epollwait/kevent/..., then the subsequent calls of `netpollBreak` ought to be ignored or in other words, these calls should be converged into one.
Lynn Boger [Thu, 26 Mar 2020 20:01:40 +0000 (16:01 -0400)]
cmd/compile: add rules to eliminate unnecessary signed shifts
This change to the rules removes some unnecessary signed shifts
that appear in the math/rand functions. Existing rules did not
cover some of the signed cases.
A little improvement seen in math/rand due to removing 1 of 2
instructions generated for Int31n, which is inlined quite a bit.
Bryan C. Mills [Fri, 27 Mar 2020 03:10:32 +0000 (23:10 -0400)]
cmd/go/internal/work: disallow testgo binary from installing to GOROOT
Installing to GOROOT makes tests non-parallelizable, since each test
depends on the installed contents of GOROOT already being up-to-date
and may reasonably assume that those contents do not change over the
course of the test.
Fixes #37573
Updates #30316
Change-Id: I2afe95ad11347bee3bb7c2d77a657db6d691cf05
Reviewed-on: https://go-review.googlesource.com/c/go/+/225897 Reviewed-by: Michael Matloob <matloob@golang.org>
Bryan C. Mills [Tue, 17 Mar 2020 14:22:39 +0000 (10:22 -0400)]
run.{bash,bat,rc}: use ../bin/go instead of the go binary in $PATH
https://golang.org/doc/contribute.html#quick_test currently suggests
running 'make.bash' and 'run.bash' separately, but 'run.bash'
potentially uses a 'go' command resolved from the wrong GOROOT,
which in turn sets the wrong GOROOT for further commands.
Bryan C. Mills [Thu, 26 Mar 2020 16:00:07 +0000 (12:00 -0400)]
cmd/dist: skip API check on plan9 builders
The plan9-arm builder has a very slow filesystem and frequently times
out on this test. The api check verifies the API for all supported
GOOS/GOARCH/CGO_ENABLED combination anyway, so if we skip it on one
builder (or even most builders) there should be no loss of coverage.
Updates #37951
Change-Id: I86a93df2ec60a6af6d942e3954eef09ce67bb39e
Reviewed-on: https://go-review.googlesource.com/c/go/+/225662 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Jay Conrod [Thu, 12 Mar 2020 22:56:40 +0000 (18:56 -0400)]
cmd/go: add support for GOPROXY fallback on unexpected errors
URLs in GOPROXY may now be separated with commas (,) or pipes (|). If
a request to a proxy fails with any error (including connection errors
and timeouts) and the proxy URL is followed by a pipe, the go command
will try the request with the next proxy in the list. If the proxy is
followed by a comma, the go command will only try the next proxy if
the error a 404 or 410 HTTP response.
The go command will determine how to connect to the checksum database
using the same logic. Before accessing the checksum database, the go
command sends a request to <proxyURL>/sumdb/<sumdb-name>/supported.
If a proxy responds with 404 or 410, or if any other error occurs and
the proxy URL in GOPROXY is followed by a pipe, the go command will
try the request with the next proxy. If all proxies respond with 404
or 410 or are configured to fall back on errors, the go command will
connect to the checksum database directly.
This CL does not change the default value or meaning of GOPROXY.
Fixes #37367
Change-Id: If53152ec1c3282c67d4909818b666af58884fb2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/223257
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Bryan C. Mills [Thu, 26 Mar 2020 02:17:04 +0000 (22:17 -0400)]
cmd/go/internal/base: rename EnvForDir to AppendPWD
EnvForDir does not immediately evoke “append”, and thus may not prompt
the reader to consider the possibility of aliasing bugs (as in
issue #38077). To make this behavior more obvious at the call site, rename
cmd/go/internal/base.EnvForDir to AppendPWD and swap the order of
arguments to a conventional “append” function (similar to those in the
strconv package).
For #38077
Change-Id: I16f09aa0fa8a269d51f0511eb402a44e2759eb94
Reviewed-on: https://go-review.googlesource.com/c/go/+/225578 Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Bryan C. Mills [Thu, 26 Mar 2020 02:24:44 +0000 (22:24 -0400)]
cmd/go: do not append to the global cfg.OrigEnv slice
Appending to a global slice is only safe if its length is already
equal to its capacity. That property is not guaranteed for slices in
general, and empirically does not hold for this one.
This is a minimal fix to make it easier to backport.
A more robust cleanup of the base.EnvForDir function will be sent in a
subsequent CL.
Fixes #38077
Updates #37940
Change-Id: I731d5bbd0e516642c2cf43e713eeea15402604e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/225577
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Michael Anthony Knyszek [Wed, 18 Mar 2020 15:09:40 +0000 (15:09 +0000)]
runtime: ensure minTriggerRatio never exceeds maxTriggerRatio
Currently, the capping logic for the GC trigger ratio is such that if
gcpercent is low, we may end up setting the trigger ratio far too high,
breaking the promise of SetGCPercent and GOGC has a trade-off knob (we
won't start a GC early enough, and we will use more memory).
This change modifies the capping logic for the trigger ratio by scaling
the minTriggerRatio with gcpercent the same way we scale
maxTriggerRatio.
This makes all modern public keys in the standard library implement a
common interface (below) that can be used by applications for better
type safety and allows for checking that public (and private keys via
Public()) are equivalent.
interface {
Equal(crypto.PublicKey) bool
}
Equality for ECDSA keys is complicated, we take a strict interpretation
that works for all secure applications (the ones not using the
unfortunate non-constant time CurveParams implementation) and fails
closed otherwise.
Tests in separate files to make them x_tests and avoid an import loop
with crypto/x509.
Re-landing of CL 223754. Dropped the test that was assuming named curves
are not implemented by CurveParams, because it's not true for all
curves, and anyway is not a property we need to test. There is still a
test to check that different curves make keys not Equal.
Xiangdong Ji [Thu, 5 Dec 2019 03:22:34 +0000 (03:22 +0000)]
runtime: fix threshold calculation of TestPhysicalMemoryUtilization
Variable 'procs' used to calculate the threshold of overuse in
TestPhysicalMemoryUtilization should be updated if GOMAXPROCS
gets changed, otherwise the threshold could be a large number,
making the test meaningless.
Change-Id: I876cbf11457529f56bae77af1e35f4538a721f95
Reviewed-on: https://go-review.googlesource.com/c/go/+/210297
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Rob Pike [Wed, 25 Mar 2020 22:32:27 +0000 (09:32 +1100)]
scan: for style, adjust code for bad scan read counts
Make the code more consistent with the rest of the file.
Should have caught this in review of CL 225357.
Change-Id: I12824cb436539c31604684e043ebb7587cc92471
Reviewed-on: https://go-review.googlesource.com/c/go/+/225557 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Michael Anthony Knyszek [Wed, 25 Mar 2020 16:41:20 +0000 (16:41 +0000)]
runtime: prevent preemption while timer is in timerModifying
Currently if a goroutine is preempted while owning a timer in the
timerModifying state, it could self-deadlock. When the goroutine is
preempted and calls into the scheduler, it could call checkTimers. If
checkTimers encounters the timerModifying timer and calls runtimer on
it, then runtimer will spin, waiting for that timer to leave the
timerModifying state, which it never will.
So far we got lucky that for the most part that there were no preemption
points while timerModifying is happening, however CL 221077 seems to
have introduced one, leading to sporadic self-deadlocks.
This change disables preemption explicitly while a goroutines holds a
timer in timerModifying. Since only checkTimers (and thus runtimer) is
called from the scheduler, this is sufficient to prevent
preemption-based self-deadlocks.
Fixes #38070.
Updates #37894.
Change-Id: Idbfac310889c92773023733ff7e2ff87e9896f0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/225497
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Fri, 20 Mar 2020 19:15:35 +0000 (15:15 -0400)]
cmd/api: make NewWatcher populate its own package and import metadata
This partially undoes the optimizations of CL 177597, but makes up
some of the difference by caching the package list and import metadata
and making the initial calls concurrently, including in TestMain.
That reduces the critical path from two sequential 'go list'
invocations to just one (run many times concurrently), and eliminates
the need for assumptions about the consistency of the 'std' dependency
graph across platforms (and hard-coded special cases for packages that
violate those assumptions).
In the process, this simplifies and fixes TestBenchmark (which has
been silently broken since CL 164623).
This increases 'time go tool dist test api' on my workstation from
0m8.4s / 0m13.8s / 0m1.7s to 0m10.5s / 0m23.1s / 0m5.1s,
compared to 0m12.4s / 0m23.2s / 0m4.7s before CL 177597.
(That is, this change retains about half of the wall-time speedup, but
almost none of the user-time speedup.)
Tested manually using 'go test -race -bench=. cmd/api'.
Fixes #37951
Change-Id: Icd537e035e725e1ee7c41d97da5c6651233b927e
Reviewed-on: https://go-review.googlesource.com/c/go/+/224619
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
fanzha02 [Wed, 25 Mar 2020 03:11:24 +0000 (11:11 +0800)]
cmd/link/internal/arm64: increase the function alignment to 16
On arm64, a function's address is 16 bytes aligned, and
the assembler aligns the size of function symbol to 16 bytes,
so to keep the consistent, this patch changes the function
alignment in the linker to 16 bytes.
cmd/internal/obj/arm64: add support of PCALIGN directive
Recently, we get requirements of instructions and functions alignment
from the gVisor project. To fit the alignment requirement of interrupt
table, they require an instruction's address to be aligned 128 bytes
and a function's entry address to be aligned 2K bytes. Thus we add
support for PCALIGN directive first. Below is a discussion about this
topic. https://groups.google.com/forum/m/#!topic/golang-dev/RPj90l5x86I
Functions in Go are aligned to 16 bytes on arm64, thus now we only
support 8 and 16 bytes alignment.
This patch adds support for PCALIGN directive. This directive can be
used within Go asm to align instruction by padding NOOP directives.
This patch also adds a test to verify the correnctness of the PCALIGN
directive. The test is contributed by Fannie Zhang <Fannie.Zhang@arm.com>.
Bryan C. Mills [Tue, 24 Mar 2020 20:18:02 +0000 (16:18 -0400)]
test: make runindir tests pass regardless of whether module mode is in use
The "runindir" tests used "go run", but relied on relative imports
(which are not supported by "go run" in module mode). Instead, such
tests must use fully-qualified imports, which require either a go.mod
file (in module mode) or that the package be in an appropriate
subdirectory of GOPATH/src (in GOPATH mode).
To set up such a directory, we use yet another copy of the same
overlayDir function currently found in the misc subdirectory of this
repository.
Fixes #33912
Updates #30228
Change-Id: If3d7ea2f7942ba496d98aaaf24a90bcdcf4df9f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/225205 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joel Sing [Mon, 9 Mar 2020 16:31:22 +0000 (03:31 +1100)]
cmd/compile: fold constants into immediate instructions on riscv64
Where possible, fold constants into versions of instructions that take
an immediate. This avoids the need to allocate a register and load the
immediate into it.
Change-Id: If911ca41235e218490679aed2ce5f48bf807a2b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/222639
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Lynn Boger [Tue, 17 Mar 2020 13:24:47 +0000 (09:24 -0400)]
cmd/internal/obj/ppc64: fix PCALIGN on ppc64le
This fixes a potential issue with the previous implementation
of PCALIGN on ppc64. Previously PCALIGN was processed inside of
asmout and indicated the padding size by setting the value in
the optab, changing it back after the alignment instructions
were added. Now PCALIGN is processed outside of asmout, and optab
is not changed.
Ian Lance Taylor [Tue, 24 Mar 2020 04:11:01 +0000 (21:11 -0700)]
internal/poll: assume we have CancelIoEX on Windows
As of the Go 1.11 release we require at least Windows 7, so CancelIoEx
is always available. This lets us simplify the code to not require
dedicated threads to handle I/O requests.
Fixes #37956
Change-Id: If1dc4ac4acb61c43e4f2a9f26f225869050262a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/225060
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Dan Scales [Fri, 20 Mar 2020 16:31:20 +0000 (09:31 -0700)]
runtime: fix code so defer record is not added to g0 defer list during panic
newdefer() actually adds the new defer to the current g's defer chain. That
happens even if we are on the system stack, in which case the g will be the g0
stack. For open-coded defers, we call newdefer() (only during panic processing)
while on the system stack, so the new defer is unintentionally added to the
g0._defer defer list. The code later correctly adds the defer to the user g's
defer list.
The g0._defer list is never used. However, that pointer on the g0._defer list can
keep a defer struct alive that is intended to be garbage-collected (smaller defers
use a defer pool, but larger-sized defer records are just GC'ed). freedefer() does
not zero out pointers when it intends that a defer become garbage-collected. So,
we can have the pointers in a defer that is held alive by g0._defer become invalid
(in particular d.link). This is the cause of the bad pointer bug in this issue
The fix is to change newdefer (only used in two places) to not add the new defer
to the gp._defer list. We just do it after the call with the correct gp pointer.
(As mentioned above, this code was already there after the newdefer in
addOneOpenDeferFrame.) That ensures that defers will be correctly
garbage-collected and eliminate the bad pointer.
This fix definitely fixes the original repro. I added a test and tried hard to
reproduce the bug (based on the original repro code), but awasn't actually able to
cause the bug. However, the test is still an interesting mix of heap-allocated,
stack-allocated, and open-coded defers.
Fixes #37688
Change-Id: I1a481b9d9e9b9ba4e8726ef718a1f4512a2d6faf
Reviewed-on: https://go-review.googlesource.com/c/go/+/224581
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Ian Lance Taylor [Tue, 24 Mar 2020 03:42:29 +0000 (20:42 -0700)]
runtime: always use GetQueuedCompletionStatusEx on Windows
We used to fall back to GetQueuedCompletionStatus if
GetQueuedCompletionStatus was not available, but as of Go 1.11 we
require Windows 7 or later, so GetQueuedCompletionStatusEx is always
available.
Fixes #37957
Change-Id: I7d8d49a92ab7b1f5afdc54a442f696aaf4a5168e
Reviewed-on: https://go-review.googlesource.com/c/go/+/225059
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
nao20010128nao [Sat, 21 Mar 2020 06:52:58 +0000 (06:52 +0000)]
syscall/js: make wasm_exec.js compatible with Webpack
In Webpack, require("fs") will always be empty. This behavior throws an error: "fs.writeSync is not function". It happens when you did "fmt.Println".
This PR avoids such problem and use polyfill in wasm_exec.js on Webpack.