Damien Neil [Fri, 1 Sep 2023 18:17:19 +0000 (11:17 -0700)]
path/filepath: fix various issues in parsing Windows paths
On Windows, A root local device path is a path which begins with
\\?\ or \??\. A root local device path accesses the DosDevices
object directory, and permits access to any file or device on the
system. For example \??\C:\foo is equivalent to common C:\foo.
The Clean, IsAbs, IsLocal, and VolumeName functions did not
recognize root local device paths beginning with \??\.
Clean could convert a rooted path such as \a\..\??\b into
the root local device path \??\b. It will now convert this
path into .\??\b.
IsAbs now correctly reports paths beginning with \??\
as absolute.
IsLocal now correctly reports paths beginning with \??\
as non-local.
VolumeName now reports the \??\ prefix as a volume name.
Join(`\`, `??`, `b`) could convert a seemingly innocent
sequence of path elements into the root local device path
\??\b. It will now convert this to \.\??\b.
In addition, the IsLocal function did not correctly
detect reserved names in some cases:
- reserved names followed by spaces, such as "COM1 ".
- "COM" or "LPT" followed by a superscript 1, 2, or 3.
IsLocal now correctly reports these names as non-local.
Joel Sing [Fri, 25 Aug 2023 18:19:40 +0000 (04:19 +1000)]
cmd/internal/obj/riscv: support subtraction with a constant
Allow SUB and SUBW to be specified with a constant, which are mapped
to ADDI and ADDIW with negated values.
Change-Id: I7dc55692febc81ea87393b0a3a7d23a43c30313b
Reviewed-on: https://go-review.googlesource.com/c/go/+/538915
Run-TryBot: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: M Zhuo <mzh@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Mark Ryan <markdryan@rivosinc.com> Reviewed-by: Wang Yaduo <wangyaduo@linux.alibaba.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Joel Sing [Sat, 4 Nov 2023 15:32:08 +0000 (02:32 +1100)]
syscall: provide and use ioctlPtr for all BSD platforms
Provide ioctlPtr for all BSD platforms, then use this for BPF.
This reduces darwin specific code, as well as avoiding the use of
an indirect system call on OpenBSD.
Joel Sing [Sat, 4 Nov 2023 15:19:46 +0000 (02:19 +1100)]
syscall: provide and use fcntlPtr for all BSD platforms
Provide fcntlPtr for all BSD platforms, then use this for FcntlFlock.
This reduces darwin and openbsd specific code, as well as avoiding
the use of an indirect system call on OpenBSD.
Wang Yaduo [Fri, 27 Oct 2023 06:48:25 +0000 (14:48 +0800)]
cmd/internal/obj/riscv: fix the offset of JALR transformed from JAL
Currently, the offset of JALR is zero all the time, which is transformed
from JAL with over ±1MB offset. This causes the segment fault for the
wrong address.
Change-Id: I4dcb3eb13bd1ea71e9eb27f07c03ffec376608ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/538135
Run-TryBot: M Zhuo <mzh@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: M Zhuo <mzh@golangcn.org> Reviewed-by: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Mui <cherryyz@google.com>
net/http: remove Content-Encoding header in roundtrip_js
The fetch api will decode the gzip, but Content-Encoding not be deleted.
To ensure that the behavior of roundtrip_js is consistent with native. delete the Content-Encoding header when the response body is decompressed by js fetch api.
Fixes #63139
Change-Id: Ie35b3aa050786e2ef865f9ffa992e30ab060506e
Reviewed-on: https://go-review.googlesource.com/c/go/+/530155
Commit-Queue: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Mauri de Souza Meneguzzo [Wed, 1 Nov 2023 23:19:39 +0000 (23:19 +0000)]
net/netip: allow only valid prefix digits in ParsePrefix
The prefix bits for a call to ParsePrefix are passed raw to
strconv.Atoi, this means that it can accept +- signs as well as leading
zeroes, which are not allowed prefix values following RFC 4632 Section
3.1 and RFC 4291 Section 2.3.
Validate non-digit characters as well as leading zeroes and return an
error accordingly.
* Request is a range request.
* Content is encoded (e.g., gzip compressed).
* Content-Length of the encoded content has been set before calling
http.ServeContent, as suggested in https://github.com/golang/go/issues/19420.
The issue is that http.ServeContent currently sees Content-Length as
something optional when Content-Encoding is set, but that is a problem
with range request which can send a payload of different size. So this
reverts https://go.dev/cl/4538111 and makes Content-Length be set
always to the number of bytes which will actually be send (both for
range and non-range requests).
Switch the "comdatDefinitions" map to lazy allocation; we only need it
for loading PE objects, no point doing an allocation during package
init if we don't need it.
Change-Id: Ie33f2c56e964f35ac2e137840ac021cfaaa897c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/540255 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Jayanth Krishnamurthy [Wed, 1 Nov 2023 19:43:42 +0000 (14:43 -0500)]
cmd/compile: adding rule to eliminate ANDCCconst
For example, the Slicemask rule in PPC64 generates a sequence wherein there is andi operation, after an sradi, which can be replaced by srdi. This new rule eliminates ANDCCconst.
Evan Jones [Tue, 31 Oct 2023 13:04:15 +0000 (09:04 -0400)]
testing: correct comments on runCleanup
The comment on runCleanup states "If catchPanic is true ...", but
there is no catchPanic argument or variable. This was introduced
in CL 214822, which introduced the panicHandling type. The code was
updated during code review, but the comment was missed.
Mauri de Souza Meneguzzo [Sun, 5 Nov 2023 16:48:15 +0000 (16:48 +0000)]
runtime: fix badmorestackg0 never called on wasm
Previously, badmorestackg0 was never called since it was behind a g ==
R1 check, R1 holding g.m. This is clearly wrong, since we want to check
if g == g0. Fixed by using R2 that holds the value of g0.
Michael Anthony Knyszek [Fri, 3 Nov 2023 17:16:51 +0000 (17:16 +0000)]
runtime: donate racectx to g0 in ReadMetricsSlow
ReadMetricsSlow was updated to call the core of readMetrics on the
systemstack to prevent issues with stat skew if the stack gets moved
between readmemstats_m and readMetrics. However, readMetrics calls into
the map implementation, which has race instrumentation. The system stack
typically has no racectx set, resulting in crashes.
Donate racectx to g0 like the tracer does, so that these accesses don't
crash.
For #60607.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-race
Change-Id: Ic0251af2d9b60361f071fe97084508223109480c
Reviewed-on: https://go-review.googlesource.com/c/go/+/539695 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Michael Anthony Knyszek [Thu, 2 Nov 2023 05:51:20 +0000 (05:51 +0000)]
runtime: eliminate possible stack movements in ReadMetricsSlow
Currently it's possible (and even probable, with mayMoreStackMove mode)
for a stack allocation to occur between readmemstats_m and readMetrics
in ReadMetricsSlow. This can cause tests to fail by producing metrics
that are inconsistent between the two sources.
Fix this by breaking out the critical section of readMetrics and calling
that from ReadMetricsSlow on the systemstack. Our main constraint in
calling readMetrics on the system stack is the fact that we can't
acquire the metrics semaphore from the system stack. But if we break out
the critical section, then we can acquire that semaphore before we go on
the system stack.
While we're here, add another readMetrics call before readmemstats_m.
Since we're being paranoid about ways that metrics could get skewed
between the two calls, let's eliminate all uncertainty. It's possible
for readMetrics to allocate new memory, for example for histograms, and
fail while it's reading metrics. I believe we're just getting lucky
today with the order in which the metrics are produced. Another call to
readMetrics will preallocate this data in the samples slice. One nice
thing about this second read is that now we effectively have a way to
check if readMetrics really will allocate if called a second time on the
same samples slice.
Fixes #60607.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Change-Id: If6ce666530903239ef9f02dbbc3f1cb6be71e425
Reviewed-on: https://go-review.googlesource.com/c/go/+/539117
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Dmitri Shuralyov [Sun, 1 Oct 2023 21:17:44 +0000 (17:17 -0400)]
cmd/go/internal/modfetch: show real URL in response body read errors
CL 233437 added a redactedURL field to proxyRepo, a struct that already
had a field named 'url'. Neither fields were documented, so the similar
names suggest the most natural interpretation that proxyRepo.redactedURL
is equivalent to proxyRepo.url.Redacted() rather than something else.
That's possibly why it was joined with the module version in CL 406675.
It turns out the two URLs differ in more than just redaction: one is the
base proxy URL with (escaped) module path joined, the other is just the
base proxy URL, in redacted form.
Document and rename the fields to make the distinction more clear, and
include all 3 of base module proxy URL + module path + module version
in the reported URL, rather than just the first and third bits as seen
in the errors at https://go.dev/issue/51323#issuecomment-1735812250.
For #51323.
Updates #38680.
Updates #52727.
Change-Id: Ib4b134b548adeec826ee88fe51a2cf580fde0516
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/532035
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Roland Shoemaker [Thu, 2 Nov 2023 17:04:21 +0000 (10:04 -0700)]
crypto/x509: fix certificate policy marshaling
CL 520535 added the new OID type, and the Certificate field Policies to
replace PolicyIdentifiers. During review I missed three problems: (1)
the marshaling of Certificate didn't take into account the case where
both fields were populated with the same OIDs (which would be the case
if you parsed a certificate and used it as a template), (2)
buildCertExtensions only generated the certificate policies extension if
PolicyIdentifiers was populated, and (3) how we would marshal an empty
OID (i.e. OID{}).
This change makes marshaling a certificate with an empty OID an error,
and only adds a single copy of any OID that appears in both Policies and
PolicyIdentifiers to the certificate policies extension. This should
make the round trip behavior for certificates reasonable.
Additionally this change documents that CreateCertificate uses the
Policies field from the template, and fixes buildCertExtensions to
populate the certificate policies extension if _either_
PolicyIdentifiers or Policies is populated, not just PolicyIdentifiers.
Fixes #63909
Change-Id: I0fcbd3ceaab7a376e7e991ff8b37e2145ffb4a61
Reviewed-on: https://go-review.googlesource.com/c/go/+/539297 Reviewed-by: Mateusz Poliwczak <mpoliwczak34@gmail.com> Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Joel Sing [Thu, 2 Nov 2023 10:46:38 +0000 (21:46 +1100)]
runtime: remove getcallerpc on riscv64
This was converted to a compiler intrinsic and no longer needs to exist
in assembly.
Change-Id: I7495c435d4642e0e71d8f7677d70af3a3ca2a6ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/539195
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: David Chase <drchase@google.com>
Dmitri Shuralyov [Thu, 2 Nov 2023 03:08:53 +0000 (23:08 -0400)]
misc/wasm: silence Wasmtime 14 CLI warning
The latest version of Wasmtime, 14.0.4 as of writing this, offers a new
CLI while also supporting the old CLI. Since this is known and tracked
in issue #63718, silence the warning that otherwise causes many tests
to fail.
Since Wasmtime 13 and older don't pay attention to WASMTIME_NEW_CLI,
this change increases compatibility of the script, letting it work
with Wasmtime 9.0.1 as currently tested by the old cmd/coordinator, and
with Wasmtime 14.0.4 as currently tested in the new LUCI infrastructure.
The rest of the transition is left as future work.
For #63718.
For #61116.
Change-Id: I77d4f74cc1d34a657e48dcaaceb6fbda7d1e9428
Cq-Include-Trybots: luci.golang.try:gotip-wasip1-wasm_wasmtime
Reviewed-on: https://go-review.googlesource.com/c/go/+/538699
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Dmitri Shuralyov [Thu, 2 Nov 2023 18:36:30 +0000 (14:36 -0400)]
syscall: copy rlimit.go's build constraint to rlimit_test.go
Tests in rlimit_test.go exist to test the behavior of automatically
bumping RLIMIT_NOFILE on Unix implemented in rlimit.go (issue #46279),
with darwin-specific behavior split out into rlimit_darwin.go and
the rest left empty in rlimit_stub.go.
Since the behavior happens only on Unix, it doesn't make sense to test
it on other platforms. Copy rlimit.go's 'unix' build constraint to
rlimit_test.go to accomplish that.
Also simplify the build constraint in rlimit_stub.go while here,
so that its maintenance is easier and it starts to match all
non-darwin Unix GOOS values (previously, 'hurd' happened to be missed).
In particular, this fixes a problem where TestOpenFileLimit was
failing in some environments when testing the wasip1/wasm port.
The RLIMIT_NOFILE bumping behavior isn't implemented there, so
the test was testing the environment and not the Go project.
Updates #46279.
For #61116.
Change-Id: Ic993f9cfc021d4cda4fe3d7fed8e2e180f78a2ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/539435 Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Michael Pratt [Thu, 2 Nov 2023 18:16:16 +0000 (14:16 -0400)]
cmd/compile: fix unstable selection of hottest edge
When selecting the hottest edge to use for PGO-based devirtualization,
edges are order by:
1. Edge weight
2. If weights are equal, prefer the edge with IR available in the
package.
3. Otherwise, simply sort lexicographically.
The existing logic for (2) is incomplete.
If the hottest edge so far is missing IR, but the new edge has IR, then
it works as expected and selects the new edge.
But if the hottest edge so far has IR and the new edge is missing IR, we
want to always keep the hottest edge so far, but this logic will fall
through and use lexicographical ordering instead.
Adjust the check to always make an explicit choice when IR availability
differs.
Change-Id: Ia7fcc286aa9a62ac209fd978cfce60463505f4cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/539475
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Bryan C. Mills [Thu, 2 Nov 2023 14:29:08 +0000 (10:29 -0400)]
net/http: remove arbitrary timeouts in tests of Server.ErrorLog
This also allows us to remove the chanWriter helper from the test,
using a simpler strings.Builder instead, relying on
clientServerTest.close for synchronization.
(I don't think this runs afoul of #38370, because the handler
functions themselves in these tests should never be executed,
let alone result in an asynchronous write to the error log.)
Guoqi Chen [Tue, 24 Oct 2023 08:24:39 +0000 (16:24 +0800)]
debug/elf,cmd/link: add additional relocations for loong64
The Linker Relaxation feature on Loong64 is already supported in binutils 2.41.
The intermediate code generated after enabling this feature introduces three
reloc types R_LARCH_B26, R_LARCH_ADD32 and R_LARCH_SUB32.
The other relocation types are not currently used when running all.bash, but
in order to avoid the host tool chain making the decision to use it we don't
have to catch it every time.
The LoongArch ABI at here:
https://github.com/loongson/la-abi-specs/blob/release/la-abi.adoc
Change-Id: I891115cfdbcf785ab494c881d5f9d1bf8748da8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/537615 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Meidan Li <limeidan@loongson.cn>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Dmitri Shuralyov [Sun, 1 Oct 2023 20:03:01 +0000 (16:03 -0400)]
cmd/go/internal/modfetch: avoid path.Join in URL errors, part 2
CL 406675 added more detail to bare errors from net/http in two places.
CL 461682 improved one of the two places to stop folding "//" into "/".
This CL applies the same change to the other place.
For #52727.
Change-Id: I3fc13f30cf0f054949ce78269c52b7fafd477e70
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/532015 Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Sat, 28 Oct 2023 17:42:51 +0000 (17:42 +0000)]
runtime: move userArenaHeapBitsSetType into mbitmap.go
This will make the upcoming GOEXPERIMENT easier to implement, since this
function relies on a lot of heap bitmap internals.
Change-Id: I2ab76e928e7bfd383dcdb5bfe72c9b23c2002a4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/537979 Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Sat, 28 Oct 2023 16:20:04 +0000 (16:20 +0000)]
runtime: split out pointer/scalar metadata from heapArena
We're going to want to fork this data in the near future for a
GOEXPERIMENT, so break it out now.
Change-Id: Ia7ded850bb693c443fe439c6b7279dcac638512c
Reviewed-on: https://go-review.googlesource.com/c/go/+/537978 Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Michael Anthony Knyszek [Sat, 28 Oct 2023 16:30:57 +0000 (16:30 +0000)]
reflect: pass the right element type in verifyGCBitsSlice
Currently verifyGCBitsSlice creates a new array type to represent the
slice backing store, but passes the element type as the slice type in
this construction. This is incorrect, but the tests currently don't care
about it. They will in a follow-up CL, so fix it now.
Change-Id: I6ed8a9808ae78c624be316db1566376fa0e12758
Reviewed-on: https://go-review.googlesource.com/c/go/+/537981 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Russ Cox [Thu, 2 Nov 2023 12:15:44 +0000 (08:15 -0400)]
cmd/cgo: disable #cgo noescape/nocallback until Go 1.23
Go 1.21 and earlier do not understand this line, causing
"go mod vendor" of //go:build go1.22-tagged code that
uses this feature to fail.
The solution is to include the go/build change to skip over
the line in Go 1.22 (making "go mod vendor" from Go 1.22 onward
work with this change) and then wait to deploy the cgo change
until Go 1.23, at which point Go 1.21 and earlier will be unsupported.
For #56378.
Fixes #63293.
Change-Id: Ifa08b134eac5a6aa15d67dad0851f00e15e1e58b
Reviewed-on: https://go-review.googlesource.com/c/go/+/539235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
Michael Pratt [Wed, 1 Nov 2023 21:06:08 +0000 (17:06 -0400)]
os/signal: remove go t.Run from TestNohup
Since CL 226138, TestNohup has a bit of a strange construction: it wants
to run the "uncaught" subtests in parallel with each other, and the
"nohup" subtests in parallel with each other, but also needs join
between "uncaught" and "nohop" so it can Stop notifying for SIGHUP.
It achieves this by doing `go t.Run` with a WaitGroup rather than using
`t.Parallel` in the subtest (which would make `t.Run` return immediately).
However, this makes things more difficult to understand than necessary.
As noted on https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks,
a second layer of subtest can be used to join parallel subtests.
Switch to this form, which makes the test simpler to follow
(particularly the cleanup that goes with "uncaught").
Joel Sing [Tue, 31 Oct 2023 14:34:33 +0000 (01:34 +1100)]
runtime: add crash stack support for riscv64
Change-Id: Ib89a71e20f9c6b86c97814c75cb427e9bd7075e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/538735 Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
Robert Griesemer [Tue, 31 Oct 2023 21:22:05 +0000 (14:22 -0700)]
go/parser: better error messages for incorrect type parameter list
This is a port of CL 538856 from the syntax parser to go/parser.
As part of the port, make more portions of parseParameterList
matching the equivalent paramList method (from the syntax parser).
As a result, this now also produces a better error message in cases
where the missing piece might not be a type parameter name but a
constraint (this fixes a TODO in a test).
Improve comments in the code and adjust the corresponding comments
in the syntax parser.
Change references to issues to use the format go.dev/issue/ddddd.
For #60812.
Change-Id: Ia243bd78161ed8543d3dc5deb20ca4a215c5b1e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/538858 Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Joel Sing [Mon, 30 Oct 2023 13:25:52 +0000 (00:25 +1100)]
runtime: remove map stack version handling for openbsd
OpenBSD 6.3 is more than five years old and has not been supported for
the last four years (only 7.3 and 7.4 are currently supported). As such,
remove special handling of MAP_STACK for 6.3 and earlier.
Change-Id: I1086c910bbcade7fb3938bb1226813212794b587
Reviewed-on: https://go-review.googlesource.com/c/go/+/538458 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Aaron Bieber <aaron@bolddaemon.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Joel Sing <joel@sing.id.au>
Robert Griesemer [Tue, 31 Oct 2023 22:38:35 +0000 (15:38 -0700)]
spec: document range over integer expression
This CL is partly based on CL 510535.
For #61405.
Change-Id: Ic94f6726f9eb34313f11bec7b651921d7e5c18d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/538859 Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Bryan C. Mills [Thu, 26 Oct 2023 16:06:04 +0000 (12:06 -0400)]
os: report IO_REPARSE_TAG_DEDUP files as regular in Stat and Lstat
Prior to CL 460595, Lstat reported most reparse points as regular
files. However, reparse points can in general implement unusual
behaviors (consider IO_REPARSE_TAG_AF_UNIX or IO_REPARSE_TAG_LX_CHR),
and Windows allows arbitrary user-defined reparse points, so in
general we must not assume that an unrecognized reparse tag represents
a regular file; in CL 460595, we began marking them as irregular.
As it turns out, the Data Deduplication service on Windows Server runs
an Optimization job that turns regular files into reparse files with
the tag IO_REPARSE_TAG_DEDUP. Those files still behave more-or-less
like regular files, in that they have well-defined sizes and support
random-access reads and writes, so most programs can treat them as
regular files without difficulty. However, they are still reparse
files: as a result, on servers with the Data Deduplication service
enabled, files could arbitrarily change from “regular” to “irregular”
without explicit user intervention.
Since dedup files are converted in the background and otherwise behave
like regular files, this change adds a special case to report DEDUP
reparse points as regular.
Fixes #63429.
No test because to my knowledge we don't have any Windows builders
that have the deduplication service enabled, nor do we have a way to
reliably guarantee the existence of an IO_REPARSE_TAG_DEDUP file.
(In theory we could add a builder with the service enabled on a
specific volume, write a test that encodes knowledge of that volume,
and use the GO_BUILDER_NAME environment variable to run that test only
on the specially-configured builders. However, I don't currently have
the bandwidth to reconfigure the builders in this way, and given the
simplicity of the change I think it is unlikely to regress
accidentally.)
Change-Id: I649e7ef0b67e3939a980339ce7ec6a20b31b23a1
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/537915
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Robert Griesemer [Tue, 31 Oct 2023 20:29:25 +0000 (13:29 -0700)]
cmd/compile/internal/syntax: better error messages for incorrect type parameter list
When parsing a declaration of the form
type a [b[c]]d
where a, b, c, d stand for identifiers, b[c] is parsed as a type
constraint (because an array length must be constant and an index
expression b[c] is never constant, even if b is a constant string
and c a constant index - this is crucial for disambiguation of the
various possibilities).
As a result, the error message referred to a missing type parameter
name and not an invalid array declaration.
Recognize this special case and report both possibilities (because
we can't be sure without type information) with the new error:
"missing type parameter name or invalid array length"
ALso, change the previous error message
"type parameter must be named"
to
"missing type parameter name"
which is more fitting as the error refers to an absent type parameter
(rather than a type parameter that's somehow invisibly present but
unnamed).
Fixes #60812.
Change-Id: Iaad3b3a9aeff9dfe2184779f3d799f16c7500b34
Reviewed-on: https://go-review.googlesource.com/c/go/+/538856
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
Joel Sing [Mon, 30 Oct 2023 13:51:25 +0000 (00:51 +1100)]
syscall: regenerate zsyscall for dragonfly/freebsd/netbsd
The sysctl declaration was moved in CL 141639, however the files were
presumably not regenerated. There is no functional change, however
regenerating avoids unrelated noise in future diffs.
Change-Id: Ifb840b5853f3f1c3c88a3f94df21b6f6d3c635d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/538118
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Mui <cherryyz@google.com>
Keith Randall [Wed, 25 Oct 2023 20:35:13 +0000 (13:35 -0700)]
cmd/compile: ensure pointer arithmetic happens after the nil check
Have nil checks return a pointer that is known non-nil. Users of
that pointer can use the result, ensuring that they are ordered
after the nil check itself.
The order dependence goes away after scheduling, when we've fixed
an order. At that point we move uses back to the original pointer
so it doesn't change regalloc any.
This prevents pointer arithmetic on nil from being spilled to the
stack and then observed by a stack scan.
Fixes #63657
Change-Id: I1a5fa4f2e6d9000d672792b4f90dfc1b7b67f6ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/537775 Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com>
Cuong Manh Le [Tue, 31 Oct 2023 11:34:42 +0000 (18:34 +0700)]
hash/maphash: weaken avalanche test a bit more
CL 495415 weaken avalanche, making allowed range from 43% to 57%. Since
then, we only see a failure with 58% on linux-386-longtest builder, so
let give the test a bit more wiggle room: 40% to 59%.
Fixes #60170
Change-Id: I9528ebc8601975b733c3d9fd464ce41429654273
Reviewed-on: https://go-review.googlesource.com/c/go/+/538655 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Joel Sing [Mon, 30 Oct 2023 13:27:58 +0000 (00:27 +1100)]
runtime: allocate crash stack via stackalloc
On some platforms (notably OpenBSD), stacks must be specifically allocated
and marked as being stack memory. Allocate the crash stack using stackalloc,
which ensures these requirements are met, rather than using a global Go
variable.
Fixes #63794
Change-Id: I6513575797dd69ff0a36f3bfd4e5fc3bd95cbf50
Reviewed-on: https://go-review.googlesource.com/c/go/+/538457
Run-TryBot: Joel Sing <joel@sing.id.au> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Robert Griesemer [Tue, 31 Oct 2023 00:00:07 +0000 (17:00 -0700)]
cmd/compile/internal/syntax: set up dummy name and type if func name is missing
We do the same elsewhere (e.g. in parser.name when a name is missing).
This ensures functions have a (dummy) name and a non-nil type.
Avoids a crash in the type-checker (verified manually).
A test was added here (rather than the type checker) because type-
checker tests are shared between types2 and go/types and error
recovery in this case is different.
Fixes #63835.
Change-Id: I1460fc88d23d80b8d8c181c774d6b0a56ca06317
Reviewed-on: https://go-review.googlesource.com/c/go/+/538059 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Joel Sing [Wed, 9 Aug 2023 17:32:21 +0000 (03:32 +1000)]
cmd/dist,internal/platform: enable openbsd/ppc64 port
Updates #56001
Change-Id: I16440114ecf661e9fc17d304ab3b16bc97ef82f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/517935
Run-TryBot: Joel Sing <joel@sing.id.au> Reviewed-by: Paul Murphy <murp@ibm.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
Quan Tong [Mon, 16 Oct 2023 08:39:48 +0000 (15:39 +0700)]
cmd/go/internal/help: update the documentation to match the design and implementation
The existing documentation imply that the build constraints
should be ignored after a block comments, but actually it's not.
Fixes #63502
Change-Id: I0597934b7a7eeab8908bf06e1312169b3702bf05
Reviewed-on: https://go-review.googlesource.com/c/go/+/535635 Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Mark Pictor <mark.pictor@contrastsecurity.com>
Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
Russ Cox [Tue, 6 Jun 2023 17:50:26 +0000 (13:50 -0400)]
math/rand/v2: delete Mitchell/Reeds source
These slowdowns are because we are now using PCG instead of the
Mitchell/Reeds LFSR for the benchmarks. PCG is in fact a bit slower
(but generates statically far better random numbers).
On the other hand, compared to the original "update benchmarks" CL,
the cleanups we've made more than compensate for PCG being a bit
slower than LFSR, at least on 64-bit x86. ARM64 (Apple M1) is a bit
slower: perhaps the 64x64→128 multiply is slower there for some reason.
386 is noticeably slower, but it's also a non-SSA backend.
Russ Cox [Tue, 6 Jun 2023 17:50:08 +0000 (13:50 -0400)]
math/rand/v2: add PCG-DXSM
For the original math/rand, we ported Plan 9's random number
generator, which was a refinement by Ken Thompson of an algorithm
by Don Mitchell and Jim Reeds, which Mitchell in turn recalls as
having been derived from an algorithm by Marsaglia. At its core,
it is an additive lagged Fibonacci generator (ALFG).
Whatever the details of the history, this generator is nowhere
near the current state of the art for simple, pseudo-random
generators.
This CL adds an implementation of Melissa O'Neill's PCG, specifically
the variant PCG-DXSM, which she defined after writing the PCG paper
and which is now the default in Numpy. The update is slightly slower
(a few multiplies and adds, instead of a few adds), but the state
is dramatically smaller (2 words instead of 607). The statistical
output properties are better too.
A followup CL will delete the old generator.
PCG is the only change here, so no benchmarks should be affected.
Including them anyway as further evidence for caution.
Russ Cox [Tue, 6 Jun 2023 14:51:09 +0000 (10:51 -0400)]
math/rand/v2: simplify Perm
The compiler says Perm is being inlined into BenchmarkPerm,
and yet BenchmarkPerm30ViaShuffle, which you'd think is the
same code, still runs significantly faster.
The benchmarks are mystifying but this is clearly still a step in
the right direction, since BenchmarkPerm30ViaShuffle is still
the fastest and we avoid having two copies of that logic.
Branden Brown [Sat, 5 Aug 2023 13:24:57 +0000 (09:24 -0400)]
math/rand/v2: remove bias in ExpFloat64 and NormFloat64
The original implementation of the ziggurat algorithm was designed for
32-bit random integer inputs. This necessitated reusing some low-order
bits for the slice selection and the random coordinate, which introduces
statistical bias. The result is that PractRand consistently fails the
math/rand normal and exponential sequences (transformed to uniform)
within 2 GB of variates.
This change adjusts the ziggurat procedures to use 63-bit random inputs,
so that there is no need to reuse bits between the slice and coordinate.
This is sufficient for the normal sequence to survive to 256 GB of
PractRand testing.
An alternative technique is to recalculate the ziggurats to use 1024
rather than 128 or 256 slices to make full use of 64-bit inputs. This
improves the survival of the normal sequence to far beyond 256 GB and
additionally provides a 6% performance improvement due to the improved
rejection procedure efficiency. However, doing so increases the total
size of the ziggurat tables from 4.5 kB to 48 kB.
Russ Cox [Tue, 6 Jun 2023 14:49:19 +0000 (10:49 -0400)]
math/rand/v2: optimize Float32, Float64
We realized too late after Go 1 that float64(r.Uint64())/(1<<64)
is not a correct implementation: it occasionally rounds to 1.
The correct implementation is float64(r.Uint64()&(1<<53-1))/(1<<53)
but we couldn't change the implementation for compatibility, so we
changed it to retry only in the "round to 1" cases.
The change to v2 lets us update the algorithm to the simpler,
faster one.
Note that this implementation cannot generate 2⁻⁵⁴, nor 2⁻¹⁰⁰,
nor any of the other numbers between 0 and 2⁻⁵³. A slower algorithm
could shift some of the probability of generating these two boundary
values over to the values in between, but that would be much slower
and not necessarily be better. In particular, the current
implementation has the property that there are uniform gaps between
the possible returned floats, which might help stability. Also, the
result is often scaled and shifted, like Float64()*X+Y. Multiplying by
X>1 would open new gaps, and adding most Y would erase all the
distinctions that were introduced.
The only changes to benchmarks should be in Float32 and Float64.
The other changes remain a cautionary tale.
Russ Cox [Tue, 6 Jun 2023 13:43:20 +0000 (09:43 -0400)]
math/rand/v2: add, optimize N, UintN, Uint32N, Uint64N
Now that we can break the value stream, we can take advantage
of better algorithms that have been suggested since the original
code was written.
Also optimizes IntN, Int32N, Int64N, Perm (indirectly).
All the N variants (IntN, Int32N, Int64N, UintN, N, etc) now
return the same values given a Source and parameter n, so that
for example uint(r.IntN(10)) and r.UintN(10) and r.N(uint(10))
are completely interchangeable.
Int64N4e18 gets slower but that is a near worst case for
the algorithm and is extremely unlikely in practice.
32-bit Int32N variants got slower too, by 15-30%, in exchange
for speeding up everything on 64-bit systems and consistency
across the N functions.
Also rename previously missed benchmark
GlobalInt63Parallel to GlobalInt64Parallel.
Russ Cox [Tue, 6 Jun 2023 13:16:34 +0000 (09:16 -0400)]
math/rand/v2: change Source to use uint64
This should make Uint64-using functions faster and leave
other things alone. It is a mystery why so much got faster.
A good cautionary tale not to read too much into minor
jitter in the benchmarks.
cmd/compile: optimize right shifts of int32 on riscv64
The compiler is currently sign extending 32 bit signed integers to
64 bits before right shifting them using a 64 bit shift instruction.
There's no need to do this as RISC-V has instructions for right
shifting 32 bit signed values (sraw and sraiw) which sign extend
the result of the shift to 64 bits. Change the compiler so that
it uses sraw and sraiw for shifts of signed 32 bit integers reducing
in most cases the number of instructions needed to perform the shift.
Here are some examples of code sequences that are changed by this
patch:
int32(a) >> 2
before:
sll x5,x10,0x20
sra x10,x5,0x22
after:
sraw x10,x10,0x2
int32(v) >> int(s)
before:
sext.w x5,x10
sltiu x6,x11,64
add x6,x6,-1
or x6,x11,x6
sra x10,x5,x6
after:
sltiu x5,x11,32
add x5,x5,-1
or x5,x11,x5
sraw x10,x10,x5
int32(v) >> (int(s) & 31)
before:
sext.w x5,x10
and x6,x11,63
sra x10,x5,x6
after:
and x5,x11,31
sraw x10,x10,x5
int32(100) >> int(a)
before:
bltz x10,<target address calls runtime.panicshift>
sltiu x5,x10,64
add x5,x5,-1
or x5,x10,x5
li x6,100
sra x10,x6,x5
after:
bltz x10,<target address calls runtime.panicshift>
sltiu x5,x10,32
add x5,x5,-1
or x5,x10,x5
li x6,100
sraw x10,x6,x5
int32(v) >> (int(s) & 63)
before:
sext.w x5,x10
and x6,x11,63
sra x10,x5,x6
after:
and x5,x11,63
sltiu x6,x5,32
add x6,x6,-1
or x5,x5,x6
sraw x10,x10,x5
In most cases we eliminate one instruction. In the case where
we shift a int32 constant by a variable the number of instructions
generated is identical. A sra is simply replaced by a sraw. In the
unusual case where we shift right by a variable anded with a constant
> 31 but < 64, we generate two additional instructions. As this is
an unusual case we do not try to optimize for it.
Some improvements can be seen in some of the existing benchmarks,
notably in the utf8 package which performs right shifts of runes
which are signed 32 bit integers.
Change-Id: I60a91cbede9ce65597571c7b7dd9943eeb8d3cc2
Reviewed-on: https://go-review.googlesource.com/c/go/+/535115
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: M Zhuo <mzh@golangcn.org> Reviewed-by: David Chase <drchase@google.com>
Russ Cox [Tue, 6 Jun 2023 16:44:46 +0000 (12:44 -0400)]
math/rand/v2: update benchmarks
Change the benchmarks to use the result of the calls,
as I found that in certain cases inlining resulted in
discarding part of the computation in the benchmark loop.
Add various benchmarks that will be relevant in future CLs.
Russ Cox [Tue, 6 Jun 2023 12:46:45 +0000 (08:46 -0400)]
math/rand/v2: start of new API
This is the beginning of the math/rand/v2 package from proposal #61716.
Start by copying old API. This CL copies math/rand/* to math/rand/v2
and updates references to math/rand to add v2 throughout.
Later CLs will make the v2 changes.
For #61716.
Change-Id: I1624ccffae3dfa442d4ba2461942decbd076e11b
Reviewed-on: https://go-review.googlesource.com/c/go/+/502495 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Cherry Mui [Tue, 17 Oct 2023 02:36:57 +0000 (22:36 -0400)]
cmd/compile: rework TestPGOHash to not rebuild dependencies
TestPGOHash may rebuild dependencies as we pass -trimpath to the
go command. This CL makes it pass -trimpath compiler flag to only
the current package instead, as we only need the current package
to have a stable source file path.
Also refactor buildPGOInliningTest to only take compiler flags,
not go flags, to avoid accidental rebuild.
Should fix #63733.
Change-Id: Iec6c4e90cf659790e21083ee2e697f518234c5b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/535915 Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Bryan Mills <bcmills@google.com>
Cherry Mui [Fri, 27 Oct 2023 16:30:53 +0000 (12:30 -0400)]
internal/testenv: use cmd.Environ in CleanCmdEnv
In CleanCmdEnv, use cmd.Environ instead of os.Environ, so it
sets the PWD environment variable if cmd.Dir is set. This ensures
the child process sees a canonical path for its working directory.
Cherry Mui [Mon, 28 Aug 2023 18:57:29 +0000 (14:57 -0400)]
runtime: print a stack trace at "morestack on g0"
Error like "morestack on g0" is one of the errors that is very
hard to debug, because often it doesn't print a useful stack trace.
The runtime doesn't directly print a stack trace because it is
a bad stack state to call print. Sometimes the SIGABRT may trigger
a traceback, but sometimes not especially in a cgo binary. Even if
it triggers a traceback it often does not include the stack trace
of the bad stack.
This CL makes it explicitly print a stack trace and throw. The
idea is to have some space as an "emergency" crash stack. When the
stack is in a really bad state, we switch to the crash stack and
do a traceback.
Currently only implemented on AMD64 and ARM64.
TODO: also handle errors like "morestack on gsignal" and bad
systemstack. Also handle other architectures.
Change-Id: Ibfc397202f2bb0737c5cbe99f2763de83301c1c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/419435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>