]> Cypherpunks.ru repositories - gostls13.git/log
gostls13.git
13 months agomisc/wasm: default to /tmp if TMPDIR is unset
Bryan C. Mills [Wed, 26 Apr 2023 19:54:28 +0000 (15:54 -0400)]
misc/wasm: default to /tmp if TMPDIR is unset

Change-Id: Ibf460d86ced08687099725bcd8ea8f38d7e8484c
Reviewed-on: https://go-review.googlesource.com/c/go/+/489435
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>

13 months agonet/http/cgi: propagate LD_LIBRARY_PATH on Android
Bryan C. Mills [Wed, 26 Apr 2023 19:38:08 +0000 (15:38 -0400)]
net/http/cgi: propagate LD_LIBRARY_PATH on Android

Android is functionally a variant on linux, and should be
treated as such.

Change-Id: I08056f00bf98c1935c8cc3c859a6c72fe1a48efa
Reviewed-on: https://go-review.googlesource.com/c/go/+/489395
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
13 months agocmd/go: simplify code that still assumed the build cache could be nil
Brad Fitzpatrick [Wed, 26 Apr 2023 17:45:01 +0000 (10:45 -0700)]
cmd/go: simplify code that still assumed the build cache could be nil

cache.Default always returns a non-nil value since Go 1.12; the docs were
updated in https://go.dev/cl/465555.

This updates all the callers of cache.Default that were checking whether
the result was nil so the code isn't misleading/confusing to readers.

Change-Id: Ia63567dd70affef6041c744259f65cea79a2752e
Reviewed-on: https://go-review.googlesource.com/c/go/+/489355
Auto-Submit: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
13 months agoruntime/cgo: store M for C-created thread in pthread key
Michael Pratt [Mon, 17 Apr 2023 18:51:28 +0000 (14:51 -0400)]
runtime/cgo: store M for C-created thread in pthread key

This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and
CL 485316 incorporated.

CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 482975 is a followup fix to a C declaration in testprogcgo.
CL 485315 is a followup fix for x_cgo_getstackbound on Illumos.
CL 485316 is a followup cleanup for ppc64 assembly.

[Original CL 481061 description]

This reapplies CL 392854, with the followup fixes in CL 479255,
CL 479915, and CL 481057 incorporated.

CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 479255 is a followup fix for a small bug in ARM assembly code.
CL 479915 is another followup fix to address C to Go calls after
the C code uses some stack, but that CL is also buggy.
CL 481057, by Michael Knyszek, is a followup fix for a memory leak
bug of CL 479915.

[Original CL 392854 description]

In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.

When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key,  only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.

When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.

This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.

This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.

For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz

[CL 479915 description]

Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.

This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.

[CL 485500 description]

CL 479915 passed the G to _cgo_getstackbound for direct updates to
gp.stack.lo. A G can be reused on a new thread after the previous thread
exited. This could trigger the C TSAN race detector because it couldn't
see the synchronization in Go (lockextra) preventing the same G from
being used on multiple threads at the same time.

We work around this by passing the address of a stack variable to
_cgo_getstackbound rather than the G. The stack is generally unique per
thread, so TSAN won't see the same address from multiple threads. Even
if stacks are reused across threads by pthread, C TSAN should see the
synchonization in the stack allocator.

A regression test is added to misc/cgo/testsanitizer.

Fixes #51676.
Fixes #59294.
Fixes #59678.

Change-Id: Ic62be31a06ee83568215e875a891df37084e08ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/485500
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>

13 months agocmd/link/internal/loadelf: set AttrExternal on text section symbols
Paul E. Murphy [Tue, 25 Apr 2023 21:41:51 +0000 (16:41 -0500)]
cmd/link/internal/loadelf: set AttrExternal on text section symbols

PPC64 processes external object relocations against the section
symbols. This needs to be set correctly to determine the type of
PLT stub to generate when both Go and External code make PLT calls.

Change-Id: I5abdd5a0473866164083c33e80324dffcc1707f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/488895
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agocoverage: fix count vs emit discrepancy in coverage counter data writing
Than McIntosh [Thu, 13 Apr 2023 18:27:33 +0000 (14:27 -0400)]
coverage: fix count vs emit discrepancy in coverage counter data writing

This patch revises the way coverage counter data writing takes place
to avoid problems where useful counter data (for user-written functions)
is skipped in favor of counter data from stdlib functions that are
executed "late in the game", during the counter writing process itself.

Reading counter values from a running "--coverpkg=all" program is an
inherently racy operation; while the the code that scans the coverage
counter segment is reading values, the program is still executing,
potentially updating those values, and updates can include execution
of previously un-executed functions. The existing counter data writing
code was using a two-pass model (initial sweep over the counter
segment to count live functions, second sweep to actually write data),
and wasn't properly accounting for the fact that the second pass could
see more functions than the first.

In the bug in question, the first pass discovered an initial set of
1240 functions, but by the time the second pass kicked in, several
additional new functions were also live. The second pass scanned the
counter segment again to write out exactly 1240 functions, but since
some of the counters for the newly executed functions were earlier in
the segment (due to linker layout quirks) than the user's selected
function, the sweep terminated before writing out counters for the
function of interest.

The fix rewrites the counter data file encoder to make a single sweep
over the counter segment instead of using a two-pass scheme.

Fixes #59563.

Change-Id: I5e908e226bb224adb90a2fb783013e52deb341da
Reviewed-on: https://go-review.googlesource.com/c/go/+/484535
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>

13 months agointernal/coverage/slicewriter: fix off-by-1 error in seek utilities
Than McIntosh [Mon, 24 Apr 2023 14:33:01 +0000 (10:33 -0400)]
internal/coverage/slicewriter: fix off-by-1 error in seek utilities

The slicewriter Seek method was being too restrictive on offsets
accepted, due to an off-by-one problem in the error checking code.
This fixes the problem and touches up the unit tests.

Change-Id: I75d6121551de19ec9275f0e331810db231db6ea9
Reviewed-on: https://go-review.googlesource.com/c/go/+/488116
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agoruntime: add a alignment check
ioworker0 [Mon, 24 Apr 2023 06:37:52 +0000 (06:37 +0000)]
runtime: add a alignment check

The Linux implementation requires that the address addr be
page-aligned, and allows length to be zero.

See Linux notes:
https://man7.org/linux/man-pages/man2/madvise.2.html

Change-Id: Ic49960c32991ef12f23de2de76e9689567c82d03
GitHub-Last-Rev: 35e7f8e5cc0b045043a88d9f304ef5bb1e9c1ab2
GitHub-Pull-Request: golang/go#59793
Reviewed-on: https://go-review.googlesource.com/c/go/+/488015
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>

13 months agocrypto/sha512: add WriteString and WriteByte method
Ian Lance Taylor [Wed, 12 Apr 2023 01:55:23 +0000 (18:55 -0700)]
crypto/sha512: add WriteString and WriteByte method

This can reduce allocations when hashing a string or byte
rather than []byte.

For #38776

Change-Id: I4926ae2749f6b167edbebb73d8f68763ffb2f0c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/483816
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Auto-Submit: Ian Lance Taylor <iant@google.com>

13 months agocrypto/sha1: add WriteString and WriteByte method
Ian Lance Taylor [Tue, 11 Apr 2023 22:28:23 +0000 (15:28 -0700)]
crypto/sha1: add WriteString and WriteByte method

This can reduce allocations when hashing a string or byte
rather than []byte.

For #38776

Change-Id: I7c1fbdf15abf79d2faf360f75adf4bc550a607e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/483815
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
13 months agocrypto/sha256: add WriteString and WriteByte method
Ian Lance Taylor [Mon, 3 Apr 2023 04:21:24 +0000 (21:21 -0700)]
crypto/sha256: add WriteString and WriteByte method

This can reduce allocations when hashing a string or byte
rather than []byte.

For #38776

Change-Id: I1c6dd1bc018220784a05939e92b47558c0562110
Reviewed-on: https://go-review.googlesource.com/c/go/+/481478
Reviewed-by: Joel Sing <joel@sing.id.au>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>

13 months agocmd/dist: actually only compile tests with -compile-only
Austin Clements [Mon, 24 Apr 2023 18:26:39 +0000 (14:26 -0400)]
cmd/dist: actually only compile tests with -compile-only

Currently, "dist test -compile-only" still runs the test binaries,
just with -run=^$ so no tests are run. It does this because, until
recently, "go test -c" would fail if passed multiple test packages.
But this has some unexpected consequences: init code still runs,
TestMain still runs, and we generally can't test cross-compiling of
tests.

Now that #15513 is fixed, we can pass multiple packages to "go test
-c". Hence, this CL make dist just use "go test -c" as one would
expect.

Found in the course of working on #37486, though it doesn't really
affect that.

Change-Id: If7d3c72c9e0f74d4ea0dd422411e5ee93b314be4
Reviewed-on: https://go-review.googlesource.com/c/go/+/488275
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>

13 months agoruntime: fallback to TEB arbitrary pointer when TLS slots are full
qmuntal [Thu, 20 Apr 2023 15:30:38 +0000 (17:30 +0200)]
runtime: fallback to TEB arbitrary pointer when TLS slots are full

The Go runtime allocates the TLS slot in the TEB TLS slots instead of
using the TEB arbitrary pointer. See CL 431775 for more context.

The problem is that the TEB TLS slots array only has capacity for 64
indices, allocating more requires some complex logic that we don't
support yet.

Although the Go runtime only allocates one index, a Go DLL can be
loaded in a process with more than 64 TLS slots allocated,
in which case it abort.

This CL avoids aborting by falling back to the older behavior, that
is to use the TEB arbitrary pointer.

Fixes #59213

Change-Id: I39c73286fe2da95aa9c5ec5657ee0979ecbec533
Reviewed-on: https://go-review.googlesource.com/c/go/+/486816
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agoruntime/metrics: specify that bucket counts increase monotonically for histogram...
Nayef Ghattas [Fri, 21 Apr 2023 12:12:07 +0000 (14:12 +0200)]
runtime/metrics: specify that bucket counts increase monotonically for histogram metrics

Make it explicit in the documentation that the histogram metrics
are cumulative (i.e. each bucket count increases monotonically).

Change-Id: I89119ba816ac46a63f36e607e695fad3695057ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/487315
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agointernal/cpu: add a detection for Neoverse(N2, V2) cores
fanzha02 [Thu, 20 Apr 2023 07:28:50 +0000 (07:28 +0000)]
internal/cpu: add a detection for Neoverse(N2, V2) cores

The memmove implementation relies on the variable
runtime.arm64UseAlignedLoads to select fastest code
path. Considering Neoverse N2 and V2 cores prefer aligned
loads, this patch adds code to detect them for
memmove performance.

And this patch uses a new variable ARM64.IsNeoverse to
represent all Neoverse cores, removing the more specific
versions.

Change-Id: I9e06eae01a0325a0b604ac6af1e55711dd6133f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/487815
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Fannie Zhang <Fannie.Zhang@arm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agogo/ast: add IsGenerated(*File) predicate
Alan Donovan [Sun, 23 Apr 2023 17:45:08 +0000 (13:45 -0400)]
go/ast: add IsGenerated(*File) predicate

See https://go.dev/s/generatedcode for spec.

Fixes #28089

Change-Id: Ic9bb138bdd180f136f9e8e74e187319acca5dbac
Reviewed-on: https://go-review.googlesource.com/c/go/+/487935
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
13 months agohtml/template: fix unavailable url
cui fliter [Sun, 23 Apr 2023 10:44:05 +0000 (18:44 +0800)]
html/template: fix unavailable url

The previous link is no longer accessible. use latest link.

Change-Id: I76411ee00785f3d92014c5012e4efb446924adaf
Reviewed-on: https://go-review.googlesource.com/c/go/+/487835
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Andrew Polukhin <andrewmathematics2003@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: shuang cui <imcusg@gmail.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>

13 months agoruntime: add raceFiniLock to lock ranking
Ian Lance Taylor [Sun, 23 Apr 2023 19:25:47 +0000 (12:25 -0700)]
runtime: add raceFiniLock to lock ranking

Also preserve the PC/SP in reentersyscall when doing lock ranking.
The test is TestDestructorCallbackRace with the staticlockranking
experiment enabled.

For #59711

Change-Id: I87ac1d121ec0d399de369666834891ab9e7d11b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/487955
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>

13 months agointernal/testenv: actually try to exec on ios and wasm
Bryan C. Mills [Mon, 24 Apr 2023 14:09:17 +0000 (10:09 -0400)]
internal/testenv: actually try to exec on ios and wasm

Due to a stray edit in CL 486275, the assignment to tryExecOk
in tryExec on ios would be immediately overwritten back to false.
This change fixes the stray edit.

Change-Id: I4f45fbf130dc912305e5f453b0d1a622ba199ad4
Reviewed-on: https://go-review.googlesource.com/c/go/+/488076
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>

13 months agoarchive/zip: reject overflowing directorySize & directoryOffset
Roland Shoemaker [Mon, 24 Apr 2023 16:31:06 +0000 (09:31 -0700)]
archive/zip: reject overflowing directorySize & directoryOffset

We added a check for incorrect baseOffset in CL 408734, but in doing so
we introduced a panic when directoryOffset overflowed a int64. The zip
spec uses uint64, but since io.SectionReader requires int64 we convert,
and possibly introduce an overflow. If offset < 0 && size-offset < 0,
SectionReader will panic when we attempt to read from it.

Since it's extremely unlikely we're ever going to process a zip file
larger than 1<<63-1 byte, just limit directory size and offset to the
max int64.

Change-Id: I1aaa755cf4da927a6e12ef59f97dfc83a3426d86
Reviewed-on: https://go-review.googlesource.com/c/go/+/488195
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>

13 months agodoc: fix typo in Go 1.21 release notes
Sameer Ajmani [Mon, 24 Apr 2023 13:34:24 +0000 (09:34 -0400)]
doc: fix typo in Go 1.21 release notes

Change-Id: Ib32567fdd12079cd171a4e1bc118ce27d8ce2a5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/488035
Run-TryBot: Sameer Ajmani <sameer@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agoruntime: fix CgoRaceprof and CgoRaceSignal build failures
Bryan C. Mills [Mon, 24 Apr 2023 14:18:07 +0000 (10:18 -0400)]
runtime: fix CgoRaceprof and CgoRaceSignal build failures

TestRaceProf and TestRaceSignal were changed to run on all platforms
that support the race detector as of CL 487575, but the testprogcgo
source files needed to run the test rely on POSIX threads and were
still build-constrained to only linux/amd64 and freebsd/amd64.

Since the C test program appears to require only POSIX APIs, update
the constraint to build the source file on all Unix platforms, and
update the tests to skip on Windows.

This may slightly increase testprogcgo build time on Unix platforms
that do not support the race detector.

Change-Id: I704dd496d475a3cd2e2da2a09c7d2e3bb8e96d02
Reviewed-on: https://go-review.googlesource.com/c/go/+/488115
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>

13 months agotesting/slogtest: tests for slog handlers
Jonathan Amsterdam [Sun, 23 Apr 2023 15:23:53 +0000 (11:23 -0400)]
testing/slogtest: tests for slog handlers

Add a package for testing that a slog.Handler implementation
satisfies that interface's documented requirements.

Code copied from x/exp/slog/slogtest.

Updates #56345.

Change-Id: I89e94d93bfbe58e3c524758f7ac3c3fba2a2ea96
Reviewed-on: https://go-review.googlesource.com/c/go/+/487895
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
13 months agocmd/link: don't sort data symbols by name
Cherry Mui [Fri, 21 Apr 2023 20:28:57 +0000 (16:28 -0400)]
cmd/link: don't sort data symbols by name

For data symbols, we currently sort them by size, then by name if
the size is the same. Sorting by name is not really necessary.
Instead, we sort by symbol index. Like name, the symbol index is
deterministic, and pretty stable if only a small portion of the
input is changed, and also naturally partitioned by packages. This
reduces the CPU time for reading the symbol names and comparing
strings.

Linking cmd/compile (on macOS/amd64),

Dodata    57.2ms ± 6%    54.5ms ± 4%   -4.74%  (p=0.000 n=19+17)

Change-Id: I1c4f2b83dbbb4b984b2c8ab4a7e8543b9f7f22b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/487515
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agocmd/link: use uint32 as symbol index
Cherry Mui [Fri, 21 Apr 2023 16:02:30 +0000 (12:02 -0400)]
cmd/link: use uint32 as symbol index

Currently, a symbol's global index, the Sym type, is defined as an
int, which is 64-bit on 64-bit machines. We're unlikely to have
more than 4 billion symbols in the near future. Even if we will,
we will probably hit some other limit (e.g. section size) before
the symbol number limit. Use a 32-bit type to reduce memory usage.

E,g, linking cmd/compile in external linking mode (on macOS/amd64)

Munmap_GC    43.2M ± 0%     35.5M ± 1%   -17.74%  (p=0.000 n=16+20)

This brings the memory usage back before the previous CL, and even
lower.

Change-Id: Ie185f1586638fe70d8121312bfa9410942d518c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/487416
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
13 months agocmd/go: declare net hosts in script tests
Bryan C. Mills [Fri, 3 Mar 2023 14:48:10 +0000 (14:48 +0000)]
cmd/go: declare net hosts in script tests

Although we aren't precise about enforcing the hosts just yet,
we can eventually use the declared hostnames to selectively skip
tests (for example, if an external service has an outage while
a Go release is being tested).

Also relax the constraint to [short] in tests that require only
vcs-test.golang.org, which has redirected to an in-process server
since around CL 427914.

Also enforce that tests that use the network actually use the [net]
constraint, by setting TESTGONETWORK=panic in the test environment
until the condition is evaluated.

For #52545.
For #54503.
Updates #27494.

Change-Id: I13be6b42a9beee97657eb45424882e787ac164c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/473276
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Bypass: Bryan Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>

13 months agoruntime/metrics: set /sched/latencies:seconds as cumulative
Nayef Ghattas [Wed, 12 Apr 2023 09:09:01 +0000 (11:09 +0200)]
runtime/metrics: set /sched/latencies:seconds as cumulative

The current implementation for this metric populates a histogram
that is never reset, i.e. where each bucket count increases
monotonically.

The comment in the definition of the Cumulative attribute calls
out that cumulative means that if the metric is a distribution,
then each bucket count increases monotonically.

In that sense, the cumulative attribute should be set to true for
this metric.

Change-Id: Ifc34e965a62f2d7881b5c8e8cbb8b7207a4d5757
Reviewed-on: https://go-review.googlesource.com/c/go/+/486755
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
13 months agocmd/link: use slice and bitmap for some attributes
Cherry Mui [Fri, 21 Apr 2023 03:33:42 +0000 (23:33 -0400)]
cmd/link: use slice and bitmap for some attributes

Currently, a symbol's outer symbol, the "special" attribute, and
whether a symbol is a generator symbol are represented as maps,
and are accessed in some loops over nearly all reachable symbols.
The map lookups are a bit expensive.

For outer symbol, a non-trivial portion of the symbols have outer
symbol set (e.g. type symbols, which we put into container symbols
like "type:*"). Using a slice to access more efficiently.

For the special and generator symbol attributes, use a bitmap.
There are not many symbols have those attributes, so the bitmap is
quite sparse. The bitmap is not too large anyway, so use it for
now. If we want to further reduce memory usage we could consider
some other data structure like a Bloom filter.

Linking cmd/compile in external linking mode (on macOS/amd64)

Symtab   12.9ms ± 9%     6.4ms ± 5%   -50.08%  (p=0.000 n=19+18)
Dodata   64.9ms ±12%    57.1ms ±12%   -11.90%  (p=0.000 n=20+20)
Asmb     36.7ms ±11%    32.8ms ± 9%   -10.61%  (p=0.000 n=20+18)
Asmb2    26.6ms ±15%    21.9ms ±12%   -17.75%  (p=0.000 n=20+18)

There is some increase of memory usage

Munmap_GC   40.9M ± 1%     43.2M ± 0%    +5.54%  (p=0.000 n=20+19)

The next CL will bring the memory usage back.

Change-Id: Ie4347eb96c51f008b9284270de37fc880bb52d2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/487415
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
13 months agocrypto/tls: fix PSK binder calculation
Tero Saarni [Wed, 12 Apr 2023 10:07:07 +0000 (10:07 +0000)]
crypto/tls: fix PSK binder calculation

When server and client have mismatch in curve preference, the server will
send HelloRetryRequest during TLSv1.3 PSK resumption. There was a bug
introduced by Go1.19.6 or later and Go1.20.1 or later, that makes the client
calculate the PSK binder hash incorrectly. Server will reject the TLS
handshake by sending alert: invalid PSK binder.

Fixes #59424

Change-Id: I2ca8948474275740a36d991c057b62a13392dbb9
GitHub-Last-Rev: 1aad9bcf27f563449c1a7ed6d0dd1d247cc65713
GitHub-Pull-Request: golang/go#59425
Reviewed-on: https://go-review.googlesource.com/c/go/+/481955
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>

13 months agoruntime: call _exit, not exit, on AIX and Solaris
Ian Lance Taylor [Sat, 22 Apr 2023 01:48:38 +0000 (18:48 -0700)]
runtime: call _exit, not exit, on AIX and Solaris

This is the AIX and Solaris equivalent of CL 269378.

On AIX and Solaris, where we use libc for syscalls, when the runtime exits,
it calls the libc exit function, which may call back into user code,
such as invoking functions registered with atexit. In particular, it
may call back into Go. But at this point, the Go runtime is
already exiting, so this wouldn't work.

On non-libc platforms we use exit syscall directly, which doesn't
invoke any callbacks. Use _exit on AIX and Solaris to achieve the same
behavior.

Test is TestDestructorCallback.

For #59711

Change-Id: I666f75538d3e3d8cf3b697b4c32f3ecde8332890
Reviewed-on: https://go-review.googlesource.com/c/go/+/487635
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>

13 months agoruntime: use platform.RaceDetectorSupported for -race tests
Ian Lance Taylor [Fri, 21 Apr 2023 22:28:19 +0000 (15:28 -0700)]
runtime: use platform.RaceDetectorSupported for -race tests

Don't try to duplicate the list of targets that support -race.

Change-Id: I889d5c2f4884de89d88f8efdc89608aa73584a8a
Reviewed-on: https://go-review.googlesource.com/c/go/+/487575
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>

13 months agointernal/testenv: in HasExec, try to actually exec on ios and wasm platforms
Bryan C. Mills [Wed, 19 Apr 2023 14:46:26 +0000 (10:46 -0400)]
internal/testenv: in HasExec, try to actually exec on ios and wasm platforms

Some iOS environments may support exec. wasip1 and js do not, but
trying to exec on those platforms is inexpensive anyway and gives
better test coverage for the ios path.

Change-Id: I4baffb2ef5dc7d81e6a260f69033bfb229f13d92
Reviewed-on: https://go-review.googlesource.com/c/go/+/486275
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>

13 months agocmd/compile: support -buildmode=c-shared on linux/mips64{,le}
Xiaodong Liu [Thu, 9 Sep 2021 08:07:38 +0000 (16:07 +0800)]
cmd/compile: support -buildmode=c-shared on linux/mips64{,le}

The modification of these rules is optimization to load/store global
variables. If there are a sequence of loads/stores nearby a global
variable address, the address can only be loaded from GOT once instead
of every time.

For #43264

Change-Id: Idedaf6c81f085955371320f51bca148ffb42a2d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/348732
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>

13 months agoflag: panic if a flag is defined after being set
Keith Randall [Wed, 29 Mar 2023 16:12:20 +0000 (09:12 -0700)]
flag: panic if a flag is defined after being set

As part of developing #57411, we ran into cases where a flag was
defined in one package init and Set in another package init, and there
was no init ordering implied by the spec between those two
packages. Changes in initialization ordering as part of #57411 caused
a Set to happen before the definition, which makes the Set silently
fail.

This CL makes the Set fail loudly in that situation.

Currently Set *does* fail kinda quietly in that situation, in that it
returns an error. (It seems that no one checks the error from Set,
at least for string flags.) Ian suggsted that instead we panic at
the definition site if there was previously a Set called on that
(at the time undefined) flag.

So Set on an undefined flag is ok and returns an error (as before),
but defining a flag which has already been Set causes a panic.  (The
API for flag definition has no way to return an error, and does
already panic in some situations like a duplicate definition.)

Update #57411

Change-Id: I39b5a49006f9469de0b7f3fe092afe3a352e4fcb
Reviewed-on: https://go-review.googlesource.com/c/go/+/480215
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
13 months agomisc/wasm: support wasmtime in wasip1
Johan Brandhorst-Satzkorn [Tue, 18 Apr 2023 05:13:52 +0000 (22:13 -0700)]
misc/wasm: support wasmtime in wasip1

Allow switching to wasmtime through the GOWASIRUNTIME variable. This
will allow builders to run the wasip1 standard library tests against
the wasmtime WASI runtime.

For #59583

Change-Id: I4d5200df7bb27b66e041f00e89d4c2e585f5da7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/485615
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Bypass: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
13 months agoruntime: in __tsan_fini tell scheduler we are entering non-Go code
Ian Lance Taylor [Wed, 19 Apr 2023 21:16:37 +0000 (14:16 -0700)]
runtime: in __tsan_fini tell scheduler we are entering non-Go code

__tsan_fini will call exit which will call destructors which
may in principle call back into Go functions. Prepare the scheduler
by calling entersyscall before __tsan_fini.

Fixes #59711

Change-Id: Ic4df8fba3014bafa516739408ccfc30aba4f22ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/486615
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>

13 months agocmd/compile: introduce separate memory op combining pass
Keith Randall [Tue, 21 Mar 2023 16:25:43 +0000 (09:25 -0700)]
cmd/compile: introduce separate memory op combining pass

Memory op combining is currently done using arch-specific rewrite rules.
Instead, do them as a arch-independent rewrite pass. This ensures that
all architectures (with unaligned loads & stores) get equal treatment.

This removes a lot of rewrite rules.

The new pass is a bit more comprehensive. It handles things like out-of-order
writes and is careful not to apply partial optimizations that then block
further optimizations.

Change-Id: I780ff3bb052475cd725a923309616882d25b8d9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/478475
Reviewed-by: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
13 months agoruntime: skip TestG0StackOverflow on ios
Bryan C. Mills [Fri, 21 Apr 2023 14:47:52 +0000 (10:47 -0400)]
runtime: skip TestG0StackOverflow on ios

This test fails when run on ios. (Although ios does not normally
support "exec", in the corellium environment it does.)

For #26061.

Change-Id: Idfdc53758aaabf0cb87ae50f9a4666deebf57fd6
Reviewed-on: https://go-review.googlesource.com/c/go/+/487355
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>

13 months agoruntime: prevent double lock in checkdead by unlocking before throws
Lucien Coffe [Fri, 21 Apr 2023 15:20:19 +0000 (17:20 +0200)]
runtime: prevent double lock in checkdead by unlocking before throws

This change resolves an issue where checkdead could result in a double lock when shedtrace is enabled. This fix involves adding unlocks before all throws in the checkdead function to ensure the scheduler lock is properly released.

Fixes #59758

Change-Id: If3ddf9969f4582c3c88dee9b9ecc355a63958103
Reviewed-on: https://go-review.googlesource.com/c/go/+/487375
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
13 months agolog: avoid leaking goroutines in TestOutputRace
Bryan C. Mills [Fri, 21 Apr 2023 15:42:20 +0000 (11:42 -0400)]
log: avoid leaking goroutines in TestOutputRace

Leaked goroutines are the only explanation I can think of for excess
allocs in TestDiscard, and TestOutputRace is the only place I can see
where the log package leaks goroutines. Let's fix that leak and see if
it eliminates the TestDiscard flakes.

Fixes #58797 (maybe).

Change-Id: I2d54dcba3eb52bd10a62cd1c380131add6a2f651
Reviewed-on: https://go-review.googlesource.com/c/go/+/487356
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
13 months agoruntime: tidy _Stack* constant naming
Austin Clements [Wed, 19 Apr 2023 18:58:47 +0000 (14:58 -0400)]
runtime: tidy _Stack* constant naming

For #59670.

Change-Id: I0efa743edc08e48dc8d906803ba45e9f641369db
Reviewed-on: https://go-review.googlesource.com/c/go/+/486977
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>

13 months agoruntime, cmd: rationalize StackLimit and StackGuard
Austin Clements [Wed, 19 Apr 2023 18:01:05 +0000 (14:01 -0400)]
runtime, cmd: rationalize StackLimit and StackGuard

The current definitions of StackLimit and StackGuard only indirectly
specify the NOSPLIT stack limit and duplicate a literal constant
(928). Currently, they define the stack guard delta, and from there
compute the NOSPLIT limit.

Rationalize these by defining a new constant, abi.StackNosplitBase,
which consolidates and directly specifies the NOSPLIT stack limit (in
the default case). From this we then compute the stack guard delta,
inverting the relationship between these two constants. While we're
here, we rename StackLimit to StackNosplit to make it clearer what's
being limited.

This change does not affect the values of these constants in the
default configuration. It does slightly change how
StackGuardMultiplier values other than 1 affect the constants, but
this multiplier is a pretty rough heuristic anyway.

                    before after
stackNosplit           800   800
_StackGuard            928   928
stackNosplit -race    1728  1600
_StackGuard -race     1856  1728

For #59670.

Change-Id: Ia94094c5e47897e7c088d24b4a5e33f5c2768db5
Reviewed-on: https://go-review.googlesource.com/c/go/+/486976
Auto-Submit: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agointernal/abi, runtime, cmd: merge StackSmall, StackBig consts into internal/abi
Austin Clements [Wed, 19 Apr 2023 17:21:02 +0000 (13:21 -0400)]
internal/abi, runtime, cmd: merge StackSmall, StackBig consts into internal/abi

For #59670.

Change-Id: I91448363be2fc678964ce119d85cd5fae34a14da
Reviewed-on: https://go-review.googlesource.com/c/go/+/486975
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Auto-Submit: Austin Clements <austin@google.com>

13 months agointernal/abi, runtime, cmd: merge PCDATA_* and FUNCDATA_* consts into internal/abi
Austin Clements [Mon, 17 Apr 2023 20:50:00 +0000 (16:50 -0400)]
internal/abi, runtime, cmd: merge PCDATA_* and FUNCDATA_* consts into internal/abi

We also rename the constants related to unsafe-points: currently, they
follow the same naming scheme as the PCDATA table indexes, but are not
PCDATA table indexes.

For #59670.

Change-Id: I06529fecfae535be5fe7d9ac56c886b9106c74fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/485497
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
13 months agointernal/abi, runtime, cmd: merge funcFlag_* consts into internal/abi
Austin Clements [Mon, 17 Apr 2023 19:58:47 +0000 (15:58 -0400)]
internal/abi, runtime, cmd: merge funcFlag_* consts into internal/abi

For #59670.

Change-Id: Ie784ba4dd2701e4f455e1abde4a6bfebee4b1387
Reviewed-on: https://go-review.googlesource.com/c/go/+/485496
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Auto-Submit: Austin Clements <austin@google.com>

13 months agointernal/abi, runtime, cmd: merge funcID_* consts into internal/abi
Austin Clements [Mon, 17 Apr 2023 19:43:29 +0000 (15:43 -0400)]
internal/abi, runtime, cmd: merge funcID_* consts into internal/abi

For #59670.

Change-Id: I517e97ea74cf232e5cfbb77b127fa8804f74d84b
Reviewed-on: https://go-review.googlesource.com/c/go/+/485495
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>

13 months agocmd/go: assert on more of the version string in TestScript/gotoolchain
Bryan C. Mills [Wed, 19 Apr 2023 19:39:01 +0000 (15:39 -0400)]
cmd/go: assert on more of the version string in TestScript/gotoolchain

The previous assert triggers whenever the 40-character git commit
contains the substring "999", which happens with a probability
decidedly greater than zero.

For #57001.

Change-Id: If0f1bc1a3dd0e6b7e66768d0cf3a79545ee4e5ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/486399
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agocmd/go/internal/vcweb: replace ioutil with os and io
Oleksandr Redko [Mon, 17 Apr 2023 09:39:37 +0000 (12:39 +0300)]
cmd/go/internal/vcweb: replace ioutil with os and io

Change-Id: I251788cbbb6d740ef24e7561cc4bee880b7bdff8
Reviewed-on: https://go-review.googlesource.com/c/go/+/485017
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agocmd/internal/obj/ppc64: modify PCALIGN to ensure alignment
Lynn Boger [Mon, 17 Apr 2023 15:02:48 +0000 (10:02 -0500)]
cmd/internal/obj/ppc64: modify PCALIGN to ensure alignment

The initial purpose of PCALIGN was to identify code
where it would be beneficial to align code for performance,
but avoid cases where too many NOPs were added. On p10, it
is now necessary to enforce a certain alignment in some
cases, so the behavior of PCALIGN needs to be slightly
different.  Code will now be aligned to the value specified
on the PCALIGN instruction regardless of number of NOPs added,
which is more intuitive and consistent with power assembler
alignment directives.

This also adds 64 as a possible alignment value.

The existing values used in PCALIGN were modified according to
the new behavior.

A testcase was updated and performance testing was done to
verify that this does not adversely affect performance.

Change-Id: Iad1cf5ff112e5bfc0514f0805be90e24095e932b
Reviewed-on: https://go-review.googlesource.com/c/go/+/485056
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Archana Ravindar <aravind5@in.ibm.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Paul Murphy <murp@ibm.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
13 months agocmd/link/internal/ppc64: Use PCrel relocs in runtime.addmoduledata if supported
Paul E. Murphy [Thu, 12 Jan 2023 21:57:51 +0000 (15:57 -0600)]
cmd/link/internal/ppc64: Use PCrel relocs in runtime.addmoduledata if supported

This is another step towards supporting TOC-free operations.

Change-Id: I77edcf066c757b8ec815c701d7f6d72cd645eca9
Reviewed-on: https://go-review.googlesource.com/c/go/+/483437
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
13 months agointernal/bytealg: rewrite indexbytebody on PPC64
Paul E. Murphy [Mon, 6 Mar 2023 22:51:31 +0000 (16:51 -0600)]
internal/bytealg: rewrite indexbytebody on PPC64

Use P8 instructions throughout to be backwards compatible, but
otherwise not impede performance. Use overlapping loads where
possible, and prioritize larger checks over smaller check.

However, some newer instructions can be used surgically when
targeting a newer GOPPC64. These can lead to noticeable
performance improvements with minimal impact to readability.

All tests run below on a Power10/ppc64le, and use a small
modification to BenchmarkIndexByte to ensure the IndexByte
wrapper call is inlined (as it likely is under realistic usage).
This wrapper adds substantial overhead if not inlined.

Previous (power9 path, GOPPC64=power8) vs. GOPPC64=power8:

IndexByte/1       3.81ns ± 8%     3.11ns ± 5%  -18.39%
IndexByte/2       3.82ns ± 3%     3.20ns ± 6%  -16.23%
IndexByte/3       3.61ns ± 4%     3.25ns ± 6%  -10.13%
IndexByte/4       3.66ns ± 5%     3.08ns ± 1%  -15.91%
IndexByte/5       3.82ns ± 0%     3.75ns ± 2%   -1.94%
IndexByte/6       3.83ns ± 0%     3.87ns ± 4%   +1.04%
IndexByte/7       3.83ns ± 0%     3.82ns ± 0%   -0.27%
IndexByte/8       3.82ns ± 0%     2.92ns ±11%  -23.70%
IndexByte/9       3.70ns ± 2%     3.08ns ± 2%  -16.87%
IndexByte/10      3.74ns ± 2%     3.04ns ± 0%  -18.75%
IndexByte/11      3.75ns ± 0%     3.31ns ± 8%  -11.79%
IndexByte/12      3.74ns ± 0%     3.04ns ± 0%  -18.86%
IndexByte/13      3.83ns ± 4%     3.04ns ± 0%  -20.64%
IndexByte/14      3.80ns ± 1%     3.30ns ± 8%  -13.18%
IndexByte/15      3.77ns ± 1%     3.04ns ± 0%  -19.33%
IndexByte/16      3.81ns ± 0%     2.78ns ± 7%  -26.88%
IndexByte/17      4.12ns ± 0%     3.04ns ± 1%  -26.11%
IndexByte/18      4.27ns ± 6%     3.05ns ± 0%  -28.64%
IndexByte/19      4.30ns ± 4%     3.02ns ± 2%  -29.65%
IndexByte/20      4.43ns ± 7%     3.45ns ± 7%  -22.15%
IndexByte/21      4.12ns ± 0%     3.03ns ± 1%  -26.35%
IndexByte/22      4.40ns ± 6%     3.05ns ± 0%  -30.82%
IndexByte/23      4.40ns ± 6%     3.01ns ± 2%  -31.48%
IndexByte/24      4.32ns ± 5%     3.07ns ± 0%  -28.98%
IndexByte/25      4.76ns ± 2%     3.04ns ± 1%  -36.11%
IndexByte/26      4.82ns ± 0%     3.05ns ± 0%  -36.66%
IndexByte/27      4.82ns ± 0%     2.97ns ± 3%  -38.39%
IndexByte/28      4.82ns ± 0%     2.96ns ± 3%  -38.57%
IndexByte/29      4.82ns ± 0%     3.34ns ± 9%  -30.71%
IndexByte/30      4.82ns ± 0%     3.05ns ± 0%  -36.77%
IndexByte/31      4.81ns ± 0%     3.05ns ± 0%  -36.70%
IndexByte/32      3.52ns ± 0%     3.44ns ± 1%   -2.15%
IndexByte/33      4.77ns ± 1%     3.35ns ± 0%  -29.81%
IndexByte/34      5.01ns ± 5%     3.35ns ± 0%  -33.15%
IndexByte/35      4.92ns ± 9%     3.35ns ± 0%  -31.89%
IndexByte/36      4.81ns ± 5%     3.35ns ± 0%  -30.37%
IndexByte/37      4.99ns ± 6%     3.35ns ± 0%  -32.86%
IndexByte/38      5.06ns ± 5%     3.35ns ± 0%  -33.84%
IndexByte/39      5.02ns ± 5%     3.48ns ± 9%  -30.58%
IndexByte/40      5.21ns ± 9%     3.55ns ± 4%  -31.82%
IndexByte/41      5.18ns ± 0%     3.42ns ± 2%  -33.98%
IndexByte/42      5.19ns ± 0%     3.55ns ±11%  -31.56%
IndexByte/43      5.18ns ± 0%     3.45ns ± 5%  -33.46%
IndexByte/44      5.18ns ± 0%     3.39ns ± 0%  -34.56%
IndexByte/45      5.18ns ± 0%     3.43ns ± 4%  -33.74%
IndexByte/46      5.18ns ± 0%     3.47ns ± 1%  -33.03%
IndexByte/47      5.18ns ± 0%     3.44ns ± 2%  -33.54%
IndexByte/48      5.18ns ± 0%     3.39ns ± 0%  -34.52%
IndexByte/49      5.69ns ± 0%     3.79ns ± 0%  -33.45%
IndexByte/50      5.70ns ± 0%     3.70ns ± 3%  -34.98%
IndexByte/51      5.70ns ± 0%     3.70ns ± 2%  -35.05%
IndexByte/52      5.69ns ± 0%     3.80ns ± 1%  -33.35%
IndexByte/53      5.69ns ± 0%     3.78ns ± 0%  -33.54%
IndexByte/54      5.69ns ± 0%     3.78ns ± 1%  -33.51%
IndexByte/55      5.69ns ± 0%     3.78ns ± 0%  -33.61%
IndexByte/56      5.69ns ± 0%     3.81ns ± 3%  -33.12%
IndexByte/57      6.20ns ± 0%     3.79ns ± 4%  -38.89%
IndexByte/58      6.20ns ± 0%     3.74ns ± 2%  -39.58%
IndexByte/59      6.20ns ± 0%     3.69ns ± 2%  -40.47%
IndexByte/60      6.20ns ± 0%     3.79ns ± 1%  -38.81%
IndexByte/61      6.20ns ± 0%     3.77ns ± 1%  -39.23%
IndexByte/62      6.20ns ± 0%     3.79ns ± 0%  -38.89%
IndexByte/63      6.20ns ± 0%     3.79ns ± 0%  -38.90%
IndexByte/64      4.17ns ± 0%     3.47ns ± 3%  -16.70%
IndexByte/65      5.38ns ± 0%     4.21ns ± 0%  -21.59%
IndexByte/66      5.38ns ± 0%     4.21ns ± 0%  -21.58%
IndexByte/67      5.38ns ± 0%     4.22ns ± 0%  -21.58%
IndexByte/68      5.38ns ± 0%     4.22ns ± 0%  -21.59%
IndexByte/69      5.38ns ± 0%     4.22ns ± 0%  -21.56%
IndexByte/70      5.38ns ± 0%     4.21ns ± 0%  -21.59%
IndexByte/71      5.37ns ± 0%     4.21ns ± 0%  -21.51%
IndexByte/72      5.37ns ± 0%     4.22ns ± 0%  -21.46%
IndexByte/73      5.71ns ± 0%     4.22ns ± 0%  -26.20%
IndexByte/74      5.71ns ± 0%     4.21ns ± 0%  -26.21%
IndexByte/75      5.71ns ± 0%     4.21ns ± 0%  -26.17%
IndexByte/76      5.71ns ± 0%     4.22ns ± 0%  -26.22%
IndexByte/77      5.71ns ± 0%     4.22ns ± 0%  -26.22%
IndexByte/78      5.71ns ± 0%     4.21ns ± 0%  -26.22%
IndexByte/79      5.71ns ± 0%     4.22ns ± 0%  -26.21%
IndexByte/80      5.71ns ± 0%     4.21ns ± 0%  -26.19%
IndexByte/81      6.20ns ± 0%     4.39ns ± 0%  -29.13%
IndexByte/82      6.20ns ± 0%     4.36ns ± 0%  -29.67%
IndexByte/83      6.20ns ± 0%     4.36ns ± 0%  -29.63%
IndexByte/84      6.20ns ± 0%     4.39ns ± 0%  -29.21%
IndexByte/85      6.20ns ± 0%     4.36ns ± 0%  -29.64%
IndexByte/86      6.20ns ± 0%     4.36ns ± 0%  -29.63%
IndexByte/87      6.20ns ± 0%     4.39ns ± 0%  -29.21%
IndexByte/88      6.20ns ± 0%     4.36ns ± 0%  -29.65%
IndexByte/89      6.74ns ± 0%     4.36ns ± 0%  -35.33%
IndexByte/90      6.75ns ± 0%     4.37ns ± 0%  -35.22%
IndexByte/91      6.74ns ± 0%     4.36ns ± 0%  -35.30%
IndexByte/92      6.74ns ± 0%     4.36ns ± 0%  -35.34%
IndexByte/93      6.74ns ± 0%     4.37ns ± 0%  -35.20%
IndexByte/94      6.74ns ± 0%     4.36ns ± 0%  -35.33%
IndexByte/95      6.75ns ± 0%     4.36ns ± 0%  -35.32%
IndexByte/96      4.83ns ± 0%     4.34ns ± 2%  -10.24%
IndexByte/97      5.91ns ± 0%     4.65ns ± 0%  -21.24%
IndexByte/98      5.91ns ± 0%     4.65ns ± 0%  -21.24%
IndexByte/99      5.91ns ± 0%     4.65ns ± 0%  -21.23%
IndexByte/100     5.90ns ± 0%     4.65ns ± 0%  -21.21%
IndexByte/101     5.90ns ± 0%     4.65ns ± 0%  -21.22%
IndexByte/102     5.90ns ± 0%     4.65ns ± 0%  -21.23%
IndexByte/103     5.91ns ± 0%     4.65ns ± 0%  -21.23%
IndexByte/104     5.91ns ± 0%     4.65ns ± 0%  -21.24%
IndexByte/105     6.25ns ± 0%     4.65ns ± 0%  -25.59%
IndexByte/106     6.25ns ± 0%     4.65ns ± 0%  -25.59%
IndexByte/107     6.25ns ± 0%     4.65ns ± 0%  -25.60%
IndexByte/108     6.25ns ± 0%     4.65ns ± 0%  -25.58%
IndexByte/109     6.24ns ± 0%     4.65ns ± 0%  -25.50%
IndexByte/110     6.25ns ± 0%     4.65ns ± 0%  -25.56%
IndexByte/111     6.25ns ± 0%     4.65ns ± 0%  -25.60%
IndexByte/112     6.25ns ± 0%     4.65ns ± 0%  -25.59%
IndexByte/113     6.76ns ± 0%     5.05ns ± 0%  -25.37%
IndexByte/114     6.76ns ± 0%     5.05ns ± 0%  -25.31%
IndexByte/115     6.76ns ± 0%     5.05ns ± 0%  -25.38%
IndexByte/116     6.76ns ± 0%     5.05ns ± 0%  -25.31%
IndexByte/117     6.76ns ± 0%     5.05ns ± 0%  -25.38%
IndexByte/118     6.76ns ± 0%     5.05ns ± 0%  -25.31%
IndexByte/119     6.76ns ± 0%     5.05ns ± 0%  -25.38%
IndexByte/120     6.76ns ± 0%     5.05ns ± 0%  -25.36%
IndexByte/121     7.35ns ± 0%     5.05ns ± 0%  -31.33%
IndexByte/122     7.36ns ± 0%     5.05ns ± 0%  -31.42%
IndexByte/123     7.38ns ± 0%     5.05ns ± 0%  -31.60%
IndexByte/124     7.38ns ± 0%     5.05ns ± 0%  -31.59%
IndexByte/125     7.38ns ± 0%     5.05ns ± 0%  -31.60%
IndexByte/126     7.38ns ± 0%     5.05ns ± 0%  -31.58%
IndexByte/128     5.28ns ± 0%     5.10ns ± 0%   -3.41%
IndexByte/256     7.27ns ± 0%     7.28ns ± 2%   +0.13%
IndexByte/512     12.1ns ± 0%     11.8ns ± 0%   -2.51%
IndexByte/1K      23.1ns ± 3%     22.0ns ± 0%   -4.66%
IndexByte/2K      42.6ns ± 0%     42.4ns ± 0%   -0.41%
IndexByte/4K      90.3ns ± 0%     89.4ns ± 0%   -0.98%
IndexByte/8K       170ns ± 0%      170ns ± 0%   -0.59%
IndexByte/16K      331ns ± 0%      330ns ± 0%   -0.27%
IndexByte/32K      660ns ± 0%      660ns ± 0%   -0.08%
IndexByte/64K     1.30µs ± 0%     1.30µs ± 0%   -0.08%
IndexByte/128K    2.58µs ± 0%     2.58µs ± 0%   -0.04%
IndexByte/256K    5.15µs ± 0%     5.15µs ± 0%   -0.04%
IndexByte/512K    10.3µs ± 0%     10.3µs ± 0%   -0.03%
IndexByte/1M      20.6µs ± 0%     20.5µs ± 0%   -0.03%
IndexByte/2M      41.1µs ± 0%     41.1µs ± 0%   -0.03%
IndexByte/4M      82.2µs ± 0%     82.1µs ± 0%   -0.02%
IndexByte/8M       164µs ± 0%      164µs ± 0%   -0.01%
IndexByte/16M      328µs ± 0%      328µs ± 0%   -0.01%
IndexByte/32M      657µs ± 0%      657µs ± 0%   -0.00%

GOPPC64=power8 vs GOPPC64=power9. The Improvement is
most noticed between 16 and 64B, and goes away around
128B.

IndexByte/16      2.78ns ± 7%     2.65ns ±15%   -4.74%
IndexByte/17      3.04ns ± 1%     2.80ns ± 3%   -7.85%
IndexByte/18      3.05ns ± 0%     2.71ns ± 4%  -11.00%
IndexByte/19      3.02ns ± 2%     2.76ns ±10%   -8.74%
IndexByte/20      3.45ns ± 7%     2.91ns ± 0%  -15.46%
IndexByte/21      3.03ns ± 1%     2.84ns ± 9%   -6.33%
IndexByte/22      3.05ns ± 0%     2.67ns ± 1%  -12.38%
IndexByte/23      3.01ns ± 2%     2.67ns ± 1%  -11.24%
IndexByte/24      3.07ns ± 0%     2.92ns ±12%   -4.79%
IndexByte/25      3.04ns ± 1%     3.15ns ±15%   +3.63%
IndexByte/26      3.05ns ± 0%     2.83ns ±13%   -7.33%
IndexByte/27      2.97ns ± 3%     2.98ns ±10%   +0.56%
IndexByte/28      2.96ns ± 3%     2.96ns ± 9%   -0.05%
IndexByte/29      3.34ns ± 9%     3.03ns ±12%   -9.33%
IndexByte/30      3.05ns ± 0%     2.68ns ± 1%  -12.05%
IndexByte/31      3.05ns ± 0%     2.83ns ±12%   -7.27%
IndexByte/32      3.44ns ± 1%     3.21ns ±10%   -6.78%
IndexByte/33      3.35ns ± 0%     3.41ns ± 2%   +1.95%
IndexByte/34      3.35ns ± 0%     3.13ns ± 0%   -6.53%
IndexByte/35      3.35ns ± 0%     3.13ns ± 0%   -6.54%
IndexByte/36      3.35ns ± 0%     3.13ns ± 0%   -6.52%
IndexByte/37      3.35ns ± 0%     3.13ns ± 0%   -6.52%
IndexByte/38      3.35ns ± 0%     3.24ns ± 4%   -3.30%
IndexByte/39      3.48ns ± 9%     3.44ns ± 2%   -1.19%
IndexByte/40      3.55ns ± 4%     3.46ns ± 2%   -2.44%
IndexByte/41      3.42ns ± 2%     3.39ns ± 4%   -0.86%
IndexByte/42      3.55ns ±11%     3.46ns ± 1%   -2.65%
IndexByte/43      3.45ns ± 5%     3.44ns ± 2%   -0.31%
IndexByte/44      3.39ns ± 0%     3.43ns ± 3%   +1.23%
IndexByte/45      3.43ns ± 4%     3.50ns ± 1%   +2.07%
IndexByte/46      3.47ns ± 1%     3.46ns ± 2%   -0.31%
IndexByte/47      3.44ns ± 2%     3.47ns ± 1%   +0.78%
IndexByte/48      3.39ns ± 0%     3.46ns ± 2%   +1.96%
IndexByte/49      3.79ns ± 0%     3.47ns ± 0%   -8.41%
IndexByte/50      3.70ns ± 3%     3.64ns ± 5%   -1.66%
IndexByte/51      3.70ns ± 2%     3.75ns ± 0%   +1.40%
IndexByte/52      3.80ns ± 1%     3.77ns ± 0%   -0.70%
IndexByte/53      3.78ns ± 0%     3.77ns ± 0%   -0.46%
IndexByte/54      3.78ns ± 1%     3.53ns ± 7%   -6.74%
IndexByte/55      3.78ns ± 0%     3.47ns ± 0%   -8.17%
IndexByte/56      3.81ns ± 3%     3.45ns ± 0%   -9.43%
IndexByte/57      3.79ns ± 4%     3.47ns ± 0%   -8.45%
IndexByte/58      3.74ns ± 2%     3.55ns ± 4%   -5.16%
IndexByte/59      3.69ns ± 2%     3.61ns ± 4%   -2.01%
IndexByte/60      3.79ns ± 1%     3.45ns ± 0%   -9.09%
IndexByte/61      3.77ns ± 1%     3.47ns ± 0%   -7.93%
IndexByte/62      3.79ns ± 0%     3.45ns ± 0%   -8.97%
IndexByte/63      3.79ns ± 0%     3.47ns ± 0%   -8.44%
IndexByte/64      3.47ns ± 3%     3.18ns ± 0%   -8.41%

GOPPC64=power9 vs GOPPC64=power10. Only sizes <16 will
show meaningful changes.

IndexByte/1       3.27ns ± 8%     2.36ns ± 2%  -27.58%
IndexByte/2       3.06ns ± 4%     2.34ns ± 1%  -23.42%
IndexByte/3       3.77ns ±11%     2.48ns ± 7%  -34.03%
IndexByte/4       3.18ns ± 8%     2.33ns ± 1%  -26.69%
IndexByte/5       3.18ns ± 5%     2.34ns ± 4%  -26.26%
IndexByte/6       3.13ns ± 3%     2.35ns ± 1%  -24.97%
IndexByte/7       3.25ns ± 1%     2.33ns ± 1%  -28.22%
IndexByte/8       2.79ns ± 2%     2.36ns ± 1%  -15.32%
IndexByte/9       2.90ns ± 0%     2.34ns ± 2%  -19.36%
IndexByte/10      2.99ns ± 3%     2.31ns ± 1%  -22.70%
IndexByte/11      3.13ns ± 7%     2.31ns ± 0%  -26.08%
IndexByte/12      3.01ns ± 4%     2.32ns ± 1%  -22.91%
IndexByte/13      2.98ns ± 3%     2.31ns ± 1%  -22.72%
IndexByte/14      2.92ns ± 2%     2.61ns ±16%  -10.58%
IndexByte/15      3.02ns ± 5%     2.69ns ± 7%  -10.90%
IndexByte/16      2.65ns ±15%     2.29ns ± 1%  -13.61%

Change-Id: I4482f762d25eabf60def4981a0b2bc0c10ccf50c
Reviewed-on: https://go-review.googlesource.com/c/go/+/478656
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Archana Ravindar <aravind5@in.ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agocmd/link,cmd/internal/obj/ppc64: enable PCrel on power10/ppc64/linux
Paul E. Murphy [Mon, 17 Apr 2023 14:25:45 +0000 (09:25 -0500)]
cmd/link,cmd/internal/obj/ppc64: enable PCrel on power10/ppc64/linux

A CI machine has been set up to verify GOPPC64=power10 on ppc64/linux.

This should be sufficient to verify the PCrel relocation support works
for BE.

Note, power10/ppc64/linux is an oddball case. Today, it can only link
internally. Furthermore, all PCrel relocs are resolved at link time,
so it works despite ELFv1 having no official support for PCrel relocs
today.

Change-Id: Ibf79df69406ec6f9352c9d7d941ad946dba74e73
Reviewed-on: https://go-review.googlesource.com/c/go/+/485075
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
13 months agoos/user: skip tests that invoke Current if it returns an expected error
Bryan C. Mills [Thu, 20 Apr 2023 17:56:20 +0000 (13:56 -0400)]
os/user: skip tests that invoke Current if it returns an expected error

Today Current may fail if the binary is not built with cgo
and USER and/or HOME is not set in the environment.
That should not cause the test to fail.

After this change,

GOCACHE=$(go env GOCACHE) CGO_ENABLED=0 USER= HOME= go test os/user

now passes on linux/amd64.

For #59583.

Change-Id: Id290cd1088051e930d73f0dd554177124796e8f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/487015
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agogo/types: add test case for CL 486398
Russ Cox [Fri, 21 Apr 2023 14:31:34 +0000 (10:31 -0400)]
go/types: add test case for CL 486398

Forgot to 'git add' this test case in the CL.

Change-Id: Idde1d3d4525a99bdab0d3d69ac635a96a7cd5d73
Reviewed-on: https://go-review.googlesource.com/c/go/+/487335
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
13 months agocrypto/x509: use the RFC 6125 terminology in the VerifyHostname docs
Mateusz Poliwczak [Thu, 20 Apr 2023 18:08:55 +0000 (18:08 +0000)]
crypto/x509: use the RFC 6125 terminology in the VerifyHostname docs

RFC6125 uses the "complete left-most label" to describe star and full stop wildcards.

Change-Id: Ib7bcf77288ae7d72fb8664b663de2e52b378aaf6
GitHub-Last-Rev: b48c284c42205483fcc5e02b9d68d7bbd6a1c370
GitHub-Pull-Request: golang/go#59743
Reviewed-on: https://go-review.googlesource.com/c/go/+/487035
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>

13 months agogo/types, types2: abort type unification if no progress is made
Robert Griesemer [Fri, 21 Apr 2023 02:27:23 +0000 (19:27 -0700)]
go/types, types2: abort type unification if no progress is made

Fixes #59740.
For #59750.

Change-Id: I153d0a412bdfb15f81d6999e29691dc093fd0fcb
Reviewed-on: https://go-review.googlesource.com/c/go/+/487197
Reviewed-by: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
13 months agocmd/compile: expose ir.Func to ssa
Michael Pratt [Tue, 11 Apr 2023 20:40:12 +0000 (16:40 -0400)]
cmd/compile: expose ir.Func to ssa

ssagen.ssafn already holds the ir.Func, and ssa.Frontend.SetWBPos and
ssa.Frontend.Lsym are simple wrappers around parts of the ir.Func.

Expose the ir.Func through ssa.Frontend, allowing us to remove these
wrapper methods and allowing future access to additional features of the
ir.Func if needed.

While we're here, drop ssa.Frontend.Line, which is unused.

For #58298.

Change-Id: I30c4cbd2743e9ad991d8c6b388484a7d1e95f3ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/484215
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>

13 months agoruntime: turn frame pointer unwinding for tracing off by default
Nick Ripley [Wed, 19 Apr 2023 18:42:30 +0000 (14:42 -0400)]
runtime: turn frame pointer unwinding for tracing off by default

Frame pointer unwinding during execution tracing sometimes crashes.
Until this is diagnosed and fixed, it should be turned off by default.

Updates #59692

Change-Id: I0f2ca24b6d48435b0acfd3da8e4f25b9cfa4ec19
Reviewed-on: https://go-review.googlesource.com/c/go/+/486382
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Nick Ripley <nick.ripley@datadoghq.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>

13 months agoruntime: bring back minHeapIdx in scavenge index
Michael Anthony Knyszek [Thu, 20 Apr 2023 02:41:08 +0000 (02:41 +0000)]
runtime: bring back minHeapIdx in scavenge index

The scavenge index currently doesn't guard against overflow, and CL
436395 removed the minHeapIdx optimization that allows the chunk scan to
skip scanning chunks that haven't been mapped for the heap, and are only
available as a consequence of chunks' mapped region being rounded out to
a page on both ends.

Because the 0'th chunk is never mapped, minHeapIdx effectively prevents
overflow, fixing the iOS breakage.

This change also refactors growth and initialization a little bit to
decouple it from pageAlloc a bit and share code across platforms.

Change-Id: If7fc3245aa81cf99451bf8468458da31986a9b0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/486695
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>

13 months agolog/slog: add Source type for source location
Jonathan Amsterdam [Fri, 7 Apr 2023 13:52:56 +0000 (09:52 -0400)]
log/slog: add Source type for source location

Add a struct called Source that holds the function, file and line
of a location in the program's source code.

When HandleOptions.AddSource is true, the ReplaceAttr function will
get an Attr whose key is SourceKey and whose value is a *Source.

We use *Source instead of Source to save an allocation. The pointer
and the value each cause one allocation up front: the pointer when it
is created, and the value when it is assigned to the `any` field of a
slog.Value (handle.go:283). If a ReplaceAttr function wanted to modify
a Source value, it would have to create a new slog.Value to return,
causing a second allocation, but the function can modify a *Source in
place.

TextHandler displays a Source as "file:line".

JSONHandler displays a Source as a group of its non-zero fields.

This replaces the previous design, where source location was always a
string with the format "file:line". The new design gives users more
control over how to output and consume source locations.

Fixes #59280.

Change-Id: I84475abd5ed83fc354b50e34325c7b246cf327c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/486376
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
13 months agocmd/dist: refactor generated cgo-support logic
Bryan C. Mills [Tue, 11 Apr 2023 14:27:45 +0000 (14:27 +0000)]
cmd/dist: refactor generated cgo-support logic

During bootstrapping, cmd/dist writes a file indicating which
GOOS/GOARCH combinations are valid, and which support cgo-enabled
builds. That information previously went into the go/build package,
but today it fits in more naturally in the internal/platform package
(which already has a number of functions indicating feature support
for GOOS/GOARCH combinations).

Moreover, as of CL 450739 the cmd/go logic for determining whether to
use cgo is somewhat more nuanced than the go/build logic: cmd/go
checks for the presence of a C compiler, whereas go/build does not
(mostly because it determines its configuration at package-init time,
and checking $PATH for a C compiler is somewhat expensive).

To simplify this situation, this change:

- consolidates the “cgo supported” check in internal/platform
  (alongside many other platform-support checks) instead of making
  it a one-off in go/build,

- and updates a couple of tests to use testenv.HasCGO instead of
  build.Default.CgoEnabled to decide whether to test a cgo-specific
  behavior.

For #58884.
For #59500.

Change-Id: I0bb2502dba4545a3d98c9e915727382ce536a0f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/483695
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>

13 months agoio: ReadAll: do not check for realloc in first round
Olivier Mengué [Tue, 18 Apr 2023 20:13:27 +0000 (22:13 +0200)]
io: ReadAll: do not check for realloc in first round

Refactor io.ReadAll to check for realloc of the buffer only after the
first read.

Fixes: #59702
Change-Id: I93b99139e6756f21738d47e7d9ad08e1d167258e
Reviewed-on: https://go-review.googlesource.com/c/go/+/486236
Auto-Submit: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
13 months agoRevert "internal/abi, runtime, cmd: merge StackSmall, StackBig consts into internal...
Austin Clements [Thu, 20 Apr 2023 16:10:15 +0000 (16:10 +0000)]
Revert "internal/abi, runtime, cmd: merge StackSmall, StackBig consts into internal/abi"

This reverts commit CL 486379.

Submitted out of order and breaks bootstrap.

Change-Id: Ie20a61cc56efc79a365841293ca4e7352b02d86b
Reviewed-on: https://go-review.googlesource.com/c/go/+/486917
TryBot-Bypass: Austin Clements <austin@google.com>
Reviewed-by: David Chase <drchase@google.com>
13 months agoRevert "runtime, cmd: rationalize StackLimit and StackGuard"
Austin Clements [Thu, 20 Apr 2023 16:09:48 +0000 (16:09 +0000)]
Revert "runtime, cmd: rationalize StackLimit and StackGuard"

This reverts commit CL 486380.

Submitted out of order and breaks bootstrap.

Change-Id: I67bd225094b5c9713b97f70feba04d2c99b7da76
Reviewed-on: https://go-review.googlesource.com/c/go/+/486916
Reviewed-by: David Chase <drchase@google.com>
TryBot-Bypass: Austin Clements <austin@google.com>

13 months agoRevert "runtime: tidy _Stack* constant naming"
Austin Clements [Thu, 20 Apr 2023 16:08:13 +0000 (16:08 +0000)]
Revert "runtime: tidy _Stack* constant naming"

This reverts commit CL 486381.

Submitted out of order and breaks bootstrap.

Change-Id: Ia472111cb966e884a48f8ee3893b3bf4b4f4f875
Reviewed-on: https://go-review.googlesource.com/c/go/+/486915
Reviewed-by: David Chase <drchase@google.com>
TryBot-Bypass: Austin Clements <austin@google.com>

13 months agoruntime: tidy _Stack* constant naming
Austin Clements [Wed, 19 Apr 2023 18:58:47 +0000 (14:58 -0400)]
runtime: tidy _Stack* constant naming

For #59670.

Change-Id: I4476d6f92663e8a825d063d6e6a7fc9a2ac99d4d
Reviewed-on: https://go-review.googlesource.com/c/go/+/486381
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
13 months agoruntime, cmd: rationalize StackLimit and StackGuard
Austin Clements [Wed, 19 Apr 2023 18:01:05 +0000 (14:01 -0400)]
runtime, cmd: rationalize StackLimit and StackGuard

The current definitions of StackLimit and StackGuard only indirectly
specify the NOSPLIT stack limit and duplicate a literal constant
(928). Currently, they define the stack guard delta, and from there
compute the NOSPLIT limit.

Rationalize these by defining a new constant, abi.StackNosplitBase,
which consolidates and directly specifies the NOSPLIT stack limit (in
the default case). From this we then compute the stack guard delta,
inverting the relationship between these two constants. While we're
here, we rename StackLimit to StackNosplit to make it clearer what's
being limited.

This change does not affect the values of these constants in the
default configuration. It does slightly change how
StackGuardMultiplier values other than 1 affect the constants, but
this multiplier is a pretty rough heuristic anyway.

                    before after
stackNosplit           800   800
_StackGuard            928   928
stackNosplit -race    1728  1600
_StackGuard -race     1856  1728

For #59670.

Change-Id: Ibe20825ebe0076bbd7b0b7501177b16c9dbcb79e
Reviewed-on: https://go-review.googlesource.com/c/go/+/486380
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agointernal/abi, runtime, cmd: merge StackSmall, StackBig consts into internal/abi
Austin Clements [Wed, 19 Apr 2023 17:21:02 +0000 (13:21 -0400)]
internal/abi, runtime, cmd: merge StackSmall, StackBig consts into internal/abi

For #59670.

Change-Id: I04a17079b351b9b4999ca252825373c17afb8a88
Reviewed-on: https://go-review.googlesource.com/c/go/+/486379
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
13 months agoruntime: mix a bit more in arm64 hash function
Keith Randall [Wed, 19 Apr 2023 23:41:37 +0000 (16:41 -0700)]
runtime: mix a bit more in arm64 hash function

We really need 3 mix steps between the data being hashed and the output.
One mix can only spread a 1 bit change to 32 bits. The second mix
can spread to all 128 bits, but the spread is not complete. A third mix
spreads out ~evenly to all 128 bits.

The amd64 version has 3 mix steps.

Fixes #59643

Change-Id: I54ad8686ca42bcffb6d0ec3779d27af682cc96e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/486616
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agoruntime: initialize scavengeIndex fields properly
Michael Anthony Knyszek [Wed, 19 Apr 2023 22:00:23 +0000 (22:00 +0000)]
runtime: initialize scavengeIndex fields properly

Currently these fields are uninitialized causing failures on aix-ppc64,
which has a slightly oddly-defined address space compared to the rest.

Change-Id: I2aa46731174154dce86c2074bd0b00eef955d86d
Reviewed-on: https://go-review.googlesource.com/c/go/+/486655
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>

13 months agogo/types: fix version downgrade bug without Config.GoVersion
Russ Cox [Wed, 19 Apr 2023 20:05:37 +0000 (16:05 -0400)]
go/types: fix version downgrade bug without Config.GoVersion

The gVisor team reported a regression in their checkers,
which don't set Config.GoVersion, processing files that say
//go:build go1.13 but still use 'any' (which happened in Go 1.18).
That situation should continue to work, since it worked before,
so add a special case for not knowing the GoVersion.

Change-Id: I8820d8ccbdf76d304e2c7e45f6aaa993ff3d16a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/486398
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
13 months agodoc: add release notes for new context functions
Damien Neil [Wed, 19 Apr 2023 19:31:27 +0000 (12:31 -0700)]
doc: add release notes for new context functions

For #40221
For #56661
For #57928

Change-Id: Iaf7425bb26eeb9c23235d13c786d5bb572159481
Reviewed-on: https://go-review.googlesource.com/c/go/+/486535
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Sameer Ajmani <sameer@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agolog/slog: remove calls to Value.Resolve from core
Jonathan Amsterdam [Fri, 14 Apr 2023 10:31:02 +0000 (06:31 -0400)]
log/slog: remove calls to Value.Resolve from core

Remove calls to Value.Resolve from Record.AddAttrs, Record.Add and Logger.With.
Handlers must resolve values themselves; document that in Handler.

Call Value.Resolve in the built-in handlers.

Updates #59292.

Change-Id: I00ba2131be0b16e3b1a22741249fd6f81c3efde1
Reviewed-on: https://go-review.googlesource.com/c/go/+/486375
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
13 months agostd: fix various nilness findings
Alan Donovan [Wed, 19 Apr 2023 22:48:00 +0000 (18:48 -0400)]
std: fix various nilness findings

Found by running
$ go run golang.org/x/tools/go/analysis/passes/nilness/cmd/nilness@latest std

No actual bugs--other than one panic(nil)--but a
few places where error nilness was unclear.

Change-Id: Ia916ba30f46f29c1bcf928cc62280169b922463a
Reviewed-on: https://go-review.googlesource.com/c/go/+/486675
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Alan Donovan <adonovan@google.com>

13 months agocrypto/tls: retry DialWithTimeout until the listener accepts a connection
Bryan C. Mills [Mon, 17 Apr 2023 14:56:09 +0000 (10:56 -0400)]
crypto/tls: retry DialWithTimeout until the listener accepts a connection

The point of DialWithTimeout seems to be to test what happens when the
connection times out during handshake. However, the test wasn't
actually verifying that the connection made it into the handshake at
all. That would not only fail to test the intended behavior, but also
leak the Accept goroutine until arbitrarily later, at which point it
may call t.Error after the test t is already done.

Instead, we now:

- retry the test with a longer timeout if we didn't accept a
  connection, and

- wait for the Accept goroutine to actually complete when the test
  finishes.

Fixes #59646.

Change-Id: Ie56ce3297e2c183c02e67b8f6b26a71e50964558
Reviewed-on: https://go-review.googlesource.com/c/go/+/485115
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Commit-Queue: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agonet: remove unused _C_FreeCString
Tobias Klauser [Wed, 19 Apr 2023 18:55:47 +0000 (20:55 +0200)]
net: remove unused _C_FreeCString

It's unused since CL 486015.

Change-Id: Ic9397f5f84940a09f05499da2774b952fe6d1365
Reviewed-on: https://go-review.googlesource.com/c/go/+/486475
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>

13 months agocontext: add AfterFunc
Damien Neil [Wed, 5 Apr 2023 23:06:36 +0000 (16:06 -0700)]
context: add AfterFunc

Add an AfterFunc function, which registers a function to run after
a context has been canceled.

Add support for contexts that implement an AfterFunc method, which
can be used to avoid the need to start a new goroutine watching
the Done channel when propagating cancellation signals.

Fixes #57928

Change-Id: If0b2cdcc4332961276a1ff57311338e74916259c
Reviewed-on: https://go-review.googlesource.com/c/go/+/482695
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Sameer Ajmani <sameer@golang.org>
13 months agoslices: optimize Index and Compact for large types
Egon Elbre [Wed, 19 Apr 2023 08:43:51 +0000 (11:43 +0300)]
slices: optimize Index and Compact for large types

Using `for i, v := range` loops causes extra copies.
Try to get rid of as much copies as possible.

  goos: windows
  goarch: amd64
  pkg: slices
  cpu: AMD Ryzen Threadripper 2950X 16-Core Processor
                       │   old.txt~    │               new.txt~               │
                       │    sec/op     │    sec/op     vs base                │
  EqualFunc_Large-32       1.077m ± 1%   1.072m ±  1%        ~ (p=0.631 n=10)
  Index_Large-32         346.329µ ± 1%   6.510µ ± 24%  -98.12% (p=0.000 n=10)
  IndexFunc_Large-32       502.9µ ± 0%   381.2µ ±  1%  -24.21% (p=0.000 n=10)
  Compact_Large-32         409.5µ ± 1%   145.2µ ±  9%  -64.54% (p=0.000 n=10)
  CompactFunc_Large-32     693.5µ ± 1%   663.1µ ±  3%   -4.39% (p=0.000 n=10)
  geomean                  556.3µ        191.3µ        -65.61%

Change-Id: I36a2f2172cf30d97a5aa6f8d7cf6981d67daec62
Reviewed-on: https://go-review.googlesource.com/c/go/+/486235
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
13 months agomake.{bash,bat}: check unmodified $PATH for $GOROOT/bin presence
Dmitri Shuralyov [Mon, 17 Apr 2023 21:37:37 +0000 (17:37 -0400)]
make.{bash,bat}: check unmodified $PATH for $GOROOT/bin presence

Previously, all.bash and all.bat restored the original $PATH before
calling 'dist banner', so that it would do its job of checking whether
the user still needs to add $GOROOT/bin to their $PATH. That worked for
those scripts, but had no effect on make.bash nor make.bat.

Instead of trying to extend that logic to more scripts, change the
approach to provide dist an unmodified copy of $PATH via an internal
to dist environment variable $DIST_UNMODIFIED_PATH. The make.bash and
make.bat scripts happen to use dist env -p to modify $PATH, making it
viable to add the internal variable there instead of in each script.

It currently works by adding semicolon terminators to dist env output
so that make.bash's 'eval $(dist env -p)' works as before but is able to
export DIST_UNMODIFIED_PATH for following dist invocations to observe.
Nothing needs to be done for Windows since its 'set ENV=val' format
already has that effect.

Plan 9 doesn't use the -p flag of dist env, and checks that GOROOT/bin
is bound before /bin rather than looking at the $PATH env var like other
OSes, so it may not have this bug. I don't have easy access to Plan 9
and haven't tried to confirm.

Fixes #42563.

Change-Id: I74691931167e974a930f7589d22a48bb6b931163
Reviewed-on: https://go-review.googlesource.com/c/go/+/485896
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>

13 months agoruntime: manage huge pages explicitly
Michael Anthony Knyszek [Fri, 23 Sep 2022 16:32:34 +0000 (16:32 +0000)]
runtime: manage huge pages explicitly

This change makes it so that on Linux the Go runtime explicitly marks
page heap memory as either available to be backed by hugepages or not
using heuristics based on density.

The motivation behind this change is twofold:
1. In default Linux configurations, khugepaged can recoalesce hugepages
   even after the scavenger breaks them up, resulting in significant
   overheads for small heaps when their heaps shrink.
2. The Go runtime already has some heuristics about this, but those
   heuristics appear to have bit-rotted and result in haphazard
   hugepage management. Unlucky (but otherwise fairly dense) regions of
   memory end up not backed by huge pages while sparse regions end up
   accidentally marked MADV_HUGEPAGE and are not later broken up by the
   scavenger, because it already got the memory it needed from more
   dense sections (this is more likely to happen with small heaps that
   go idle).

In this change, the runtime uses a new policy:

1. Mark all new memory MADV_HUGEPAGE.
2. Track whether each page chunk (4 MiB) became dense during the GC
   cycle. Mark those MADV_HUGEPAGE, and hide them from the scavenger.
3. If a chunk is not dense for 1 full GC cycle, make it visible to the
   scavenger.
4. The scavenger marks a chunk MADV_NOHUGEPAGE before it scavenges it.

This policy is intended to try and back memory that is a good candidate
for huge pages (high occupancy) with huge pages, and give memory that is
not (low occupancy) to the scavenger. Occupancy is defined not just by
occupancy at any instant of time, but also occupancy in the near future.
It's generally true that by the end of a GC cycle the heap gets quite
dense (from the perspective of the page allocator).

Because we want scavenging and huge page management to happen together
(the right time to MADV_NOHUGEPAGE is just before scavenging in order to
break up huge pages and keep them that way) and the cost of applying
MADV_HUGEPAGE and MADV_NOHUGEPAGE is somewhat high, the scavenger avoids
releasing memory in dense page chunks. All this together means the
scavenger will now more generally release memory on a ~1 GC cycle delay.

Notably this has implications for scavenging to maintain the memory
limit and the runtime/debug.FreeOSMemory API. This change makes it so
that in these cases all memory is visible to the scavenger regardless of
sparseness and delays the page allocator in re-marking this memory with
MADV_NOHUGEPAGE for around 1 GC cycle to mitigate churn.

The end result of this change should be little-to-no performance
difference for dense heaps (MADV_HUGEPAGE works a lot like the default
unmarked state) but should allow the scavenger to more effectively take
back fragments of huge pages. The main risk here is churn, because
MADV_HUGEPAGE usually forces the kernel to immediately back memory with
a huge page. That's the reason for the large amount of hysteresis (1
full GC cycle) and why the definition of high density is 96% occupancy.

Fixes #55328.

Change-Id: I8da7998f1a31b498a9cc9bc662c1ae1a6bf64630
Reviewed-on: https://go-review.googlesource.com/c/go/+/436395
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agoruntime: disable huge pages for GC metadata for small heaps
Michael Anthony Knyszek [Tue, 3 Jan 2023 17:59:48 +0000 (17:59 +0000)]
runtime: disable huge pages for GC metadata for small heaps

For #55328.

Change-Id: I8792161f09906c08d506cc0ace9d07e76ec6baa6
Reviewed-on: https://go-review.googlesource.com/c/go/+/460316
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agoos: check for ErrNotExist instead of ErrExist in TestNonPollable
Bryan C. Mills [Wed, 19 Apr 2023 13:56:45 +0000 (09:56 -0400)]
os: check for ErrNotExist instead of ErrExist in TestNonPollable

An apparent typo in CL 484837 caused the test to check for ErrExist
instead of ErrNotExist when opening /dev/net/tun for read. That causes
the test to fail on platforms where /dev/net/ton does not exist,
such as on the darwin-amd64-longtest builder.

Updates #59545.

Change-Id: I9402ce0dba11ab459674e8358ae9a8b97eabc8d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/486255
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Commit-Queue: Bryan Mills <bcmills@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agocmd/compile/internal/types2: only mark variables as used if they are
Robert Griesemer [Tue, 18 Apr 2023 23:43:11 +0000 (16:43 -0700)]
cmd/compile/internal/types2: only mark variables as used if they are

Marking variables in erroneous variable declarations as used is
convenient for tests but doesn't necessarily hide follow-on errors
in real code: either the variable is not supposed to be declared in
the first place and then we should get an error if it is not used,
or it is there because it is intended to be used, and the we expect
an error it if is not used.

This brings types2 closer to go/types.

Change-Id: If7ee1298fc770f7ad0cefe7e968533fd50ec2343
Reviewed-on: https://go-review.googlesource.com/c/go/+/486175
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agogo/types, types2: be deliberate with Checker.use in Checker.assignVar
Robert Griesemer [Wed, 19 Apr 2023 02:20:28 +0000 (19:20 -0700)]
go/types, types2: be deliberate with Checker.use in Checker.assignVar

Follow-up on comment in CL 486135. Cleanup.

Change-Id: Ib0480d52e30687350f8328965c18fa83262df0a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/486215
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
13 months agogo/types, types2: don't panic for invalid assignments of comma-ok expressions
Robert Griesemer [Tue, 18 Apr 2023 22:10:46 +0000 (15:10 -0700)]
go/types, types2: don't panic for invalid assignments of comma-ok expressions

The relevant code was broken with CL 478218. Before that CL,
Checker.assignVar used to return the assigned type, or nil,
in case of failure. Checker.recordCommaOkTypes used to take
two types (not two operands), and if one of those types was
nil, it would simply not record. CL 478218, lost that (nil)
signal.

This change consistently reports an assignment check failure
by setting x.mode to invalid for initVar and assignVar and
then tests if x.mode != invalid before recording a comma-ok
expression.

Fixes #59371.

Change-Id: I193815ff3e4b43e3e510fe25bd0e72e0a6a816c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/486135
Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agonet: check for NUL bytes in strings passed to C functions
Ian Lance Taylor [Tue, 18 Apr 2023 21:48:36 +0000 (14:48 -0700)]
net: check for NUL bytes in strings passed to C functions

Use syscall.BytePtrFromString and syscall.ByteSliceFromString.

Change-Id: I9409ecd93aaca82390bf3f34be56ec354148a241
Reviewed-on: https://go-review.googlesource.com/c/go/+/486015
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>

13 months agogo/types, types2: consistently set x.mode in Checker.assignment
Robert Griesemer [Tue, 18 Apr 2023 17:20:15 +0000 (10:20 -0700)]
go/types, types2: consistently set x.mode in Checker.assignment

Per the doc string, Checker.assignment must set x.mode to invalid
in case of failure.

(It may be simpler to return a bool, but the operand x may be tested
by callers several stack frames above.)

Change-Id: Ia1789b0396e8338103c0e707761c46f8d253fd31
Reviewed-on: https://go-review.googlesource.com/c/go/+/485875
Reviewed-by: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
13 months agoruntime: adjust frame pointer on stack copy on ARM64
Cherry Zhang [Mon, 6 Jul 2020 20:03:33 +0000 (16:03 -0400)]
runtime: adjust frame pointer on stack copy on ARM64

Frame pointer is enabled on ARM64. When copying stacks, the
saved frame pointers need to be adjusted.

Updates #39524, #40044.
Fixes #58432.

Change-Id: I73651fdfd1a6cccae26a5ce02e7e86f6c2fb9bf7
Reviewed-on: https://go-review.googlesource.com/c/go/+/241158
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agoruntime: consolidate function descriptor definitions on PPC64
Michael Pratt [Mon, 17 Apr 2023 18:41:10 +0000 (14:41 -0400)]
runtime: consolidate function descriptor definitions on PPC64

This reapplies CL 481075, which was a reappliation of CL 478917.

This CL has been reverted twice now due to conflicts with CL 392854 /
CL 481061, which had bugs and had to be reverted.

Now this CL skips the conflicting changes to runtime/cgo/asm_ppc64x.s,
which will be merged directly into a new version of CL 392854 /
CL 481061. That way, if there are _more_ issues, this CL need not be
involved in any more reverts.

Change-Id: I2801b918faf9418dd0edff19f2a63f4d9e08896c
Reviewed-on: https://go-review.googlesource.com/c/go/+/485335
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
13 months agocmd/compile: remove memequal call from string compares in more cases
Keith Randall [Mon, 17 Apr 2023 22:51:29 +0000 (15:51 -0700)]
cmd/compile: remove memequal call from string compares in more cases

Add more rules to ensure that order doesn't matter.

Add memequal 0 rule.

Try to use a constant argument to memequal when one is available.

Fixes #59684

Change-Id: I36e85ffbd949396ed700ed6e8ec2bc3ae013f5d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/485535
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agodebug/elf: support zstd compression
Ian Lance Taylor [Fri, 3 Mar 2023 22:29:44 +0000 (14:29 -0800)]
debug/elf: support zstd compression

Test cases added to debug/dwarf because that is where it matters in practice.

The new test binary line-gcc-zstd.elf built with

    gcc -g -no-pie -Wl,--compress-debug-sections=zstd line[12].c

using

    gcc (Debian 12.2.0-10) 12.2.0

with a development version of the GNU binutils.

Fixes #55107

Change-Id: I48507c96902e1f83a174e5647b5cc403d965b52b
Reviewed-on: https://go-review.googlesource.com/c/go/+/473256
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
13 months agointernal/zstd: new internal package for zstd decompression
Ian Lance Taylor [Fri, 3 Mar 2023 19:42:07 +0000 (11:42 -0800)]
internal/zstd: new internal package for zstd decompression

This package only does zstd decompression, which is starting to
be used for ELF debug sections. If we need zstd compression we
should use github.com/klauspost/compress/zstd. But for now that
is a very large package to vendor into the standard library.

For #55107

Change-Id: I60ede735357d491be653477ed419cf5f2f0d3f71
Reviewed-on: https://go-review.googlesource.com/c/go/+/473356
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>

13 months agoruntime: add and use pollDesc fd sequence field
Ian Lance Taylor [Fri, 14 Apr 2023 22:34:10 +0000 (15:34 -0700)]
runtime: add and use pollDesc fd sequence field

It is possible for a netpoll file to be closed and for the pollDesc
to be reused while a netpoll is running. This normally only causes
spurious wakeups, but if there is an error on the old file then the
new file can be incorrectly marked as having an error.

Fix this problem on most systems by introducing an fd sequence field
and using that as a tag in a taggedPointer. The taggedPointer is
stored in epoll or kqueue or whatever is being used. If the taggedPointer
returned by the kernel has a tag that does not match the fd
sequence field, the notification is for a closed file, and we
can ignore it. We check the tag stored in the pollDesc, and we also
check the tag stored in the pollDesc.atomicInfo.

This approach does not work on 32-bit systems where the kernel
only provides a 32-bit field to hold a user value. On those systems
we continue to use the older method without the sequence protection.
This is not ideal, but it is not an issue on Linux because the kernel
provides a 64-bit field, and it is not an issue on Windows because
there are no poller errors on Windows. It is potentially an issue
on *BSD systems, but on those systems we already call fstat in newFile
in os/file_unix.go to avoid adding non-pollable files to kqueue.
So we currently don't know of any cases that will fail.

Fixes #59545

Change-Id: I9a61e20dc39b4266a7a2978fc16446567fe683ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/484837
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Orlando Labao <orlando.labao43@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Bypass: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>

13 months agoruntime: change lfstack support to taggedPointer
Ian Lance Taylor [Fri, 14 Apr 2023 21:36:12 +0000 (14:36 -0700)]
runtime: change lfstack support to taggedPointer

This is a refactoring with no change in behavior, in preparation
for future netpoll work.

For #59545

Change-Id: I493c5fd0f49f31b75787f7b5b89c544bed73f64f
Reviewed-on: https://go-review.googlesource.com/c/go/+/484836
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Orlando Labao <orlando.labao43@gmail.com>
13 months agocmd/go: skip over all workspace modules in go mod verify
Michael Matloob [Mon, 17 Apr 2023 20:42:18 +0000 (16:42 -0400)]
cmd/go: skip over all workspace modules in go mod verify

This was a remaining place where we made the assumption that there is
only one workspace module. So we'd only skip the first workspace
module when running go mod verify. Instead skip over the first
MainModules.Len() modules of the buildlist, which are all the main
modules.

Fixes #54372

Change-Id: Ife687c907ae4326759c43cc35f78d429d5113b19
Reviewed-on: https://go-review.googlesource.com/c/go/+/485475
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
13 months agocmd/go: include internal/godebugs in TestNewReleaseRebuildsStalePackagesInGOPATH
Bryan C. Mills [Tue, 18 Apr 2023 17:48:14 +0000 (13:48 -0400)]
cmd/go: include internal/godebugs in TestNewReleaseRebuildsStalePackagesInGOPATH

This fixes a lingering failure on the 'longtest' builders,
introduced in CL 476280.

For #57001.

Change-Id: I18872b1bc990dd16c0b86ebc5878c0a04ce9b2d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/485897
Run-TryBot: Bryan Mills <bcmills@google.com>
Commit-Queue: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agosrc/internal/godebugs: add a skip for missing godebug.md
Michael Matloob [Tue, 18 Apr 2023 16:16:43 +0000 (12:16 -0400)]
src/internal/godebugs: add a skip for missing godebug.md

Currently android doesn't include godebug.md in its doc folder, and
TestAll in godebugs_test.go is failing because it can't open the file.
Add a skip in case the file is missing (except for linux so we can
catch the case where we stop generating the file).

Change-Id: I37a711e49a494c33bc92bf3e31cf40471ea9d5b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/485795
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

13 months agocmd/go: mark gocacheverify godebug as undocumented
Michael Matloob [Tue, 18 Apr 2023 16:11:20 +0000 (12:11 -0400)]
cmd/go: mark gocacheverify godebug as undocumented

This avoids the panic for godebugs without documentation. Perhaps we
eventually want to remove this godebug.

Change-Id: I032cdded0f3ccf6681821286e2cb6f0b90a318c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/485756
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Bypass: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
13 months agocmd/go: stub out gotoolchain.go for wasip1 os build tag
Michael Matloob [Tue, 18 Apr 2023 15:20:58 +0000 (11:20 -0400)]
cmd/go: stub out gotoolchain.go for wasip1 os build tag

There's a stub for gotoolchain.go for the js build tag because js/wasm
doesn't define syscall.Exec. But there are builders that are wasm but
not js, which also don't have syscall.Exec. The wasip1 GOOS is one
example. Stub out gotoolchain.go for wasip1 also.

Change-Id: I224bb385474ad9c5d3c28a83a000f450dfb43c0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/485735
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>

13 months agoRevert "cmd/compile: rework marking of dead hidden closure functions"
Than McIntosh [Tue, 18 Apr 2023 15:30:15 +0000 (15:30 +0000)]
Revert "cmd/compile: rework marking of dead hidden closure functions"

This reverts commit http://go.dev/cl//484859

Reason for revert: causes linker errors in a number of google-internal tests.

Change-Id: I322252f784a46d2b1d447ebcdca86ce14bc0cc91
Reviewed-on: https://go-review.googlesource.com/c/go/+/485755
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>