During bootstrapping, cmd/dist writes a file indicating which
GOOS/GOARCH combinations are valid, and which support cgo-enabled
builds. That information previously went into the go/build package,
but today it fits in more naturally in the internal/platform package
(which already has a number of functions indicating feature support
for GOOS/GOARCH combinations).
Moreover, as of CL 450739 the cmd/go logic for determining whether to
use cgo is somewhat more nuanced than the go/build logic: cmd/go
checks for the presence of a C compiler, whereas go/build does not
(mostly because it determines its configuration at package-init time,
and checking $PATH for a C compiler is somewhat expensive).
To simplify this situation, this change:
- consolidates the “cgo supported” check in internal/platform
(alongside many other platform-support checks) instead of making
it a one-off in go/build,
- and updates a couple of tests to use testenv.HasCGO instead of
build.Default.CgoEnabled to decide whether to test a cgo-specific
behavior.
runtime, cmd: rationalize StackLimit and StackGuard
The current definitions of StackLimit and StackGuard only indirectly
specify the NOSPLIT stack limit and duplicate a literal constant
(928). Currently, they define the stack guard delta, and from there
compute the NOSPLIT limit.
Rationalize these by defining a new constant, abi.StackNosplitBase,
which consolidates and directly specifies the NOSPLIT stack limit (in
the default case). From this we then compute the stack guard delta,
inverting the relationship between these two constants. While we're
here, we rename StackLimit to StackNosplit to make it clearer what's
being limited.
This change does not affect the values of these constants in the
default configuration. It does slightly change how
StackGuardMultiplier values other than 1 affect the constants, but
this multiplier is a pretty rough heuristic anyway.
before after
stackNosplit 800 800
_StackGuard 928 928
stackNosplit -race 1728 1600
_StackGuard -race 1856 1728
Keith Randall [Wed, 19 Apr 2023 23:41:37 +0000 (16:41 -0700)]
runtime: mix a bit more in arm64 hash function
We really need 3 mix steps between the data being hashed and the output.
One mix can only spread a 1 bit change to 32 bits. The second mix
can spread to all 128 bits, but the spread is not complete. A third mix
spreads out ~evenly to all 128 bits.
The amd64 version has 3 mix steps.
Fixes #59643
Change-Id: I54ad8686ca42bcffb6d0ec3779d27af682cc96e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/486616
Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Michael Anthony Knyszek [Wed, 19 Apr 2023 22:00:23 +0000 (22:00 +0000)]
runtime: initialize scavengeIndex fields properly
Currently these fields are uninitialized causing failures on aix-ppc64,
which has a slightly oddly-defined address space compared to the rest.
Change-Id: I2aa46731174154dce86c2074bd0b00eef955d86d
Reviewed-on: https://go-review.googlesource.com/c/go/+/486655
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
go/types: fix version downgrade bug without Config.GoVersion
The gVisor team reported a regression in their checkers,
which don't set Config.GoVersion, processing files that say
//go:build go1.13 but still use 'any' (which happened in Go 1.18).
That situation should continue to work, since it worked before,
so add a special case for not knowing the GoVersion.
crypto/tls: retry DialWithTimeout until the listener accepts a connection
The point of DialWithTimeout seems to be to test what happens when the
connection times out during handshake. However, the test wasn't
actually verifying that the connection made it into the handshake at
all. That would not only fail to test the intended behavior, but also
leak the Accept goroutine until arbitrarily later, at which point it
may call t.Error after the test t is already done.
Instead, we now:
- retry the test with a longer timeout if we didn't accept a
connection, and
- wait for the Accept goroutine to actually complete when the test
finishes.
Damien Neil [Wed, 5 Apr 2023 23:06:36 +0000 (16:06 -0700)]
context: add AfterFunc
Add an AfterFunc function, which registers a function to run after
a context has been canceled.
Add support for contexts that implement an AfterFunc method, which
can be used to avoid the need to start a new goroutine watching
the Done channel when propagating cancellation signals.
Change-Id: I36a2f2172cf30d97a5aa6f8d7cf6981d67daec62
Reviewed-on: https://go-review.googlesource.com/c/go/+/486235
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
Dmitri Shuralyov [Mon, 17 Apr 2023 21:37:37 +0000 (17:37 -0400)]
make.{bash,bat}: check unmodified $PATH for $GOROOT/bin presence
Previously, all.bash and all.bat restored the original $PATH before
calling 'dist banner', so that it would do its job of checking whether
the user still needs to add $GOROOT/bin to their $PATH. That worked for
those scripts, but had no effect on make.bash nor make.bat.
Instead of trying to extend that logic to more scripts, change the
approach to provide dist an unmodified copy of $PATH via an internal
to dist environment variable $DIST_UNMODIFIED_PATH. The make.bash and
make.bat scripts happen to use dist env -p to modify $PATH, making it
viable to add the internal variable there instead of in each script.
It currently works by adding semicolon terminators to dist env output
so that make.bash's 'eval $(dist env -p)' works as before but is able to
export DIST_UNMODIFIED_PATH for following dist invocations to observe.
Nothing needs to be done for Windows since its 'set ENV=val' format
already has that effect.
Plan 9 doesn't use the -p flag of dist env, and checks that GOROOT/bin
is bound before /bin rather than looking at the $PATH env var like other
OSes, so it may not have this bug. I don't have easy access to Plan 9
and haven't tried to confirm.
Fixes #42563.
Change-Id: I74691931167e974a930f7589d22a48bb6b931163
Reviewed-on: https://go-review.googlesource.com/c/go/+/485896 Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Michael Anthony Knyszek [Fri, 23 Sep 2022 16:32:34 +0000 (16:32 +0000)]
runtime: manage huge pages explicitly
This change makes it so that on Linux the Go runtime explicitly marks
page heap memory as either available to be backed by hugepages or not
using heuristics based on density.
The motivation behind this change is twofold:
1. In default Linux configurations, khugepaged can recoalesce hugepages
even after the scavenger breaks them up, resulting in significant
overheads for small heaps when their heaps shrink.
2. The Go runtime already has some heuristics about this, but those
heuristics appear to have bit-rotted and result in haphazard
hugepage management. Unlucky (but otherwise fairly dense) regions of
memory end up not backed by huge pages while sparse regions end up
accidentally marked MADV_HUGEPAGE and are not later broken up by the
scavenger, because it already got the memory it needed from more
dense sections (this is more likely to happen with small heaps that
go idle).
In this change, the runtime uses a new policy:
1. Mark all new memory MADV_HUGEPAGE.
2. Track whether each page chunk (4 MiB) became dense during the GC
cycle. Mark those MADV_HUGEPAGE, and hide them from the scavenger.
3. If a chunk is not dense for 1 full GC cycle, make it visible to the
scavenger.
4. The scavenger marks a chunk MADV_NOHUGEPAGE before it scavenges it.
This policy is intended to try and back memory that is a good candidate
for huge pages (high occupancy) with huge pages, and give memory that is
not (low occupancy) to the scavenger. Occupancy is defined not just by
occupancy at any instant of time, but also occupancy in the near future.
It's generally true that by the end of a GC cycle the heap gets quite
dense (from the perspective of the page allocator).
Because we want scavenging and huge page management to happen together
(the right time to MADV_NOHUGEPAGE is just before scavenging in order to
break up huge pages and keep them that way) and the cost of applying
MADV_HUGEPAGE and MADV_NOHUGEPAGE is somewhat high, the scavenger avoids
releasing memory in dense page chunks. All this together means the
scavenger will now more generally release memory on a ~1 GC cycle delay.
Notably this has implications for scavenging to maintain the memory
limit and the runtime/debug.FreeOSMemory API. This change makes it so
that in these cases all memory is visible to the scavenger regardless of
sparseness and delays the page allocator in re-marking this memory with
MADV_NOHUGEPAGE for around 1 GC cycle to mitigate churn.
The end result of this change should be little-to-no performance
difference for dense heaps (MADV_HUGEPAGE works a lot like the default
unmarked state) but should allow the scavenger to more effectively take
back fragments of huge pages. The main risk here is churn, because
MADV_HUGEPAGE usually forces the kernel to immediately back memory with
a huge page. That's the reason for the large amount of hysteresis (1
full GC cycle) and why the definition of high density is 96% occupancy.
Fixes #55328.
Change-Id: I8da7998f1a31b498a9cc9bc662c1ae1a6bf64630
Reviewed-on: https://go-review.googlesource.com/c/go/+/436395 Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
os: check for ErrNotExist instead of ErrExist in TestNonPollable
An apparent typo in CL 484837 caused the test to check for ErrExist
instead of ErrNotExist when opening /dev/net/tun for read. That causes
the test to fail on platforms where /dev/net/ton does not exist,
such as on the darwin-amd64-longtest builder.
Robert Griesemer [Tue, 18 Apr 2023 23:43:11 +0000 (16:43 -0700)]
cmd/compile/internal/types2: only mark variables as used if they are
Marking variables in erroneous variable declarations as used is
convenient for tests but doesn't necessarily hide follow-on errors
in real code: either the variable is not supposed to be declared in
the first place and then we should get an error if it is not used,
or it is there because it is intended to be used, and the we expect
an error it if is not used.
This brings types2 closer to go/types.
Change-Id: If7ee1298fc770f7ad0cefe7e968533fd50ec2343
Reviewed-on: https://go-review.googlesource.com/c/go/+/486175
Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Robert Griesemer [Tue, 18 Apr 2023 22:10:46 +0000 (15:10 -0700)]
go/types, types2: don't panic for invalid assignments of comma-ok expressions
The relevant code was broken with CL 478218. Before that CL,
Checker.assignVar used to return the assigned type, or nil,
in case of failure. Checker.recordCommaOkTypes used to take
two types (not two operands), and if one of those types was
nil, it would simply not record. CL 478218, lost that (nil)
signal.
This change consistently reports an assignment check failure
by setting x.mode to invalid for initVar and assignVar and
then tests if x.mode != invalid before recording a comma-ok
expression.
Fixes #59371.
Change-Id: I193815ff3e4b43e3e510fe25bd0e72e0a6a816c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/486135 Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Ian Lance Taylor [Tue, 18 Apr 2023 21:48:36 +0000 (14:48 -0700)]
net: check for NUL bytes in strings passed to C functions
Use syscall.BytePtrFromString and syscall.ByteSliceFromString.
Change-Id: I9409ecd93aaca82390bf3f34be56ec354148a241
Reviewed-on: https://go-review.googlesource.com/c/go/+/486015
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Michael Pratt [Mon, 17 Apr 2023 18:41:10 +0000 (14:41 -0400)]
runtime: consolidate function descriptor definitions on PPC64
This reapplies CL 481075, which was a reappliation of CL 478917.
This CL has been reverted twice now due to conflicts with CL 392854 /
CL 481061, which had bugs and had to be reverted.
Now this CL skips the conflicting changes to runtime/cgo/asm_ppc64x.s,
which will be merged directly into a new version of CL 392854 /
CL 481061. That way, if there are _more_ issues, this CL need not be
involved in any more reverts.
Change-Id: I2801b918faf9418dd0edff19f2a63f4d9e08896c
Reviewed-on: https://go-review.googlesource.com/c/go/+/485335
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Change-Id: I48507c96902e1f83a174e5647b5cc403d965b52b
Reviewed-on: https://go-review.googlesource.com/c/go/+/473256
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
Ian Lance Taylor [Fri, 3 Mar 2023 19:42:07 +0000 (11:42 -0800)]
internal/zstd: new internal package for zstd decompression
This package only does zstd decompression, which is starting to
be used for ELF debug sections. If we need zstd compression we
should use github.com/klauspost/compress/zstd. But for now that
is a very large package to vendor into the standard library.
For #55107
Change-Id: I60ede735357d491be653477ed419cf5f2f0d3f71
Reviewed-on: https://go-review.googlesource.com/c/go/+/473356 Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Ian Lance Taylor [Fri, 14 Apr 2023 22:34:10 +0000 (15:34 -0700)]
runtime: add and use pollDesc fd sequence field
It is possible for a netpoll file to be closed and for the pollDesc
to be reused while a netpoll is running. This normally only causes
spurious wakeups, but if there is an error on the old file then the
new file can be incorrectly marked as having an error.
Fix this problem on most systems by introducing an fd sequence field
and using that as a tag in a taggedPointer. The taggedPointer is
stored in epoll or kqueue or whatever is being used. If the taggedPointer
returned by the kernel has a tag that does not match the fd
sequence field, the notification is for a closed file, and we
can ignore it. We check the tag stored in the pollDesc, and we also
check the tag stored in the pollDesc.atomicInfo.
This approach does not work on 32-bit systems where the kernel
only provides a 32-bit field to hold a user value. On those systems
we continue to use the older method without the sequence protection.
This is not ideal, but it is not an issue on Linux because the kernel
provides a 64-bit field, and it is not an issue on Windows because
there are no poller errors on Windows. It is potentially an issue
on *BSD systems, but on those systems we already call fstat in newFile
in os/file_unix.go to avoid adding non-pollable files to kqueue.
So we currently don't know of any cases that will fail.
Fixes #59545
Change-Id: I9a61e20dc39b4266a7a2978fc16446567fe683ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/484837
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Orlando Labao <orlando.labao43@gmail.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Bypass: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
Michael Matloob [Mon, 17 Apr 2023 20:42:18 +0000 (16:42 -0400)]
cmd/go: skip over all workspace modules in go mod verify
This was a remaining place where we made the assumption that there is
only one workspace module. So we'd only skip the first workspace
module when running go mod verify. Instead skip over the first
MainModules.Len() modules of the buildlist, which are all the main
modules.
Fixes #54372
Change-Id: Ife687c907ae4326759c43cc35f78d429d5113b19
Reviewed-on: https://go-review.googlesource.com/c/go/+/485475
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
Michael Matloob [Tue, 18 Apr 2023 16:16:43 +0000 (12:16 -0400)]
src/internal/godebugs: add a skip for missing godebug.md
Currently android doesn't include godebug.md in its doc folder, and
TestAll in godebugs_test.go is failing because it can't open the file.
Add a skip in case the file is missing (except for linux so we can
catch the case where we stop generating the file).
Michael Matloob [Tue, 18 Apr 2023 15:20:58 +0000 (11:20 -0400)]
cmd/go: stub out gotoolchain.go for wasip1 os build tag
There's a stub for gotoolchain.go for the js build tag because js/wasm
doesn't define syscall.Exec. But there are builders that are wasm but
not js, which also don't have syscall.Exec. The wasip1 GOOS is one
example. Stub out gotoolchain.go for wasip1 also.
Change-Id: I224bb385474ad9c5d3c28a83a000f450dfb43c0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/485735
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Archana R [Thu, 23 Mar 2023 17:35:39 +0000 (12:35 -0500)]
runtime: improve index on ppc64x/power10
Rewrite index asm function to use the new power10 instruction lxvl,
stxvl or the load, store vector with length which can specify the
number of bytes to be stored in a register. This avoids the need to
create a separator mask and extra AND instructions. It also allows
us to process the tail end of the string using a lot fewer instructions
as we can load bytes of separator length directly rather than loading
16 bytes and masking out bytes that are greater than separator length
On power9 and power8 the code remains unchanged.
The performance for smaller sizes improve the most, on larger sizes
we see minimal improvement.
Matthieu Baerts [Fri, 24 Feb 2023 16:52:00 +0000 (17:52 +0100)]
net: mptcp: add TCPConn's MultipathTCP checker
This new TCPConn method returns whether the connection is using MPTCP or
if a fallback to TCP has been done, e.g. because the other peer doesn't
support MPTCP.
When working on the new E2E test linked to MPTCP (#56539), it looks like
the user might need to know such info to be able to do some special
actions (report, stop, etc.). This also improves the test to make sure
MPTCP has been used as expected.
Regarding the implementation, from kernel version 5.16, it is possible
to use:
getsockopt(..., SOL_MPTCP, MPTCP_INFO, ...)
and check if EOPNOTSUPP (IPv4) or ENOPROTOOPT (IPv6) is returned. If it
is, it means a fallback to TCP has been done. See this link for more
details:
Before v5.16, there is no other simple way, from the userspace, to check
if the created socket did a fallback to TCP. Netlink requests could be
done to try to find more details about a specific socket but that seems
quite a heavy machinery. Instead, only the protocol is checked on older
kernels.
The E2E test has been modified to check that the MPTCP connection didn't
do any fallback to TCP, explicitely validating the two methods
(SO_PROTOCOL and MPTCP_INFO) if it is supported by the host.
This work has been co-developed by Gregory Detal
<gregory.detal@tessares.net> and Benjamin Hesmans
<benjamin.hesmans@tessares.net>.
Fixes #59166
Change-Id: I5a313207146f71c66c349aa8588a2525179dd8b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/471140
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
Oleksandr Redko [Mon, 17 Apr 2023 09:23:36 +0000 (12:23 +0300)]
plugin: fix duplicated word in comment
Change-Id: Ia3174d079e84cf874c2f2f3093a7c6337af32b02
Reviewed-on: https://go-review.googlesource.com/c/go/+/485015
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Russ Cox [Tue, 14 Mar 2023 18:25:56 +0000 (14:25 -0400)]
cmd/go: add check for unknown godebug setting
A //go:debug line mentioning an unknown or retired setting
should be diagnosed as making the program invalid. Do that.
We agreed on this in the proposal but I forgot to implement it.
Russ Cox [Fri, 10 Mar 2023 19:02:32 +0000 (14:02 -0500)]
cmd/go: change toolchain based on $GOTOOLCHAIN
For proposal #57001, add code to reinvoke a different Go toolchain
based on $GOTOOLCHAIN. The toolchain is searched for in $PATH
first and otherwise downloaded. The download is a standard module
download, so the toolchain is validated using the checksum database
before being executed or even stored in the file system.
Followup CLs will refine the exact toolchain selection and implement
other parts of the proposal. This is only the download+reinvoke code.
Michael Pratt [Mon, 17 Apr 2023 18:13:06 +0000 (14:13 -0400)]
Revert "runtime/cgo: store M for C-created thread in pthread key"
This reverts CL 481061.
Reason for revert: When built with C TSAN, x_cgo_getstackbound triggers
race detection on `g->stacklo` because the synchronization is in Go,
which isn't instrumented.
For #51676.
For #59294.
For #59678.
Change-Id: I38afcda9fcffd6537582a39a5214bc23dc147d47
Reviewed-on: https://go-review.googlesource.com/c/go/+/485275
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
Than McIntosh [Tue, 4 Apr 2023 22:31:46 +0000 (18:31 -0400)]
cmd/compile: allow more inlining of functions that construct closures
[This is a roll-forward of CL 479095, which was reverted due to a bad
interaction between inlining and escape analysis, then later fixed
fist with an attempt in CL 482355, then again in 484859 .]
Currently, when the inliner is determining if a function is
inlineable, it descends into the bodies of closures constructed by
that function. This has several unfortunate consequences:
- If the closure contains a disallowed operation (e.g., a defer), then
the outer function can't be inlined. It makes sense that the
*closure* can't be inlined in this case, but it doesn't make sense
to punish the function that constructs the closure.
- The hairiness of the closure counts against the inlining budget of
the outer function. Since we currently copy the closure body when
inlining the outer function, this makes sense from the perspective
of export data size and binary size, but ultimately doesn't make
much sense from the perspective of what should be inlineable.
- Since the inliner walks into every closure created by an outer
function in addition to starting a walk at every closure, this adds
an n^2 factor to inlinability analysis.
This CL simply drops this behavior.
In std, this makes 57 more functions inlinable, and disallows inlining
for 10 (due to the basic instability of our bottom-up inlining
approach), for an net increase of 47 inlinable functions (+0.6%).
This will help significantly with the performance of the functions to
be added for #56102, which have a somewhat complicated nesting of
closures with a performance-critical fast path.
The downside of this seems to be a potential increase in export data
and text size, but the practical impact of this seems to be
negligible:
Than McIntosh [Fri, 14 Apr 2023 18:07:37 +0000 (14:07 -0400)]
cmd/compile: rework marking of dead hidden closure functions
This patch generalizes the code in the inliner that marks unreferenced
hidden closure functions as dead. Rather than doing the marking on the
fly (previous approach), this new approach does a single pass at the
end of inlining, which catches more dead functions.
Fixes #59638.
Updates #59404.
Updates #59547.
Change-Id: I54fd63e9e37c9123b08a3e7def7d1989919bba91
Reviewed-on: https://go-review.googlesource.com/c/go/+/484859 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Robert Griesemer [Thu, 13 Apr 2023 23:11:14 +0000 (16:11 -0700)]
go/types, types2: factor out type parameter renaming from type inference
Preparation for reverse type inference where there is no need
to rename all type parameters supplied to type inference when
passing generic functions as arguments to (possibly generic)
function calls.
This also leads to a better separation of concerns.
Change-Id: Id487a5c1340b743519b9053edc43f8aa99408522
Reviewed-on: https://go-review.googlesource.com/c/go/+/484655
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
Cuong Manh Le [Mon, 20 Feb 2023 05:49:03 +0000 (12:49 +0700)]
cmd/compile: remove typecheck.EvalConst
types2 has already done most of the constant folding parts. The only
case left is unsafe.{Alignoff,Offsetof,Sizeof} with variable size
argument, which is handled separately during typecheck.
Change-Id: I8050b7613a16b19b91751726ac07253333177f73
Reviewed-on: https://go-review.googlesource.com/c/go/+/469595 Reviewed-by: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Joel Sing [Thu, 13 Apr 2023 19:47:36 +0000 (05:47 +1000)]
cmd/link/internal/ld: disable execute-only for external linking on openbsd/arm64
The Go arm64 assembler places constants into the text section of a binary.
OpenBSD 7.3 enabled xonly by default on OpenBSD/arm64. This means that any
externally linked Go binary now segfaults. Disable execute-only when invoking
the external linker on openbsd/arm64, in order to work around this issue.
Updates #59615
Change-Id: I1a291293da3c6e4409b21873d066ea15e9bfe280
Reviewed-on: https://go-review.googlesource.com/c/go/+/484555
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Aaron Bieber <deftly@gmail.com>
Run-TryBot: Joel Sing <joel@sing.id.au> Reviewed-by: Than McIntosh <thanm@google.com>
Rob Findley [Fri, 14 Apr 2023 18:25:53 +0000 (14:25 -0400)]
go/types,types2: fix panic in reverse type inference when -lang<go1.18
Due to reverse type inference, we may not have an index expression when
type-checking a function instantiation. Fix a panic when the index expr
is nil.
Joel Sing [Wed, 5 Apr 2023 20:11:01 +0000 (06:11 +1000)]
cmd/compile: add math benchmarks
This adds benchmarks for division and modulus of 64 bit signed and unsigned
integers.
Updates #59089
Change-Id: Ie757c6d74a1f355873e79619eae26ece21a8f23e
Reviewed-on: https://go-review.googlesource.com/c/go/+/482656 Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
cmd/compile: stop constructing untyped nodes when instrumenting asan
The code is using typecheck.ConvNop to convert from untyped int to
uintptr. However, that left the literal node untyped. It often does not
matter, because typecheck.EvalConst will see the OCONVNOP, and replace
the node with a new constant node.
This CL changes the code to construct the constant node directly using
typecheck.DefaultLit, so the last dependecy of typecheck.EvalConst will
go away, next CL can safely remove it from the code base.
Change-Id: Ie5a3d1ff6d3b72be7b8c43170eaa4f6cbb3206fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/484317 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
When this function get inlined, there is a check whether we can delay
declaring the result parameter until the "return" statement. For the
original function, we can't delay the result, because there's more than
one return statement. However, the constant-fold one can, because
there's on one return statement in the body now. The result parameter
~R0 ends up declaring inside the switch statement scope.
Now, when walking the switch statement, it's re-written into if-else
statement. Without typecheck.EvalConst, the if condition "if 64 == 64"
is passed as-is to the ssa generation pass. Because "64 == 64" is not a
constant, the ssagen creates normal blocks for branching the results.
This confuses the liveness analysis, because ~R0 is only live inside the
if block. With typecheck.EvalConst, "64 == 64" is evaluated to "true",
so ssagen can branch the result without emitting conditional blocks.
Instead, the constant-fold can be re-written as:
switch {
case true:
// Body
}
So it does not depend on the delay results check during inlining. Adding
a test, which will fail when typecheck.EvalConst is removed, so we can
do the cleanup without breaking things.
Change-Id: I638730bb147140de84260653741431b807ff2f15
Reviewed-on: https://go-review.googlesource.com/c/go/+/484316 Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Cuong Manh Le [Wed, 8 Feb 2023 08:45:09 +0000 (15:45 +0700)]
cmd/compile: handle string concatenation in static init inliner
Static init inliner is using typecheck.EvalConst to handle string
concatenation expressions. But static init inliner may reveal constant
expressions after substitution, and the compiler needs to evaluate those
expressions in non-constant semantic. Using typecheck.EvalConst, which
always evaluates expressions in constant semantic, is not the right
choice.
For safety, this CL fold the logic to handle string concatenation to
static init inliner, so there won't be regression in handling constant
expressions in non-constant semantic. And also, future CL can simplify
typecheck.EvalConst logic.
Updates #58293
Updates #58339
Fixes #58439
Change-Id: I74068d99c245938e576afe9460cbd2b39677bbff
Reviewed-on: https://go-review.googlesource.com/c/go/+/466277 Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Daniel Martí [Thu, 13 Apr 2023 12:17:26 +0000 (13:17 +0100)]
regexp: fix copy-paste typo on Regexp.UnmarshalText doc
I noticed that https://go.dev/cl/479401 called both methods MarshalText
in the godoc, so fix that.
While here, add more godoc links for better usability.
Change-Id: I8f10bafeca6a1ca1c1ed9be7a7dd9fdecfe991a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/484335
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: David Chase <drchase@google.com>
Russ Cox [Mon, 13 Mar 2023 17:22:34 +0000 (13:22 -0400)]
cmd/go: make go line declare minimum required Go version
For #57001, enforce the go line as declaring the minimum required
version of Go that can compile a module.
Modules that maintain compatibility with old versions of Go
but want to make use of new features in //go:build-constrained files
will be able to do so: the //go:build constraint will be interpreted
as changing the minimum Go version for that file and will unlock
the Go features allowed in that version.
Russ Cox [Tue, 14 Mar 2023 01:06:49 +0000 (21:06 -0400)]
go/types, cmd/compile/internal/types2: use per-file Go version
For #57001, compilers and others tools will need to understand that
a different Go version can be used in different files in a program,
according to the //go:build lines in those files.
Update go/types and cmd/compile/internal/types2 to track and
use per-file Go versions. The two must be updated together because
of the files in go/types that are generated from files in types2.
The effect of the //go:build go1.N line depends on the Go version
declared in the 'go 1.M' line in go.mod. If N > M, the file gets go1.N
semantics when built with a Go 1.N or later toolchain
(when built with an earlier toolchain the //go:build line will keep
the file from being built at all).
If N < M, then in general we want the file to get go1.N semantics
as well, meaning later features are disabled. However, older Go 1.M
did not apply this kind of downgrade, so for compatibility, N < M
only has an effect when M >= 21, meaning when using semantics
from Go 1.21 or later.
Daniel Frederick Crisman [Thu, 13 Apr 2023 21:34:31 +0000 (21:34 +0000)]
doc: correct spelling on placeholder
"placeholder" (no space) is already used in the spec and seems to be
the preferred option.
Removed space from "place holder".
Change-Id: I9b98f62f0e3f5adb019b99f5271cc9d19abf505e
GitHub-Last-Rev: ed5aaf9d02c294e87688066f6218e5d58b0f62bf
GitHub-Pull-Request: golang/go#59626
Reviewed-on: https://go-review.googlesource.com/c/go/+/484576 Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Michael Matloob [Wed, 12 Apr 2023 23:00:30 +0000 (19:00 -0400)]
cmd/go: parallelize part of loading test packages in list
load.TestPackagesAndErrors is given an optional done func() argument.
If set, load.TestPackagesAndErrors will perform part of its work
asynchronously and call done when done. This allows go list to run
testPackagesAndErrors so that the parallelizable parts of
TestPackagesAndErrors run in parallel, making go list -e faster.
wasm: restrict supported types in go:wasmimport function signatures
Changes the set of types supported in functions declared with the
go:wasmimport directive to only allow 32 bits and 64 bits integers
and floats, as well as unsafe.Pointer in parameters only. Both the
compiler code and the standard library are updated because the new
restrictions require modifying the use of go:wasmimport in the
syscall and runtime packages.
In preparation of enabling packages outside of the standard library
to use the go:wasmimport directive, the error messages are modified
to carry more context and use ErrorfAt instead of Fatalf to avoid
printing the compiler stack trace when a function with an invalid
signature is encountered.
Fixes #59156
Change-Id: Ied8317f8ead9c28f0297060ac35a5b5255ab49db
Reviewed-on: https://go-review.googlesource.com/c/go/+/483415 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Russ Cox [Mon, 13 Mar 2023 21:35:03 +0000 (17:35 -0400)]
go/parser: report //go:build-derived Go version in ast.File.GoVersion
For #57001, compilers and others tools will need to understand that
a different Go version can be used in different files in a program,
according to the //go:build lines in those files.
Update go/parser to populate the new ast.File.GoVersion field.
This requires running the go/scanner in ParseComments mode
always and then implementing discarding of comments in the
parser instead of the scanner. The same work is done either way,
since the scanner was already preparing the comment result
and then looping. The loop has just moved into go/parser.
Also make the same changes to cmd/compile/internal/syntax,
both because they're necessary and to keep in sync with go/parser.
Ian Lance Taylor [Wed, 12 Apr 2023 20:57:21 +0000 (13:57 -0700)]
slices: amortize allocations in Insert
Fixes #54948
Change-Id: I467afb940b539b100dcce687b05914a9da7b9ed2
Reviewed-on: https://go-review.googlesource.com/c/go/+/484159
Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Valentin Deleplace <deleplace@google.com>
path/filepath: add test cases for walking a symlink-to-symlink-to-dir
The "double link with slash" test is skipped on darwin due to an
apparent kernel / libc bug. If the bug is present on other platforms
too, I'd like to know about it.
Reason for revert: it is trickier than expected to enforce an invariant that x.typ == Typ[Invalid] => x.mode == invalid. For example, builtins have invalid type until their call is evaluated.
I think it is better to keep this defensive code for now. My bad for suggesting this strictness. I will send a follow-up CL with a test that exercises the panic discovered inside Google, and a bit more commentary about what 'invalid' means in both contexts.
Fixes #59603
Change-Id: If291f7268e7ef7ae6cd9bb861bb9af349a729cb2
Reviewed-on: https://go-review.googlesource.com/c/go/+/484375
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
Robert Griesemer [Wed, 12 Apr 2023 16:43:41 +0000 (09:43 -0700)]
go/types, types2: generalize tparamIndex to arbitrary type parameter lists
tparamIndex returns the index of a type parameter given the type
parameter and a list of type parameters. If an index >= 0 is returned,
it is the index assigned to the type parameter (TypeParam.index), and
the index of the type parameter in the provided list of parameters.
For it to work correctly, the type parameter list must be from a single
type parameter declaration.
To allow for lists of arbitrary type parameters (from different generic
signatures), change the implementation to do a linear search. The result
is the index of the type parameter in the provided type parameter list,
which may be different from the index assigned to the type parameter.
The linear search is likely fast enough since type parameter lists tend
to be very short.
Change-Id: I913f97fa4c042abeb535ee86ca6657241a4cf796
Reviewed-on: https://go-review.googlesource.com/c/go/+/483995 Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
cmd/internal/moddeps: preserve PWD more carefully in commands
On macOS, TMPDIR is typically a symlink, and the GOROOT for the
buildlet is in TMPDIR as well. PWD must be preserved in order for
os.Getwd (and functions based on it) to report paths that remain
relative to GOROOT, and paths relative to GOROOT are necessary in
order for filepath.Rel to report subdirectories as subdirectories
(rather than paths with long "../../…" prefixes).
Fortunately, the (*Cmd).Environ method added for #50599 makes
preserving PWD somewhat easier.
This fixes 'go test cmd/internal/moddeps' on the new
darwin-amd64-longtest builder.
cmd/go: skip TestScript/list_goroot_symlink on darwin
The list_goroot_symlink test relies on fsys.Walk (and ultimately
syscall.Lstat) conforming to POSIX pathname resolution semantics.
POSIX requires that symlinks ending in a slash be fully resolved,
but it appears that lstat in current darwin kernels does not fully
resolve the last pathname component when it is a symlink to a symlink.
Matthew Dempsky [Wed, 12 Apr 2023 19:41:45 +0000 (12:41 -0700)]
cmd/compile: fix reproducible build of aliased generic types
Due to a missing "&& !alias" check, the unified linker was treating
type aliases the same as defined types for the purpose of exporting
method bodies. The methods will get exported anyway alongside the
aliased type, so this mistake is normally harmless.
However, if multiple type aliases instantiated the same generic type
but with different type arguments, this could result in the
same (generic) method body being exported multiple times under
different symbol names. Further, because bodies aren't expected to be
exported multiple times, we were sorting them simply based on index.
And consequently, the sort wasn't total and is sensitive to the map
iteration order used while ranging over linker.bodies.
The fix is simply to add the missing "&& !alias" check, so that we
don't end up with duplicate bodies in the first place.
Thanks rsc@ for providing a minimal repro case.
Fixes #59571.
Change-Id: Iaa55968cc7110b601e2f0f9b620901c2d55f7014
Reviewed-on: https://go-review.googlesource.com/c/go/+/484155 Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Jonathan Amsterdam [Wed, 12 Apr 2023 14:07:35 +0000 (10:07 -0400)]
log/slog: catch panics in LogValue
If a LogValue call panics, recover and return an error instead.
The error contains some stack information to make it easier to
find the problem. A number of people complained that panics
in fmt.Formatter.Format functions are hard to debug because
there is no context.
This is an example of the error text:
LogValue panicked
called from log/slog.panickingLogValue.LogValue (/usr/local/google/home/jba/repos/go/src/log/slog/value_test.go:221)
called from log/slog.Value.resolve (/usr/local/google/home/jba/repos/go/src/log/slog/value.go:465)
called from log/slog.Value.Resolve (/usr/local/google/home/jba/repos/go/src/log/slog/value.go:446)
called from log/slog.TestLogValue (/usr/local/google/home/jba/repos/go/src/log/slog/value_test.go:192)
called from testing.tRunner (/usr/local/google/home/jba/repos/go/src/testing/testing.go:1595)
(rest of stack elided)
Fixes #59141.
Change-Id: I62e6ff6968d1aa34873e955c2d606d25418a673b
Reviewed-on: https://go-review.googlesource.com/c/go/+/484097
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
limeidan [Thu, 25 Aug 2022 03:14:15 +0000 (11:14 +0800)]
cmd/internal/sys, cmd/dist, misc/cgo/testcshared: enable c-shared feature and test on loong64
Linux kernel on loong64 has no Dup2 syscall support, so we use Dup3 to replace it like arm64 and riscv64.
Updates #53301
Fixes #58784
Change-Id: I4e0be140a71b86f4626ed39d76cf3ac78f842018
Reviewed-on: https://go-review.googlesource.com/c/go/+/425478
Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: WANG Xuerui <git@xen0n.name> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: abner chenc <chenguoqi@loongson.cn>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: xiaodong liu <teaofmoli@gmail.com> Reviewed-by: WANG Xuerui <git@xen0n.name>