Dmitri Goutnik [Wed, 8 Jun 2022 15:55:42 +0000 (10:55 -0500)]
cmd/go: enable -msan on freebsd/amd64
Enable -msan flag on freebsd/amd64 and amend PIE comment in
internal/work/init.go to indicate that MSAN requires PIE on all platforms
except linux/amd64.
R=go1.20
For #53298
Change-Id: I93d94efa95d7f292c23c433fb1d3f4301d820bde
Reviewed-on: https://go-review.googlesource.com/c/go/+/411275 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
Bryan C. Mills [Wed, 12 Oct 2022 18:57:47 +0000 (14:57 -0400)]
os: split wait6 syscall wrapper into per-platform files
There are getting to be enough special cases in this wrapper that
the increase in clarity from having a single file is starting to be
outweighed by the complexity from chained conditionals.
Dmitri Goutnik [Wed, 15 Jun 2022 19:50:19 +0000 (14:50 -0500)]
cmd/link: define ELF .note section on FreeBSD
Write .note signature section when targeting FreeBSD, similar to NetBSD
and OpenBSD. This allows binaries to declare the ABI version they were
compiled for and opt out of ASLR when compiled with -race.
Fixes #48164
Change-Id: Ie54dd5c70697a3f42a75fd640540350fd8a4dc71
Reviewed-on: https://go-review.googlesource.com/c/go/+/412494 Reviewed-by: Meng Zhuo <mzh@golangcn.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com>
Michael Matloob [Wed, 5 Oct 2022 23:06:23 +0000 (19:06 -0400)]
cmd/go: avoid setting mod=vendor in workspace mode
Workspaces with a single module would enter mod=vendor mode even when
in workspace mode. Fix that by explicitly checking that we're not in
workspace mode when deciding whether to enter vendor mode.
Bryan C. Mills [Wed, 12 Oct 2022 19:11:16 +0000 (15:11 -0400)]
os/exec: reduce arbitrary sleeps in TestWaitid
If we use the "pipetest" helper command instead of "sleep",
we can use its stdout pipe to determine when the process
is ready to handle a SIGSTOP, and we can additionally check
that sending a SIGCONT actually causes the process to continue.
This also allows us to remove the "sleep" helper command,
making the test file somewhat more concise.
Cherry Mui [Wed, 12 Oct 2022 20:24:34 +0000 (16:24 -0400)]
cmd/link: don't reset variable size when handling -X flag
The linker's -X flag allows setting/changing a string variable's
content at link time. Currently it resets its size then write a
new string header pointing to the new content. This mostly works.
But under ASAN build the string variable can have larger size
than the usual 2 words, due to the red zone. Resetting the size
can cause the variable to "overlap" (in ASAN's view) with other
variables. Don't reset the size.
Fixes #56175.
Change-Id: Ib364208201a7a2fd7f44f9b1797834198736a405
Reviewed-on: https://go-review.googlesource.com/c/go/+/442635
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
Roland Shoemaker [Wed, 28 Sep 2022 21:52:33 +0000 (14:52 -0700)]
debug: add top level security docs for dwarf, elf, macho, pe, and plan9obj
Adds a package level doc comment to the debug/dwarf, debug/elf,
debug/macho, debug/pe, and debug/plan9obj noting that these packages
are not designed to be hardened against adversarial inputs.
Change-Id: I678d01bcdc8ad01c23805f09cc59e64cec6c3f76
Reviewed-on: https://go-review.googlesource.com/c/go/+/435417 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Park Zhou [Sat, 8 Oct 2022 17:00:39 +0000 (17:00 +0000)]
crypto/x509: add blank line before package declaration
The package doc included the copying header by mistake.
Change-Id: I37ac2d14b1b8a389e6b603fbb1c6a3a33b1d80d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/440456 Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Joedian Reid <joedian@golang.org>
Robert Findley [Sat, 13 Aug 2022 16:35:57 +0000 (12:35 -0400)]
go/types, types2: optimize instance lookup in LookupFieldOrMethod
LookupFieldOrMethod appears as a hotspot when benchmarking gopls'
auto-completion. In particular, instanceLookup.add was allocating in the
common case of structs with no embedding.
This is easily fixed, by using a small array in front of the map inside
of instanceLookup. Do this, and additionally add a microbenchmark.
The benchmark improvement is significant:
name old time/op new time/op delta
LookupFieldOrMethod-12 388µs ± 6% 154µs ± 3% -60.36% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
LookupFieldOrMethod-12 152kB ± 0% 2kB ± 0% -98.77% (p=0.000 n=9+10)
name old allocs/op new allocs/op delta
LookupFieldOrMethod-12 1.41k ± 0% 0.07k ± 0% -95.38% (p=0.000 n=10+10)
It should also be noted that instanceLookup is used elsewhere, in
particular by validType. In those contexts, the scope is not just the
current type but the entire package, and so the newly added buffer is
likely to simply cause extra Identical checks. Nevertheless, those
checks are cheap, and on balance the improved LookupFieldOrMethod
performance leads overall to improved type-checking performance.
Standard library benchmark results varied by package, but type checking
speed for many packages improved by ~5%, with allocations improved by
~10%. If this weren't the case we could let the caller control the
buffer size, but that optimization doesn't seem necessary at this time.
Ian Lance Taylor [Mon, 10 Oct 2022 21:07:10 +0000 (14:07 -0700)]
debug/elf: validate phentsize and shentsize
No test case because the problem can only happen for invalid data. Let
the fuzzer find cases like this.
Fixes #56129
Change-Id: I6c81933781384c5e2c8ba0fd99cec50455b9664a
Reviewed-on: https://go-review.googlesource.com/c/go/+/441976 Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Joedian Reid <joedian@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 11 Oct 2022 22:09:02 +0000 (15:09 -0700)]
os/signal: document behavior of SIGPIPE on non-Go thread
Fixes #56150
Change-Id: Id990783562950ba8be7ce9526b7a811625f2190a
Reviewed-on: https://go-review.googlesource.com/c/go/+/442415
Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Michael Anthony Knyszek [Fri, 12 Aug 2022 23:25:56 +0000 (23:25 +0000)]
arena: add experimental arena package
This change adds the arenas package and a function to reflect for
allocating from an arena via reflection, but all the new API is placed
behind a GOEXPERIMENT.
For #51317.
Change-Id: I026d46294e26ab386d74625108c19a0024fbcedc
Reviewed-on: https://go-review.googlesource.com/c/go/+/423361 Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Michael Anthony Knyszek [Fri, 12 Aug 2022 21:40:46 +0000 (21:40 +0000)]
runtime: add safe arena support to the runtime
This change adds an API to the runtime for arenas. A later CL can
potentially export it as an experimental API, but for now, just the
runtime implementation will suffice.
The purpose of arenas is to improve efficiency, primarily by allowing
for an application to manually free memory, thereby delaying garbage
collection. It comes with other potential performance benefits, such as
better locality, a better allocation strategy, and better handling of
interior pointers by the GC.
This implementation is based on one by danscales@google.com with a few
significant differences:
* The implementation lives entirely in the runtime (all layers).
* Arena chunks are the minimum of 8 MiB or the heap arena size. This
choice is made because in practice 64 MiB appears to be way too large
of an area for most real-world use-cases.
* Arena chunks are not unmapped, instead they're placed on an evacuation
list and when there are no pointers left pointing into them, they're
allowed to be reused.
* Reusing partially-used arena chunks no longer tries to find one used
by the same P first; it just takes the first one available.
* In order to ensure worst-case fragmentation is never worse than 25%,
only types and slice backing stores whose sizes are 1/4th the size of
a chunk or less may be used. Previously larger sizes, up to the size
of the chunk, were allowed.
* ASAN, MSAN, and the race detector are fully supported.
* Sets arena chunks to fault that were deferred at the end of mark
termination (a non-public patch once did this; I don't see a reason
not to continue that).
For #51317.
Change-Id: I83b1693a17302554cb36b6daa4e9249a81b1644f
Reviewed-on: https://go-review.googlesource.com/c/go/+/423359 Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Michael Anthony Knyszek [Tue, 20 Sep 2022 18:33:03 +0000 (18:33 +0000)]
runtime: make (*mheap).sysAlloc more general
This change makes (*mheap).sysAlloc take an explicit list of hints and a
boolean as to whether or not any newly-created heapArenas should be
registered in the full arena list.
This is a refactoring in preparation for arenas.
For #51317.
Change-Id: I0584a033fce3fcb60c5d0bc033d5fb8bd23b2378
Reviewed-on: https://go-review.googlesource.com/c/go/+/432078
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Bryan C. Mills [Wed, 12 Oct 2022 17:29:01 +0000 (13:29 -0400)]
os: use the correct constant for P_PID on NetBSD
Dragonfly and FreeBSD both used numerical values for these constants
chosen to be the same as on Solaris. For some reason, NetBSD did not,
and happens to interpret value 0 as P_ALL instead of P_PID
(see https://github.com/NetBSD/src/blob/3323ceb7822f98b3d2693aa26fd55c4ded6d8ba4/sys/sys/idtype.h#L43-L44).
Using the correct value for P_PID should cause wait6 to wait for the
correct process, which may help to avoid the deadlocks reported in
Bill Zissimopoulos [Wed, 12 Oct 2022 10:39:27 +0000 (10:39 +0000)]
runtime: fix invalid pointer in windows/arm64 badsignal2
Initializes the R3 register with an available address in the stack. The addressed location is used to receive the number of bytes written by WriteFile.
Cuong Manh Le [Tue, 11 Oct 2022 04:56:51 +0000 (11:56 +0700)]
all: prevent fakePC overflow on 386 in libfuzzer mode
fakePC uses hash.Sum32, which returns an uint32. However, libfuzzer
trace/hook functions declare fakePC argument as int, causing overflow on
386 archs.
Fixing this by changing fakePC argument to uint to prevent the overflow.
Fixes #56141
Change-Id: I3994c461319983ab70065f90bf61539a363e0a2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/441996
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Robert Griesemer [Mon, 10 Oct 2022 20:13:14 +0000 (13:13 -0700)]
go/types: add errorcalls_test, apply it, and fix errorf call sites
The errorcalls_test makes sure that we use error instead of errorf
where possible. Copied from types2 and adjusted for go/types.
Change-Id: Ib0572308c87e4415bf89aec8d64e662abe94754b
Reviewed-on: https://go-review.googlesource.com/c/go/+/441958 Reviewed-by: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Michael Matloob [Thu, 6 Oct 2022 20:14:47 +0000 (16:14 -0400)]
cmd/go: set dependency go versions for go install --mod=readonly mod@vers
When running go install --mod=readonly module@version. modfetch.GoSumFile
was not set, so the checksumOk check that's done when checking whether
we need to set the GoVersion from the go mod file was failing. Bypass
the checksumOk check when there's no GoSumFile.
Ian Lance Taylor [Tue, 11 Oct 2022 18:21:13 +0000 (11:21 -0700)]
math/big: error on buffer length overflow in Rat.GobDecode
Fixes #56156
Change-Id: Ib85ff45f0b0d0eac83c39606ee20b3a312e6e919
Reviewed-on: https://go-review.googlesource.com/c/go/+/442335
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Michael Matloob [Fri, 30 Sep 2022 20:49:42 +0000 (16:49 -0400)]
cmd/go: do not exit with non-zero code from go list -e -export
go list -e -export puts errors running build actions on the load.Package
corresponding to the failed action rather than exiting with a non zero
exit code.
Bryan C. Mills [Wed, 5 Oct 2022 18:04:55 +0000 (14:04 -0400)]
os/exec: remove protection against a duplicate Close on StdinPipe
As of CL 438347, multiple concurrents calls to Close should be safe.
This removes some indirection and may also make some programs that use
type-assertions marginally more efficient. For example, if a program
calls (*exec.Cmd).StdinPipe to obtain a pipe and then sets that as the
Stdout of another command, that program will now allow the second
command to inherit the file descriptor directly instead of copying
everything through a goroutine.
This will also cause calls to Close after the first to return an error
wrapping os.ErrClosed instead of nil. However, it seems unlikely that
programs will depend on that error behavior: if a program is calling
Write in a loop followed by Close, then if a racing Close occurs it is
likely that the Write would have already reported an error. (The only
programs likely to notice a change are those that call Close — without
Write! — after a call to Wait.)
Joel Sing [Sun, 4 Sep 2022 17:03:35 +0000 (03:03 +1000)]
cmd/compile: convert SLT/SLTU with constant into immediate form on riscv64
Convert SLT/SLTU with a suitably valued constant into a SLTI/SLTIU instruction.
This can reduce instructions and avoid register loads. Now that we generate
more SLTI/SLTIU instructions, absorb these into branches when it makes sense
to do so.
Removes more than 800 instructions from the Go binary on linux/riscv64.
Change-Id: I42c4e00486697acd4da7669d441b5690795f18ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/428499 Reviewed-by: Wayne Zuo <wdvxdr@golangcn.org>
Run-TryBot: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Joedian Reid <joedian@golang.org>
Ian Lance Taylor [Sat, 8 Oct 2022 18:17:47 +0000 (11:17 -0700)]
test: add test case that caused a bogus error from gofrontend
For #56109
Change-Id: I999763e463fac57732a92f5e396f8fa8c35bd2e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/440297
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Cuong Manh Le [Sat, 8 Oct 2022 05:32:06 +0000 (12:32 +0700)]
cmd/compile: fix missing walk pass for static initialization slice
CL 403995 fixed static init of literal contains dynamic exprs, by
ensuring their init are ordered properly. However, we still need to walk
the generated init codes before appending to parent init. Otherwise,
codes that requires desugaring will be unhandled, causing the compiler
backend crashing.
Fixes #56105
Change-Id: Ic25fd4017473f5412c8e960a91467797a234edfd
Reviewed-on: https://go-review.googlesource.com/c/go/+/440455 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
Than McIntosh [Mon, 10 Oct 2022 17:39:30 +0000 (13:39 -0400)]
cmd/compile: tweak inliners handling of coverage counter updates
This patch fixes up a bug in the inliner's special case code for
coverage counter updates, which was not properly working for
-covermode=atomic compilations.
Updates #56044.
Change-Id: I9e309312b123121c3df02862623bdbab1f6c6a4b
Reviewed-on: https://go-review.googlesource.com/c/go/+/441858 Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Robert Griesemer [Fri, 7 Oct 2022 01:09:23 +0000 (18:09 -0700)]
go/types, types2: use zero error code to indicate unset error code
Use InvalidSyntaxError where the zero error code was used before.
Fix a couple of places that didn't set an error code.
Panic in error reporting if no error code is provided.
Change-Id: I3a537d42b720deb5c351bf38871e04919325e231
Reviewed-on: https://go-review.googlesource.com/c/go/+/439566
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Type checkers should use InvalidSyntaxTree as error code
for invalid syntax tree errors instead of zero. This way
the zero value can be used to mark an unset error code.
Also, add an example for BlankPkgName (and adjust the
test harness slightly to make it work).
Change-Id: Ic15fa0e8e46be698e52352f2f0e4915b75e509d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/439565 Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Robert Griesemer [Thu, 6 Oct 2022 20:51:52 +0000 (13:51 -0700)]
internal/types/errors: add package
The internal/types/errors package defines all error codes used by
the type checkers. This is the 1st step of several that factor out
the error codes from go/types and types2; the package is not yet
used.
- The file codes.go is a copy of go/types/errorcodes.go. The
only change is the updated package name (types -> errors).
- The file codes_test.go is a copy of go/types/errorcodes_test.go
with updated package name (types_test -> errors_test) and minor
changes to walkCodes so that it doesn't require the pkgFiles
helper function (the test only parses a single file).
Change-Id: Idb977b1220737b56b330de1d977f698f022daafc
Reviewed-on: https://go-review.googlesource.com/c/go/+/439560
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
Robert Griesemer [Thu, 6 Oct 2022 20:17:53 +0000 (13:17 -0700)]
go/types, types2: better error for generic type decl. with missing constraint
If a generic type declaration is missing a constraint, syntactically
it is an array type declaration with an undefined array length.
Mention the possibility of a missing constraint in the error message
for the undefined array length.
For #56064.
For #55961.
For #51145.
Change-Id: Ic161aeda9ea44faa8aa3bf3e9d62b3b13a95d4c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/439559
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Motiejus Jakštys [Sat, 8 Oct 2022 05:00:42 +0000 (05:00 +0000)]
cmd/cgo: use -O2 when testing compiler features
Add "-O2" to all compiler/linker tests. This makes compiler/linker
feature probing better resemble actual compiling later.
Why?
----
zig c++ is a clang front-end[1] that accepts, among other things, the
target over the command line. This command:
zig c++ -target x86_64-linux-gnu main.o -o main
Will:
1. Pre-compile libc++.a.
2. Link the program with libc++.a from (1).
Currently Go only is learning about one flag from the linker, that is,
"--no-gc-sections". The resulting command that tests for the flag
support is this:
c++ -Wl,--no-gc-sections -x c - -o
This causes Zig to pre-compile libc++.a in debug mode. Then the actual
compiler+linker command from CGo adds a few more flags, including "-O2":
c++ <...> -Wl,--no-gc-sections -O2 <...>
From Zig perspective, debug-mode libc++.a is different from the
optimized one; that causes Zig to compile a new libc++.a. Specifically,
Zig adds "-fno-omit-frame-pointer" for debug builds, and
"-fomit-frame-pointer" for optimized builds.
As a result, we have to two sets of libc++.a for every arch/os tuple.
That takes CPU time and a bit of disk space.
Zig performance impact
----------------------
First compilation of a simple CGo application is faster by ~2.5 seconds
or ~60%:
$ CC="zig c++ -target x86_64-linux-gnu.2.28" hyperfine \
--warmup 3 --runs 10 \
--prepare 'rm -fr ~/.cache/zig ~/.cache/go-build /tmp/go-*' \
--parameter-list go go1.19,go1.19-O2 \
'/code/go/bin/{go} build .'
Benchmark 1: /code/go/bin/go1.19 build .
Time (mean ± σ): 6.168 s ± 0.059 s [User: 7.465 s, System: 1.578 s]
Range (min … max): 6.111 s … 6.242 s 10 runs
Benchmark 2: /code/go/bin/go1.19-O2 build .
Time (mean ± σ): 3.816 s ± 0.080 s [User: 4.730 s, System: 1.130 s]
Range (min … max): 3.657 s … 3.958 s 10 runs
Summary
'/code/go/bin/go1.19-O2 build .' ran
1.62 ± 0.04 times faster than '/code/go/bin/go1.19 build .'
If we add C++ to the mix, the difference grows to almost ~23 seconds, or
almost 90%:
Benchmark 1: CC="zig c++ -target x86_64-linux-gnu.2.28" /code/go/bin/go1.19 build .
Time (mean ± σ): 51.137 s ± 0.183 s [User: 234.165 s, System: 15.005 s]
Range (min … max): 50.934 s … 51.288 s 3 runs
Benchmark 2: CC="zig c++ -target x86_64-linux-gnu.2.28" /code/go/bin/go1.19-O2 build .
Time (mean ± σ): 27.102 s ± 0.068 s [User: 119.816 s, System: 8.513 s]
Range (min … max): 27.038 s … 27.174 s 3 runs
Summary
'/code/go/bin/go1.19-O2 build .' ran
1.89 ± 0.01 times faster than '/code/go/bin/go1.19 build .'
The difference is just due to the fact that Zig will not be instructed
to compile libc++.a for debug builds; Go doesn't need that.
A.k.a. does "-O2" for this check worsen performance?
No statistically significant performance differences with both clang-15
and gcc-11. Also, it affects only the first compile of a CGo progam, as
the linker tests are cached across invocations. go1.19 binary is the
go1.19 tag; go1.19-O2 is go1.19 + this patch.
$ hyperfine --warmup 3 --runs 20 \
--prepare 'rm -fr ~/.cache/go-build/ /tmp/go-*' \
--parameter-list go go1.19,go1.19-O2 \
--parameter-list cc gcc-11,clang-15 \
'CC={cc} /code/go/bin/{go} build .'
Benchmark 1: CC=gcc-11 /code/go/bin/go1.19 build .
Time (mean ± σ): 681.1 ms ± 13.7 ms [User: 501.6 ms, System: 247.1 ms]
Range (min … max): 654.1 ms … 707.2 ms 20 runs
Benchmark 2: CC=gcc-11 /code/go/bin/go1.19-O2 build .
Time (mean ± σ): 676.8 ms ± 10.2 ms [User: 500.4 ms, System: 245.6 ms]
Range (min … max): 664.4 ms … 696.4 ms 20 runs
Benchmark 3: CC=clang-15 /code/go/bin/go1.19 build .
Time (mean ± σ): 860.1 ms ± 17.1 ms [User: 530.0 ms, System: 394.9 ms]
Range (min … max): 839.4 ms … 920.0 ms 20 runs
Benchmark 4: CC=clang-15 /code/go/bin/go1.19-O2 build .
Time (mean ± σ): 864.5 ms ± 26.6 ms [User: 537.8 ms, System: 390.1 ms]
Range (min … max): 841.9 ms … 955.5 ms 20 runs
Summary
'CC=gcc-11 /code/go/bin/go1.19-O2 build .' ran
1.01 ± 0.03 times faster than 'CC=gcc-11 /code/go/bin/go1.19 build .'
1.27 ± 0.03 times faster than 'CC=clang-15 /code/go/bin/go1.19 build .'
1.28 ± 0.04 times faster than 'CC=clang-15 /code/go/bin/go1.19-O2 build .'
1. Add "-O2" for linker-only tests. That looks like too much catering to
zig alone. If we can add it, then add for everything.
2. Add "-fomit-frame-pointer" instead of "-O2". This flag does not
universally imply debug mode, thus same argument applies as to (1).
3. Add "-O2" for this particular test (`--no-gc-sections`). This is
brittle and not future-proof: a future linker test may omit this
flag.
Hardware
--------
Tested on a 4-core (8 HT) Intel(R) Core(TM) i7-8665U CPU on Debian 11,
Linux 5.10.0-15-amd64.
Zeke Lu [Fri, 7 Oct 2022 00:49:15 +0000 (00:49 +0000)]
cmd/go/internal/modload: improve error message for failing to read module listed in go.work
Run "go build ./x" in this workspace:
-- go.work --
use ./y
-- x/go.mod --
module x
go 1.19
-- x/m.go --
package m
It fails with: "go: open /tmp/foo/y/go.mod: no such file or directory".
It's unclear where the name "y" comes from.
This change will emit error like: "go: cannot load module listed in
go.work file: open /tmp/foo/y/go.mod: no such file or directory"
Ian Lance Taylor [Tue, 4 Oct 2022 21:35:39 +0000 (14:35 -0700)]
os: use poll.fdMutex for Plan 9 files
This permits us to safely support concurrent access to files on Plan 9.
Concurrent access was already safe on other systems.
This does introduce a change: if one goroutine calls a blocking read
on a pipe, and another goroutine closes the pipe, then before this CL
the close would occur. Now the close will be delayed until the blocking
read completes.
Also add tests that concurrent I/O and Close on a pipe are OK.
For #50436
For #56043
Change-Id: I969c869ea3b8c5c2f2ef319e441a56a3c64e7bf5
Reviewed-on: https://go-review.googlesource.com/c/go/+/438347 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: David du Colombier <0intro@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
eric fang [Thu, 18 Aug 2022 09:59:35 +0000 (09:59 +0000)]
cmd/compile: enable carry chain scheduling for arm64
This is a follow up of CL 393656 on arm64.
This CL puts ScoreCarryChainTail before ScoreMemory and after
ScoreReadFlags, so that the scheduling of the carry chain will not
break the scheduling of ScoreVarDef.
Jeremy Brewer [Tue, 20 Sep 2022 00:09:56 +0000 (20:09 -0400)]
modload: provide a clearer error for standard library packages from newer releases
An older version of go compiling a main module that references a
standard library package from a newer release (e.g. net/netip added in
go 1.18) currently produces a confusing error message. This changes adds
a new error message including go version diagnostics.
Andrew Pogrebnoy [Fri, 7 Oct 2022 07:12:54 +0000 (10:12 +0300)]
runtime: move epoll syscalls to runtime/internal/syscall
This change moves Linux epoll's syscalls implementation to the
"runtime/internal/syscall" package. The intention in this CL was to
minimise behavioural changes but make the code more generalised. This
also will allow adding new syscalls (like epoll_pwait2) without the
need to implement assembly stubs for each arch.
It also drops epoll_create as not all architectures provide this call.
epoll_create1 was added to the kernel in version 2.6.27 and Go requires
Linux kernel version 2.6.32 or later since Go 1.18. So it is safe to
always use epoll_create1.
This is a resubmit as the previous CL 421994 was reverted due to test
failures after the merge with the master. The issue was fixed in
CL 438615
For #53824
For #51087
Change-Id: I1bd0f23a85b4f9b80178c5dd36fd3e95ff4f9648
Reviewed-on: https://go-review.googlesource.com/c/go/+/440115 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Wayne Zuo [Tue, 6 Sep 2022 14:12:16 +0000 (22:12 +0800)]
cmd/compile: intrinsify Add64 on loong64
This is a follow up of CL 420094 on loong64.
Reduce go toolchain size slightly on linux/loong64.
compilecmp HEAD~1 -> HEAD
HEAD~1 (8a32354219): internal/trace: use strings.Builder
HEAD (1767784ac3): cmd/compile: intrinsify Add64 on loong64
platform: linux/loong64
Eddie Lopez [Fri, 30 Sep 2022 20:51:33 +0000 (16:51 -0400)]
testing: add an example showcasing B.RunParallel with B.ReportMetric
This commit was dedicated to adding an example of using B.ReportMetrics
with B.RunParallel called ExampleB_ReportMetric_parallel. In this
example, the same algorithm for ExampleB_ReportMetric was used, instead
with a concurrent for loop using PB.Next instead of a standard one.
There is also notes noting when to use the B.ReportMetric methods when
running concurrent testing.
Damien Neil [Mon, 3 Oct 2022 23:07:48 +0000 (16:07 -0700)]
net/http: refactor tests to run most in HTTP/1 and HTTP/2 modes
Replace the ad-hoc approach to running tests in HTTP/1 and HTTP/2
modes with a 'run' function that executes a test in various modes.
By default, these modes are HTTP/1 and HTTP/2, but tests can
opt-in to HTTPS/1 as well.
The 'run' function also takes care of post-test cleanup (running the
afterTest function).
The 'run' function runs tests in parallel by default. Tests which
can't run in parallel (generally because they use global test hooks)
pass a testNotParallel option to disable parallelism.
Update clientServerTest to use t.Cleanup to clean up after itself,
rather than leaving this up to tests to handle.
Drop an unnecessary mutex in SetReadLoopBeforeNextReadHook.
Test hooks can't be set in parallel, and we want the race detector
to notify us if two simultaneous tests try to set a hook.
Fixes #56032
Change-Id: I16be64913c426fc93d84abc6ad85dbd3bc191224
Reviewed-on: https://go-review.googlesource.com/c/go/+/438137
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: David Chase <drchase@google.com>
Tobias Klauser [Tue, 4 Oct 2022 11:53:46 +0000 (13:53 +0200)]
internal/poll, internal/syscall/unix, syscall: move writev definition for solaris
Move the writev definition for solaris from package
internal/syscall/unix to package syscall. This corresponds to where
writev is defined on aix, darwin and openbsd as well and is
go:linkname'ed from internal/poll. This also allows updating the
generated wrappers more easily if needed.
Change-Id: I671ed8232d25319f8e63f549f786d77a17602148
Reviewed-on: https://go-review.googlesource.com/c/go/+/436597
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Lynn Boger [Thu, 18 Aug 2022 20:35:38 +0000 (15:35 -0500)]
cmd/compile: leverage cc ops in more cases on ppc64x
This updates some rules to use ops with CC variations to
set the condition code when the result of the operation is
zero. This allows the following compare with zero to be
removed since the equivalent condition code has already been
set.
In addition, a previous rule change to use ANDCCconst was modified
to allow any constant value, not just 1 in some cases.
kxxt [Sat, 1 Oct 2022 00:19:12 +0000 (00:19 +0000)]
all: use grep -E/-F instead of fgrep/egrep
egrep and fgrep are obsolescent now.
This PR updates all egrep and fgrep commands to grep -E and grep -F.
Running egrep/fgrep command with grep v3.8 will output the following warning to stderr:
egrep: warning: egrep is obsolescent; using grep -E
see also:
https://www.phoronix.com/news/GNU-Grep-3.8-Stop-egrep-fgrep
https://lists.gnu.org/archive/html/info-gnu/2022-09/msg00001.html
Change-Id: Iea1ca9ae72264530c67727b5e27cf1b7a362dd97
GitHub-Last-Rev: 3584884bd48cca97271ab86010fce8e4e063c0e4
GitHub-Pull-Request: golang/go#55299
Reviewed-on: https://go-review.googlesource.com/c/go/+/432256
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Bryan C. Mills [Wed, 5 Oct 2022 19:21:06 +0000 (15:21 -0400)]
os/exec: parallelize more tests
This cuts the wall duration for 'go test os/exec' and
'go test -race os/exec' roughly in half on my machine,
which is an even more significant speedup with a high '-count'.
For better or for worse, it may also increase the repro rate
of #34988.
Tests that use Setenv or Chdir or check for FDs opened during the test
still cannot be parallelized, but they are only a few of those.
Bryan C. Mills [Wed, 5 Oct 2022 19:26:43 +0000 (15:26 -0400)]
os/exec: delete TestExtraFilesFDShuffle
This test has been disabled for over nine years (since CL 12869049).
Although it still compiles, it seems likely to have rotted since then,
and if it was going to detect a real bug it also seems like that bug
would have been encountered and reported by users since then (and
would presumably have its own regression tests).
To eliminate overhead from mainining it (or skipping over it while
maintaining other tests), let's just delete it.
Zeke Lu [Tue, 13 Sep 2022 18:05:53 +0000 (18:05 +0000)]
debug/buildinfo: correct a typo in calculating next align index
When it calculates the smallest n such that:
n >= i && n % buildInfoAlign == 0
the expression should be
(i+buildInfoAlign-1)&^(buildInfoAlign-1)
instead of
(i+buildInfoAlign-1)&^buildInfoAlign
Fixes #54968.
Change-Id: Ibb7bdf568a521545b2609acc85e2ab4e05da5dae
GitHub-Last-Rev: 479ebc140af9809f0bea039e643cb95b4f857614
GitHub-Pull-Request: golang/go#54971
Reviewed-on: https://go-review.googlesource.com/c/go/+/429815 Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Wayne Zuo [Tue, 6 Sep 2022 03:43:28 +0000 (11:43 +0800)]
cmd/compile: fold constant shift with extension on riscv64
For example:
movb a0, a0
srai $1, a0, a0
the assembler will expand to:
slli $56, a0, a0
srai $56, a0, a0
srai $1, a0, a0
this CL optimize to:
slli $56, a0, a0
srai $57, a0, a0
Remove 270+ instructions from Go binary on linux/riscv64.
Change-Id: I375e19f9d3bd54f2781791d8cbe5970191297dc8
Reviewed-on: https://go-review.googlesource.com/c/go/+/428496 Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org> Reviewed-by: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
cui fliter [Wed, 5 Oct 2022 03:15:24 +0000 (03:15 +0000)]
all: remove redundant type conversion
Change-Id: I375233dc700adbc58a6d4af995d07b352bf85b11
GitHub-Last-Rev: ef129205231b892f61b0135c87bb791a5e1a126c
GitHub-Pull-Request: golang/go#55994
Reviewed-on: https://go-review.googlesource.com/c/go/+/437715
Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@google.com>
Ian Lance Taylor [Tue, 4 Oct 2022 03:08:11 +0000 (20:08 -0700)]
os: if dirFS.Open fails, undo use of backslashes in error message
This fixes a bug introduced by CL 426094 that caused the
golang.org/x/website/internal/web tests to fail.
Fixes #56034
Change-Id: Ic64967c6d440ad260b7283a18972b20023320ab6
Reviewed-on: https://go-review.googlesource.com/c/go/+/437976
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
kijimaD [Wed, 5 Oct 2022 12:56:46 +0000 (12:56 +0000)]
all: fix some typos
Change-Id: I6be77e7b7c919f26bed7b6690cce6741888ba78a
GitHub-Last-Rev: 4ef4a7b425d0b89adf398a1bee04e9f7495813bc
GitHub-Pull-Request: golang/go#56051
Reviewed-on: https://go-review.googlesource.com/c/go/+/438991 Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
Michael Matloob [Wed, 5 Oct 2022 16:54:00 +0000 (12:54 -0400)]
cmd/go/internal/modindex: ignore non-source files for index
We were saving non-go file information in the module index files,
leading in an unnecessary increase in memory usage in modules
containing many non-go files. This was a bug because this information
is never used. Don't save that information.
For #54226
Change-Id: I0644064f83f96e3a9f43b7e66ca94d69d9603376
Reviewed-on: https://go-review.googlesource.com/c/go/+/439118
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
Than McIntosh [Tue, 4 Oct 2022 19:10:30 +0000 (15:10 -0400)]
runtime/coverage: revise/shorten function names
Use shorter more Go-like names for the new APIs being added in the
runtime/coverage package for writing coverage data under user control
from server programs. Old names were a bit too clunky/verbose.
Cherry Mui [Fri, 30 Sep 2022 23:01:09 +0000 (19:01 -0400)]
runtime: don't jump stack if at entry of systemstack
The traceback code has special "jump stack" logic, to trace back
stack switches through systemstack. If we're at the entry of
systemstack, the stack switch hasn't happened, so don't jump to
user stack.
The jump stack logic is only used if we're on the g0 stack. It can
happen that we're at the entry of a recursive systemstack call on
the g0 stack. In we jump stack here, there will be two problems:
1. There are frames between entering the g0 stack and this
recursive systemstack call. Those frames will be lost.
2. Worse, we switched frame.sp but frame.fp calculation will use
the entry SP delta (0), which will be wrong, which in turn
leads wrong frame.lr and things will go off.
For now, don't jump stack if we're at entry of systemstack (SP
delta is 0).
Using a per-PC SPWRITE marker may be a better fix. If we haven't
written the SP, we haven't switched the stack so we can just
unwind like a normal function.
May fix #55851.
Change-Id: I2b624c8c086b235b34d9c7d3cebd4a37264f00f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/437299
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Youlin Feng [Mon, 3 Oct 2022 13:15:12 +0000 (13:15 +0000)]
runtime/cgo: let darwin pthread stacksize follow rlimit
On Mac OS X, the default stack size for non-main threads created by cgo is
fixed at 512KB and cannot be altered by setrlimit. This stack size is too
small for some recursive scenarios. We can solve this problem by explicitly
copying the stack size of the main thread when creating a new thread.
Change-Id: I400d5b2e929a1ee261502914a991e208759f64a8
GitHub-Last-Rev: b29c74599ea1cce4683d25e1ac8a70ffd9b86381
GitHub-Pull-Request: golang/go#53667
Reviewed-on: https://go-review.googlesource.com/c/go/+/415915
Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: hopehook <hopehook@golangcn.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Paul E. Murphy [Tue, 14 Jun 2022 21:13:51 +0000 (16:13 -0500)]
cmd/link: optimize PPC64 inline plt sequences if local
Indirect branches are much more expensive than direct. If the call is
known to be local, we can replace most of the operations with a nop,
and call directly.
Updates #53345
Change-Id: Icfff9ec1f6c7f8e4181f0f28976033308d2f53eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/412715 Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
eric fang [Wed, 17 Aug 2022 10:01:17 +0000 (10:01 +0000)]
cmd/compile: add late lower pass for last rules to run
Usually optimization rules have corresponding priorities, some need to
be run first, some run next, and some run last, which produces the best
code. But currently our optimization rules have no priority, this CL
adds a late lower pass that runs those rules that need to be run at last,
such as split unreasonable constant folding. This pass can be seen as
the second round of the lower pass.
For example:
func foo(a, b uint64) uint64 {
d := a+0x1234568
d1 := b+0x1234568
return d&d1
}
The code generated by the master branch:
0x0004 00004 ADD $19088744, R0, R2 // movz+movk+add
0x0010 00016 ADD $19088744, R1, R1 // movz+movk+add
0x001c 00028 AND R1, R2, R0
This is because the current constant folding optimization rules do not
take into account the range of constants, causing the constant to be
loaded repeatedly. This CL splits these unreasonable constants folding
in the late lower pass. With this CL the generated code:
0x0004 00004 MOVD $19088744, R2 // movz+movk
0x000c 00012 ADD R0, R2, R3
0x0010 00016 ADD R1, R2, R1
0x0014 00020 AND R1, R3, R0
This CL also adds constant folding optimization for ADDS instruction.
In addition, in order not to introduce the codegen regression, an
optimization rule is added to change the addition of a negative number
into a subtraction of a positive number.
Change-Id: I7c3cc0f48631af73cf34ad3c731c380f46c72359
Reviewed-on: https://go-review.googlesource.com/c/go/+/435257
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: hopehook <hopehook@golangcn.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>