Roland Shoemaker [Fri, 15 Jan 2021 20:14:06 +0000 (12:14 -0800)]
[release-branch.go1.14-security] all: introduce and use internal/execabs
Introduces a wrapper around os/exec, internal/execabs, for use in
all commands. This wrapper prevents exec.LookPath and exec.Command from
running executables in the current directory.
All imports of os/exec in non-test files in cmd/ are replaced with
imports of internal/execabs.
Filippo Valsorda [Fri, 8 Jan 2021 02:56:58 +0000 (03:56 +0100)]
[release-branch.go1.14-security] crypto/elliptic: fix P-224 field reduction
This patch fixes two independent bugs in p224Contract, the function that
performs the final complete reduction in the P-224 field. Incorrect
outputs due to these bugs were observable from a high-level
P224().ScalarMult() call.
The first bug was in the calculation of out3GT. That mask was supposed
to be all ones if the third limb of the value is greater than the third
limb of P (out[3] > 0xffff000). Instead, it was also set if they are
equal. That meant that if the third limb was equal, the value was always
considered greater than or equal to P, even when the three bottom limbs
were all zero. There is exactly one affected value, P - 1, which would
trigger the subtraction by P even if it's lower than P already.
The second bug was more easily hit, and is the one that caused the known
high-level incorrect output: after the conditional subtraction by P, a
potential underflow of the lowest limb was not handled. Any values that
trigger the subtraction by P (values between P and 2^224-1, and P - 1
due to the bug above) but have a zero lowest limb would produce invalid
outputs. Those conditions apply to the intermediate representation
before the subtraction, so they are hard to trace to precise inputs.
This patch also adds a test suite for the P-224 field arithmetic,
including a custom fuzzer that automatically explores potential edge
cases by combining limb values that have various meanings in the code.
contractMatchesBigInt in TestP224Contract finds the second bug in less
than a second without being tailored to it, and could eventually find
the first one too by combining 0, (1 << 28) - 1, and the difference of
(1 << 28) and (1 << 12).
The incorrect P224().ScalarMult() output was found by the
elliptic-curve-differential-fuzzer project running on OSS-Fuzz and
reported by Philippe Antoine (Catena cyber).
Carlos Amedee [Thu, 3 Dec 2020 16:09:42 +0000 (11:09 -0500)]
[release-branch.go1.14] go1.14.13
Change-Id: Ifdfd9cd7edb8c3afd785cf75e818e3d301cd8dae
Reviewed-on: https://go-review.googlesource.com/c/go/+/275133
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Carlos Amedee <carlos@golang.org>
MOVLconst must have a properly sign-extended auxint constant.
The bit operations in these rules don't enforce that invariant.
Fixes #42755
Change-Id: I729afcad18752d9b7739e49709020e3be7b3653e
Reviewed-on: https://go-review.googlesource.com/c/go/+/272030
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Ian Lance Taylor [Tue, 27 Oct 2020 23:09:40 +0000 (16:09 -0700)]
[release-branch.go1.14] runtime: block signals in needm before allocating M
Otherwise, if a signal occurs just after we allocated the M,
we can deadlock if the signal handler needs to allocate an M
itself.
For #42207
Fixes #42635
Change-Id: I76f44547f419e8b1c14cbf49bf602c6e645d8c14
Reviewed-on: https://go-review.googlesource.com/c/go/+/265759
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit 368c40116434532dc0b53b72fa04788ca6742898)
Reviewed-on: https://go-review.googlesource.com/c/go/+/271848
Ian Lance Taylor [Fri, 13 Nov 2020 19:05:37 +0000 (11:05 -0800)]
[release-branch.go1.14] cmd/go: permit CGO_LDFLAGS to appear in //go:ldflag
For #42565
Fixes #42566
Change-Id: If7cf39905d124dbd54dfac6a53ee38270498efed
Reviewed-on: https://go-review.googlesource.com/c/go/+/269818
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from commit 782cf560db4c919790fdb476d1bbe18e5ddf5ffd)
Reviewed-on: https://go-review.googlesource.com/c/go/+/270080
Katie Hockman [Tue, 10 Nov 2020 20:54:12 +0000 (15:54 -0500)]
[release-branch.go1.14-security] math/big: fix shift for recursive division
The previous s value could cause a crash
for certain inputs.
Will check in tests and documentation improvements later.
Thanks to the Go Ethereum team and the OSS-Fuzz project for reporting this.
Thanks to Rémy Oudompheng and Robert Griesemer for their help
developing and validating the fix.
Fixes CVE-2020-28362
Change-Id: Ibbf455c4436bcdb07c84a34fa6551fb3422356d3
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/899974 Reviewed-by: Roland Shoemaker <bracewell@google.com> Reviewed-by: Filippo Valsorda <valsorda@google.com>
(cherry picked from commit 28015462c2a83239543dc2bef651e9a5f234b633)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/901064
Ian Lance Taylor [Tue, 3 Nov 2020 05:31:06 +0000 (21:31 -0800)]
[release-branch.go1.14-security] cmd/go: in cgoflags, permit -DX1, prohibit -Wp,-D,opt
Restrict -D and -U to ASCII C identifiers, but do permit trailing digits.
When using -Wp, prohibit commas in -D values.
Thanks to Imre Rad (https://www.linkedin.com/in/imre-rad-2358749b) for reporting this.
Fixes CVE-2020-28367
Change-Id: Ibfc4dfdd6e6c258e131448e7682610c44eee9492
Reviewed-on: https://go-review.googlesource.com/c/go/+/267277
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/899923 Reviewed-by: Filippo Valsorda <valsorda@google.com>
Ian Lance Taylor [Fri, 6 Nov 2020 17:38:38 +0000 (09:38 -0800)]
[release-branch.go1.14-security] cmd/go, cmd/cgo: don't let bogus symbol set cgo_ldflag
A hand-edited object file can have a symbol name that uses newline and
other normally invalid characters. The cgo tool will generate Go files
containing symbol names, unquoted. That can permit those symbol names
to inject Go code into a cgo-generated file. If that Go code uses the
//go:cgo_ldflag pragma, it can cause the C linker to run arbitrary
code when building a package. If you build an imported package we
permit arbitrary code at run time, but we don't want to permit it at
package build time. This CL prevents this in two ways.
In cgo, reject invalid symbols that contain non-printable or space
characters, or that contain anything that looks like a Go comment.
In the go tool, double check all //go:cgo_ldflag directives in
generated code, to make sure they follow the existing LDFLAG restrictions.
Thanks to Chris Brown and Tempus Ex for reporting this.
Fixes CVE-2020-28366
Change-Id: Ia1ad8f3791ea79612690fa7d26ac451d0f6df7c1
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/895832 Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
(cherry picked from commit 6bc814dd2bbfeaafa41d314dd4cc591b575dfbf6)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/901055 Reviewed-by: Filippo Valsorda <valsorda@google.com> Reviewed-by: Roland Shoemaker <bracewell@google.com>
Alexander Rakoczy [Thu, 5 Nov 2020 18:12:54 +0000 (13:12 -0500)]
[release-branch.go1.14] go1.14.11
Change-Id: I1b4231179d0825113f2cbb6e84e92b3453e2ee45
Reviewed-on: https://go-review.googlesource.com/c/go/+/267878
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Alexander Rakoczy <alex@golang.org>
Tobias Klauser [Wed, 28 Oct 2020 12:44:53 +0000 (13:44 +0100)]
[release-branch.go1.14] cmd/go/internal/modfetch: drop gopkg.in/russross/blackfriday.v2 from TestCodeRepoVersions
Follow-up for CL 265819.
Given the -pre tag added recently, a new stable version is likely
tagged soon. This would break TestCodeRepoVersions on the longtest
builders again. Since the other test cases in codeRepoVersionsTests
already provide enough coverage, drop gopkg.in/russross/blackfriday.v2
to avoid breaking TestCodeRepoVersions once the release happens.
Updates #28856
Change-Id: If86a637b5e47f59faf9048fc1cbbae6e8f1dcc53
Reviewed-on: https://go-review.googlesource.com/c/go/+/265917
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
(cherry picked from commit 421d4e72de802ed65cb38317660654771cfb13e9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/266178
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
(cherry picked from commit 8687f6d924ee3a311e08db855c6dc1024c1f9349)
Reviewed-on: https://go-review.googlesource.com/c/go/+/266302
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Christopher Hlubek [Mon, 26 Oct 2020 12:44:44 +0000 (13:44 +0100)]
[release-branch.go1.14] time: fix LoadLocationFromTZData with slim tzdata
The extend information of a time zone file with last transition < now
could result in a wrong cached zone because it used the zone of the
last transition.
This could lead to wrong zones in systems with slim zoneinfo.
Fixes #42155
Change-Id: I7c57c35b5cfa58482ac7925b5d86618c52f5444d
Reviewed-on: https://go-review.googlesource.com/c/go/+/264939
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit 70e022e4a83dc996ac4f108e811fbc399ad5565b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/266303
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Ian Lance Taylor [Thu, 22 Oct 2020 22:25:00 +0000 (15:25 -0700)]
[release-branch.go1.14] time: support slim tzdata format
Backport of part of https://golang.org/cl/261877 to support the slim
tzdata format. As of tzdata 2020b, the default is to use the slim format.
We need to support that format so that Go installations continue to
work when tzdata is updated.
Relevant part of the CL description:
The reason for the failed tests was that when caching location data, the
extended time format past the end of zone transitions was not
considered. The respective change was introduced in (*Location).lookup
by CL 215539.
For #42155
Change-Id: I37f52a0917b2c6e3957e6b4612c8ef104c736e65
Reviewed-on: https://go-review.googlesource.com/c/go/+/264301
Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
(cherry picked from commit 414668cfbc41fd8cadf74e981849d1e05cc23b2e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/266298
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Thu, 15 Oct 2020 21:39:12 +0000 (14:39 -0700)]
[release-branch.go1.14] runtime: wait for preemption signals before syscall.Exec
For #41702
For #41703
For #42023
Change-Id: If07f40b1d73b8f276ee28ffb8b7214175e56c24d
Reviewed-on: https://go-review.googlesource.com/c/go/+/262817
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
(cherry picked from commit 05739d6f17c57f09264272621b88725a463234d0)
Reviewed-on: https://go-review.googlesource.com/c/go/+/264023
Ian Lance Taylor [Thu, 15 Oct 2020 21:19:10 +0000 (14:19 -0700)]
[release-branch.go1.14] syscall: use MustHaveExec in TestExec
For #41702
For #41703
Change-Id: Ib2b15e52aa1fef2f5e644b316c726150252fa9f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/262738
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
(cherry picked from commit 11cfb48df192c14d185c1cfcaad1ba3e7b84c807)
Reviewed-on: https://go-review.googlesource.com/c/go/+/264021
Ian Lance Taylor [Wed, 14 Oct 2020 23:03:48 +0000 (16:03 -0700)]
[release-branch.go1.14] runtime: stop preemption during syscall.Exec on Darwin
On current macOS versions a program that receives a signal during an
execve can fail with a SIGILL signal. This appears to be a macOS
kernel bug. It has been reported to Apple.
This CL partially works around the problem by using execLock to not
send preemption signals during execve. Of course some other stray
signal could occur, but at least we can avoid exacerbating the problem.
We can't simply disable signals, as that would mean that the exec'ed
process would start with all signals blocked, which it likely does not
expect.
For #41702
Fixes #41703
Change-Id: I91b0add967b315671ddcf73269c4d30136e579b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/262438
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
(cherry picked from commit 64fb6ae95f1c322486cbfb758552bb8439a8e6e8)
Reviewed-on: https://go-review.googlesource.com/c/go/+/262737
Cherry Zhang [Mon, 15 Jun 2020 20:20:53 +0000 (16:20 -0400)]
[release-branch.go1.14] runtime: set g to gsignal before adjustSignalStack
When a signal is received, the runtime probes whether an
alternate signal stack is set, if so, adjust gsignal's stack to
point to the alternate signal stack. This is done in
adjustSignalStack, which calls sigaltstack "syscall", which is a
libc call on darwin through asmcgocall. asmcgocall decides
whether to do stack switch based on whether we're running on g0
stack, gsignal stack, or regular g stack. If g is not set to
gsignal, asmcgocall may make wrong decision. Set g first.
adjustSignalStack is recursively nosplit, so it is okay that
temporarily gsignal.stack doesn't match the stack we're running
on.
Updates #39079.
Fixes #41991.
Change-Id: I59b2c5dc08c3c951f1098fff038bf2e06d7ca055
Reviewed-on: https://go-review.googlesource.com/c/go/+/238020
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit d286e61b6787fe2b55bf0ec8a814962ebda8d202)
Reviewed-on: https://go-review.googlesource.com/c/go/+/262557
Trust: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Alexander Rakoczy [Wed, 14 Oct 2020 17:45:23 +0000 (13:45 -0400)]
[release-branch.go1.14] go1.14.10
Change-Id: Ia983336cdedc9fa835bfc792dd1e819eef31596f
Reviewed-on: https://go-review.googlesource.com/c/go/+/262338
Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Alexander Rakoczy <alex@golang.org>
Austin Clements [Sat, 3 Oct 2020 20:44:22 +0000 (16:44 -0400)]
[release-branch.go1.14] runtime: correct signature of call16
The signature of call16 is currently missing the "typ" parameter. This
CL fixes this. This wasn't caught by vet because call16 is defined by
macro expansion (see #17544), and we didn't notice the mismatch with
the other call* functions because call16 is defined only on 32-bit
architectures and lives alone in stubs32.go.
Unfortunately, this means its GC signature is also wrong: the "arg"
parameter is treated as a scalar rather than a pointer, so GC won't
trace it and stack copying won't adjust it. This turns out to matter
in exactly one case right now: on 32-bit architectures (which are the
only architectures where call16 is defined), a stack-allocated defer
of a function with a 16-byte or smaller argument frame including a
non-empty result area can corrupt memory if the deferred function
grows the stack and is invoked during a panic. Whew. All other current
uses of reflectcall pass a heap-allocated "arg" frame (which happens
to be reachable from other stack roots, so tracing isn't a problem).
Curiously, in 2016, the signatures of all call* functions were wrong
in exactly this way. CL 31654 fixed all of them in stubs.go, but
missed the one in stubs32.go.
Michael Anthony Knyszek [Mon, 10 Aug 2020 20:02:22 +0000 (20:02 +0000)]
[release-branch.go1.14] runtime: disable stack shrinking in activeStackChans race window
Currently activeStackChans is set before a goroutine blocks on a channel
operation in an unlockf passed to gopark. The trouble is that the
unlockf is called *after* the G's status is changed, and the G's status
is what is used by a concurrent mark worker (calling suspendG) to
determine that a G has successfully been suspended. In this window
between the status change and unlockf, the mark worker could try to
shrink the G's stack, and in particular observe that activeStackChans is
false. This observation will cause the mark worker to *not* synchronize
with concurrent channel operations when it should, and so updating
pointers in the sudog for the blocked goroutine (which may point to the
goroutine's stack) races with channel operations which may also
manipulate the pointer (read it, dereference it, update it, etc.).
Fix the problem by adding a new atomically-updated flag to the g struct
called parkingOnChan, which is non-zero in the race window above. Then,
in isShrinkStackSafe, check if parkingOnChan is zero. The race is
resolved like so:
* Blocking G sets parkingOnChan, then changes status in gopark.
* Mark worker successfully suspends blocking G.
* If the mark worker observes parkingOnChan is non-zero when checking
isShrinkStackSafe, then it's not safe to shrink (we're in the race
window).
* If the mark worker observes parkingOnChan as zero, then because
the mark worker observed the G status change, it can be sure that
gopark's unlockf completed, and gp.activeStackChans will be correct.
The risk of this change is low, since although it reduces the number of
places that stack shrinking is allowed, the window here is incredibly
small. Essentially, every place that it might crash now is replaced with
no shrink.
This change adds a test, but the race window is so small that it's hard
to trigger without a well-placed sleep in park_m. Also, this change
fixes stackGrowRecursive in proc_test.go to actually allocate a 128-byte
stack frame. It turns out the compiler was destructuring the "pad" field
and only allocating one uint64 on the stack.
For #40641.
Fixes #40642.
Change-Id: I7dfbe7d460f6972b8956116b137bc13bc24464e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/247050
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit eb3c6a93c3236bbde5dee6cc5bd4ca9f8ab1647a)
Reviewed-on: https://go-review.googlesource.com/c/go/+/256301 Reviewed-by: Austin Clements <austin@google.com>
Keith Randall [Sat, 8 Aug 2020 14:58:04 +0000 (07:58 -0700)]
[release-branch.go1.14] cmd/compile: fix live variable computation for deferreturn
Taking the live variable set from the last return point is problematic.
See #40629 for details, but there may not be a return point, or it may
be before the final defer.
Additionally, keeping track of the last call as a *Value doesn't quite
work. If it is dead-code eliminated, the storage for the Value is reused
for some other random instruction. Its live variable information,
if it is available at all, is wrong.
Instead, just mark all the open-defer argument slots as live
throughout the function. (They are already zero-initialized.)
Fixes #40647
Change-Id: Ie456c7db3082d0de57eaa5234a0f32525a1cce13
Reviewed-on: https://go-review.googlesource.com/c/go/+/247522
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dan Scales <danscales@google.com>
(cherry picked from commit 32a84c99e136ed5af0686dbedd31fd7dff40fb38)
Reviewed-on: https://go-review.googlesource.com/c/go/+/248622
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Bryan C. Mills [Wed, 19 Aug 2020 02:47:12 +0000 (22:47 -0400)]
[release-branch.go1.14] testing: flush test summaries to stdout atomically when streaming output
While debugging #40771, I realized that the chatty printer should only
ever print to a single io.Writer (normally os.Stdout). The other
Writer implementations in the chain write to local buffers, but if we
wrote a test's output to a local buffer, then we did *not* write it to
stdout and we should not store it as the most recently logged test.
Because the chatty printer should only ever print to one place, it
shouldn't receive an io.Writer as an argument — rather, it shouldn't
be used at all for destinations other than the main output stream.
On the other hand, when we flush the output buffer to stdout in the
top-level flushToParent call, it is important that we not allow some
other test's output to intrude between the test summary header and the
remainder of the test's output. cmd/test2json doesn't know how to
parse such an intrusion, and it's confusing to humans too.
No test because I couldn't reproduce the user-reported error without
modifying the testing package. (This behavior seems to be very
sensitive to output size and/or goroutine scheduling.)
Fixes #40880
Updates #40771
Updates #38458
Change-Id: Ic19bf1d535672b096ba1c8583a3b74aab6d6d766
Reviewed-on: https://go-review.googlesource.com/c/go/+/249026
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit 51c0bdc6d15dcd7f753c25896039ab41ac787ebb)
Reviewed-on: https://go-review.googlesource.com/c/go/+/252638
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
Keith Randall [Thu, 24 Sep 2020 21:25:21 +0000 (14:25 -0700)]
[release-branch.go1.14] cmd/compile: prevent 387+float32+pie from clobbering registers
The 387 port needs to load a floating-point control word from a
global location to implement float32 arithmetic.
When compiling with -pie, loading that control word clobbers an
integer register. If that register had something important in it, boom.
Fix by using LEAL to materialize the address of the global location
first. LEAL with -pie works because the destination register is
used as the scratch register.
387 support is about to go away (#40255), so this will need to be
backported to have any effect.
No test. I have one, but it requires building with -pie, which
requires cgo. Our testing infrastructure doesn't make that easy.
Not worth it for a port which is about to vanish.
Fixes #41619
Change-Id: I140f9fc8fdce4e74a52c2c046e2bd30ae476d295
Reviewed-on: https://go-review.googlesource.com/c/go/+/257277
Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Keith Randall <khr@golang.org>
(cherry picked from commit ea106cc07ac73110a8a25fcc5aef07b283159db0)
Reviewed-on: https://go-review.googlesource.com/c/go/+/257208
Michael Anthony Knyszek [Wed, 9 Sep 2020 16:52:18 +0000 (16:52 +0000)]
[release-branch.go1.14] runtime: fix ReadMemStatsSlow's and CheckScavengedBits' chunk iteration
Both ReadMemStatsSlow and CheckScavengedBits iterate over the page
allocator's chunks but don't actually check if they exist. During the
development process the chunks index became sparse, so now this was a
possibility. If the runtime tests' heap is sparse we might end up
segfaulting in either one of these functions, though this will generally
be very rare.
The pattern here to return nil for a nonexistent chunk is also useful
elsewhere, so this change introduces tryChunkOf which won't throw, but
might return nil. It also updates the documentation of chunkOf.
For #41296.
Fixes #41322.
Change-Id: Id5ae0ca3234480de1724fdf2e3677eeedcf76fa0
Reviewed-on: https://go-review.googlesource.com/c/go/+/253777
Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 34835df04891a1d54394888b763af88f9476101d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/253922
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Michael Munday [Mon, 10 Aug 2020 15:01:21 +0000 (08:01 -0700)]
[release-branch.go1.14] cmd/internal/obj: fix inline marker issue on s390x
The optimization that replaces inline markers with pre-existing
instructions assumes that 'Prog' values produced by the compiler are
still reachable after the assembler has run. This was not true on
s390x where the assembler was removing NOP instructions from the
linked list of 'Prog' values. This led to broken inlining data
which in turn caused an infinite loop in the runtime traceback code.
Fix this by stopping the s390x assembler backend removing NOP
values. It does not make any difference to the output of the
assembler because NOP instructions are 0 bytes long anyway.
Note: compiler check omitted from backport to reduce risk of change.
Fixes #40694.
Change-Id: I9f9bdbe895c3478549b5e7e623f9521f841e926a
Reviewed-on: https://go-review.googlesource.com/c/go/+/248477
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
It clobbers R12 (and CTR, which is never live across a call).
As at compile time we don't know whether the binary is big and
what link mode will be used, I think we need to mark R12 as
clobbered for write barrier call. For extra safety (future-proof)
we mark caller-saved register that cannot be used for function
arguments, which includes R11, as potentially clobbered as well.
Lynn Boger [Tue, 11 Aug 2020 16:04:25 +0000 (12:04 -0400)]
[release-branch.go1.14] cmd/internal/obj/ppc64: don't remove NOP in assembler
Previously, the assembler removed NOPs from the Prog list in
obj9.go. NOPs shouldn't be removed if they were added as
an inline mark, as described in the issue below.
Fixes #40766
Once the NOPs were left in the Prog list, some instructions
were flagged as invalid because they had an operand which was
not represented in optab. In order to preserve the previous
assembler behavior, entries were added to optab for those
operand cases. They were not flagged as errors before because
the NOP instructions were removed before the code to check the
valid opcode/operand combinations.
Change-Id: Iae5145f94459027cf458e914d7c5d6089807ccf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/247842
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Paul Murphy <murp@ibm.com> Reviewed-by: Michael Munday <mike.munday@ibm.com> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit 7d7bd5abc7f7ac901830b79496f63ce86895e262)
Reviewed-on: https://go-review.googlesource.com/c/go/+/248382
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Bryan C. Mills [Mon, 17 Aug 2020 21:31:21 +0000 (17:31 -0400)]
[release-branch.go1.14] testing: treat PAUSE lines as changing the active test name
We could instead fix cmd/test2json to treat PAUSE lines as *not*
changing the active test name, but that seems like it would be more
confusing to humans, and also wouldn't fix tools that parse output
using existing builds of cmd/test2json.
Fixes #40848
Updates #40657
Change-Id: I937611778f5b1e7dd1d6e9f44424d7e725a589ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/248727
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jean de Klerk <deklerk@google.com>
(cherry picked from commit cdc77d34d7770ed02d84b9193380f9646017dce6)
Reviewed-on: https://go-review.googlesource.com/c/go/+/249098
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL ensures that responses served via CGI and FastCGI
have a Content-Type header based on the content of the
response if not explicitly set by handlers.
If the implementers of the handler did not explicitly
specify a Content-Type both CGI implementations would default
to "text/html", potentially causing cross-site scripting.
Thanks to RedTeam Pentesting GmbH for reporting this.
Fixes CVE-2020-24553
Change-Id: I82cfc396309b5ab2e8d6e9a87eda8ea7e3799473
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/823217 Reviewed-by: Russ Cox <rsc@google.com>
(cherry picked from commit 23d675d07fdc56aafd67c0a0b63d5b7e14708ff0)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/835312 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Damien Neil [Thu, 11 Jun 2020 20:30:23 +0000 (13:30 -0700)]
[release-branch.go1.14] net/http: make Transport.RoundTrip preserve Requests
Ensure that the exact Request passed to Transport.RoundTrip
is returned in the Response. Do not replace the Request with
a copy when resetting the request body.
Updates #39533.
Fixes #40973.
Change-Id: Ie6fb080c24b0f6625b0761b7aa542af3d2411817
Reviewed-on: https://go-review.googlesource.com/c/go/+/237560
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/249880
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
Russ Cox [Fri, 22 May 2020 00:46:05 +0000 (20:46 -0400)]
[release-branch.go1.14] net/http: handle body rewind in HTTP/2 connection loss better
In certain cases the HTTP/2 stack needs to resend a request.
It obtains a fresh body to send by calling req.GetBody.
This call was missing from the path where the HTTP/2
round tripper returns ErrSkipAltProtocol, meaning fall back
to HTTP/1.1. The result was that the HTTP/1.1 fallback
request was sent with no body at all.
This CL changes that code path to rewind the body before
falling back to HTTP/1.1. But rewinding the body is easier
said than done. Some requests have no GetBody function,
meaning the body can't be rewound. If we need to rewind and
can't, that's an error. But if we didn't read anything, we don't
need to rewind. So we have to track whether we read anything,
with a new ReadCloser wrapper. That in turn requires adding
to the couple places that unwrap Body values to look at the
underlying implementation.
This CL adds the new rewinding code in the main retry loop
as well.
The new rewindBody function also takes care of closing the
old body before abandoning it. That was missing in the old
rewind code.
Thanks to Aleksandr Razumov for CL 210123
and to Jun Chen for CL 234358, both of which informed
this CL.
Updates #32441.
Fixes #39279.
Change-Id: Id183758526c087c6b179ab73cf3b61ed23a2a46a
Reviewed-on: https://go-review.googlesource.com/c/go/+/234894
Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit e3491c46034cecbaf0f33928b09e1e3c0c6a0d20)
Reviewed-on: https://go-review.googlesource.com/c/go/+/242117
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Jason A. Donenfeld [Tue, 14 Jul 2020 07:41:03 +0000 (01:41 -0600)]
[release-branch.go1.14] runtime: detect services in signal handler
The service handler needs to handle CTRL+C-like events -- including
those sent by the service manager itself -- using the default Windows
implementation if no signal handler from Go is already listening to
those events. Ordinarily, the signal handler would call exit(2), but we
actually need to allow this to be passed onward to the service handler.
So, we detect if we're in a service and skip calling exit(2) in that
case, just like we do for shared libraries.
Updates #40167.
Updates #40074.
Fixes #40411.
Change-Id: Ia77871737a80e1e94f85b02d26af1fd2f646af96
Reviewed-on: https://go-review.googlesource.com/c/go/+/244958
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Michael Pratt [Mon, 27 Jul 2020 19:04:17 +0000 (15:04 -0400)]
[release-branch.go1.14] runtime: ensure startm new M is consistently visible to checkdead
If no M is available, startm first grabs an idle P, then drops
sched.lock and calls newm to start a new M to run than P.
Unfortunately, that leaves a window in which a G (e.g., returning from a
syscall) may find no idle P, add to the global runq, and then in stopm
discover that there are no running M's, a condition that should be
impossible with runnable G's.
To avoid this condition, we pre-allocate the new M ID in startm before
dropping sched.lock. This ensures that checkdead will see the M as
running, and since that new M must eventually run the scheduler, it will
handle any pending work as necessary.
Outside of startm, most other calls to newm/allocm don't have a P at
all. The only exception is startTheWorldWithSema, which always has an M
if there is 1 P (i.e., the currently running M), and if there is >1 P
the findrunnable spinning dance ensures the problem never occurs.
This has been tested with strategically placed sleeps in the runtime to
help induce the correct race ordering, but the timing on this is too
narrow for a test that can be checked in.
Michael Anthony Knyszek [Mon, 13 Jul 2020 19:51:50 +0000 (19:51 +0000)]
[release-branch.go.1.14] runtime: validate candidate searchAddr in pageAlloc.find
Currently pageAlloc.find attempts to find a better estimate for the
first free page in the heap, even if the space its looking for isn't
necessarily going to be the first free page in the heap (e.g. if npages
>= 2). However, in doing so it has the potential to return a searchAddr
candidate that doesn't actually correspond to mapped memory, but this
candidate might still be adopted. As a result, pageAlloc.alloc's fast
path may look at unmapped summary memory and segfault. This case is rare
on most operating systems since the heap is kept fairly contiguous, so
the chance that the candidate searchAddr discovered is unmapped is
fairly low. Even so, this is totally possible and outside the user's
control when it happens (in fact, it's likely to happen consistently for
a given user on a given system).
Fix this problem by ensuring that our candidate always points to mapped
memory. We do this by looking at mheap's arenas structure first. If it
turns out our candidate doesn't correspond to mapped memory, then we
look at inUse to round up the searchAddr to the next mapped address.
While we're here, clean up some documentation related to searchAddr.
For #40191.
Fixes #40192.
Change-Id: I759efec78987e4a8fde466ae45aabbaa3d9d4214
Reviewed-on: https://go-review.googlesource.com/c/go/+/242680
Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit b56791cdea5caa87ffcd585d29c294bd3d08a06a)
Reviewed-on: https://go-review.googlesource.com/c/go/+/246197
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Cherry Zhang [Mon, 29 Jun 2020 21:07:17 +0000 (17:07 -0400)]
[release-branch.go1.14] cmd/link: fix GC data reading from shared library (attempt 2)
This is a backport of CL 240621. This is not a clean cherry-pick,
as Go 1.15 switches to the new linker while it is still the old
linker here. Backporting is straightforward, though.
When linking against a Go shared library, when a global variable
in the main module has a type defined in the shared library, the
linker needs to pull the GC data from the shared library to build
the GC program for the global variable. Currently, this fails
silently, as the shared library file is closed too early and the
read failed (with no error check), causing a zero GC map emitted
for the variable, which in turn causes the runtime to treat the
variable as pointerless.
For now, fix this by keeping the file open. In the future we may
want to use mmap to read from the shared library instead.
Also add error checking. And fix a (mostly harmless) mistake in
size caluculation.
Also remove an erroneous condition for ARM64. ARM64 has a special
case to get the addend from the relocation on the gcdata field.
But that doesn't actually work. And it's no longer necessary to
have any special case, since the addend is now applied directly
to the gcdata field on ARM64, like on all the other platforms.
Matthew Dempsky [Thu, 20 Aug 2020 04:39:12 +0000 (21:39 -0700)]
[release-branch.go1.14] cmd/compile: fix checkptr handling of &^
checkptr has code to recognize &^ expressions, but it didn't take into
account that "p &^ x" gets rewritten to "p & ^x" during walk, which
resulted in false positive diagnostics.
This CL changes walkexpr to mark OANDNOT expressions with Implicit
when they're rewritten to OAND, so that walkCheckPtrArithmetic can
still recognize them later.
It would be slightly more idiomatic to instead mark the OBITNOT
expression as Implicit (as it's a compiler-generated Node), but the
OBITNOT expression might get constant folded. It's not worth the extra
complexity/subtlety of relying on n.Right.Orig, so we set Implicit on
the OAND node instead.
To atone for this transgression, I add documentation for nodeImplicit.
Updates #40917.
Fixes #40968.
Change-Id: I386304171ad299c530e151e5924f179e9a5fd5b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/249477
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/249838 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Katie Hockman [Tue, 4 Aug 2020 15:45:32 +0000 (11:45 -0400)]
[release-branch.go1.14-security] encoding/binary: read at most MaxVarintLen64 bytes in ReadUvarint
This CL ensures that ReadUvarint consumes only a limited
amount of input (instead of an unbounded amount).
On some inputs, ReadUvarint could read an arbitrary number
of bytes before deciding to return an overflow error.
After this CL, ReadUvarint returns that same overflow
error sooner, after reading at most MaxVarintLen64 bytes.
Fix authored by Robert Griesemer and Filippo Valsorda.
Thanks to Diederik Loerakker, Jonny Rhea, Raúl Kripalani,
and Preston Van Loon for reporting this.
Keith Randall [Thu, 30 Jul 2020 17:37:36 +0000 (10:37 -0700)]
[release-branch.go1.14] cmd/compile: don't addLocalInductiveFacts if there is no direct edge from if block to phi block
Currently in addLocalInductiveFacts, we only check whether
direct edge from if block to phi block exists. If not, the
following logic will treat the phi block as the first successor,
which is wrong.
This patch makes prove pass more conservative, so we disable
some cases in test/prove.go. We will do some optimization in
the following CL and enable these cases then.
Fixes #40501.
Change-Id: I27cf0248f3a82312a6f7dabe11c79a1a34cf5412
Reviewed-on: https://go-review.googlesource.com/c/go/+/244579 Reviewed-by: Zach Jones <zachj1@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/245958 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
Jean de Klerk [Mon, 1 Jun 2020 18:19:05 +0000 (12:19 -0600)]
[release-branch.go1.14] testing: capture testname on --- PASS and --- FAIL lines
This fixes an issue raised at https://github.com/golang/go/issues/38458#issuecomment-635617670
in which --- PASS and --- FAIL lines would not trigger --- CONT lines
of other tests.
Updates #38458.
For #39308.
Change-Id: I0d8cc54d682a370d0a6ea6816a11b2e462a92efe
Reviewed-on: https://go-review.googlesource.com/c/go/+/235997
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/242058 Reviewed-by: Jean de Klerk <deklerk@google.com> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Tobias Klauser [Sat, 23 May 2020 22:09:20 +0000 (00:09 +0200)]
[release-branch.go1.14] cmd/go: fix parallel chatty tests on solaris-amd64 builder
The parallel chatty tests added in CL 229085 fail on the
solaris-amd64-oraclerel builder, because a +NN:NN offset time zone is
used. Allow for the `+` character in the corresponding regex to fix
these tests. Also move the '-' to the end of the character class, so it
is not interpreted as the range 9-T.
Updates #38458.
For #39308.
Change-Id: Iec9ae82ba45d2490176f274f0dc6812666eae718
Reviewed-on: https://go-review.googlesource.com/c/go/+/234978
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/242059
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Jean de Klerk [Mon, 4 May 2020 20:06:34 +0000 (14:06 -0600)]
[release-branch.go1.14] testing: reformat test chatty output
In #24929, we decided to stream chatty test output. It looks like,
foo_test.go:138: TestFoo/sub-1: hello from subtest 1
foo_test.go:138: TestFoo/sub-2: hello from subtest 2
In this CL, we refactor the output to be grouped by === CONT lines, preserving
the old test-file-before-log-line behavior:
=== CONT TestFoo/sub-1
foo_test.go:138 hello from subtest 1
=== CONT TestFoo/sub-2
foo_test.go:138 hello from subtest 2
This should remove a layer of verbosity from tests, and make it easier to group
together related lines. It also returns to a more familiar format (the
pre-streaming format), whilst still preserving the streaming feature.
Updates #38458.
Fixes #39308.
Change-Id: Iaef94c580d69cdd541b2ef055aa004f50d72d078
Reviewed-on: https://go-review.googlesource.com/c/go/+/229085
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/242057 Reviewed-by: Jean de Klerk <deklerk@google.com> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Jean de Klerk <deklerk@google.com>
Emmanuel T Odeke [Fri, 24 Jan 2020 02:18:39 +0000 (18:18 -0800)]
[release-branch.go1.14] database/sql: backport 5 Tx rollback related CLs
Manually backported the subject CLs, because of lack of
Gerrit "forge-author" permissions, but also because the prior
cherry picks didn't apply cleanly, due to a tight relation chain.
Note:
Due to the restrictions that we cannot retroactively
introduce API changes to Go1.14.6 that weren't in Go1.14, the Conn.Validator
interface (from CL 174122, CL 223963) isn't exposed, and drivers will just be
inspected, for if they have an IsValid() bool method implemented.
For a description of the content of each CL:
* CL 174122:
database/sql: process all Session Resets synchronously
Adds a new interface, driver.ConnectionValidator, to allow
drivers to signal they should not be used again,
separatly from the session resetter interface.
This is done now that the session reset is done
after the connection is put into the connection pool.
Previous behavior attempted to run Session Resets
in a background worker. This implementation had two
problems: untested performance gains for additional
complexity, and failures when the pool size
exceeded the connection reset channel buffer size.
* CL 216197:
database/sql: check conn expiry when returning to pool, not when handing it out
With the original connection reuse strategy, it was possible that
when a new connection was requested, the pool would wait for an
an existing connection to return for re-use in a full connection
pool, and then it would check if the returned connection was expired.
If the returned connection expired while awaiting re-use, it would
return an error to the location requestiong the new connection.
The existing call sites requesting a new connection was often the last
attempt at returning a connection for a query. This would then
result in a failed query.
This change ensures that we perform the expiry check right
before a connection is inserted back in to the connection pool
for while requesting a new connection. If requesting a new connection
it will no longer fail due to the connection expiring.
* CL 216240:
database/sql: prevent Tx statement from committing after rollback
It was possible for a Tx that was aborted for rollback
asynchronously to execute a query after the rollback had completed
on the database, which often would auto commit the query outside
of the transaction.
By W-locking the tx.closemu prior to issuing the rollback
connection it ensures any Tx query either fails or finishes
on the Tx, and never after the Tx has rolled back.
* CL 216241:
database/sql: on Tx rollback, retain connection if driver can reset session
Previously the Tx would drop the connection after rolling back from
a context cancel. Now if the driver can reset the session,
keep the connection.
* CL 223963
database/sql: add test for Conn.Validator interface
This addresses comments made by Russ after
https://golang.org/cl/174122 was merged. It addes a test
for the connection validator and renames the interface to just
"Validator".
Filippo Valsorda [Fri, 19 Jun 2020 02:45:52 +0000 (22:45 -0400)]
[release-branch.go1.14-security] crypto/x509: respect VerifyOptions.KeyUsages on Windows
When using the platform verifier on Windows (because Roots is nil) we
were always enforcing server auth EKUs if DNSName was set, and none
otherwise. If an application was setting KeyUsages, they were not being
respected.
Started correctly surfacing IncompatibleUsage errors from the system
verifier, as those are the ones applications will see if they are
affected by this change.
Also refactored verify_test.go to make it easier to add tests for this,
and replaced the EKULeaf chain with a new one that doesn't have a SHA-1
signature.
[release-branch.go1.14-security] net/http: synchronize "100 Continue" write and Handler writes
The expectContinueReader writes to the connection on the first
Request.Body read. Since a Handler might be doing a read in parallel or
before a write, expectContinueReader needs to synchronize with the
ResponseWriter, and abort if a response already went out.
Keith Randall [Thu, 18 Jun 2020 19:51:35 +0000 (12:51 -0700)]
[release-branch.go1.14] reflect: zero stack slots before writing to them with write barriers
reflect.assignTo writes to the target using write barriers. Make sure
that the memory it is writing to is zeroed, so the write barrier does
not read pointers from uninitialized memory.
Fixes #39698
Change-Id: Ia64b2cacc193bffd0c1396bbce1dfb8182d4905b
Reviewed-on: https://go-review.googlesource.com/c/go/+/238760
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit 3dec253783e1211989102ac6abd34cddbf8ba0e6)
Reviewed-on: https://go-review.googlesource.com/c/go/+/238861
Keith Randall [Thu, 25 Jun 2020 03:59:18 +0000 (20:59 -0700)]
[release-branch.go1.14] cmd/compile: remove check that Zero's arg has the correct base type
It doesn't have to. The type in the aux field is authoritative.
There are cases involving casting from interface{} where pointers
have a placeholder pointer type (because the type is not known when
the IData op is generated).
The check was introduced in CL 13447.
Fixes #39849
Change-Id: Id77a57577806a271aeebd20bea5d92d08ee7aa6b
Reviewed-on: https://go-review.googlesource.com/c/go/+/239817
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
(cherry picked from commit 3b2f67a59702e4881625cb967f853ef56b0c4828)
Reviewed-on: https://go-review.googlesource.com/c/go/+/239997
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
[release-branch.go1.14] cmd/link: detect trampoline of deferreturn call
This is a backport of CL 234105. This is not a clean cherry-pick,
as CL 234105 is for the new linker, whereas we still use the old
linker here. This CL backports the logic.
The runtime needs to find the PC of the deferreturn call in a few
places. So for functions that have defer, we record the PC of
deferreturn call in its funcdata.
For very large binaries, the deferreturn call could be made
through a trampoline. The current code of finding deferreturn PC
fails in this case. This CL handles the trampoline as well.
Fixes #39991.
Updates #39049.
Change-Id: I929be54d6ae436f5294013793217dc2a35f080d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/234105
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jeremy Faller <jeremy@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/240917
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Joel Sing <joel@sing.id.au>
Daniel Martí [Sun, 14 Jun 2020 21:09:18 +0000 (22:09 +0100)]
[release-branch.go1.14] encoding/json: revert "avoid work when unquoting strings, take 2"
This reverts golang.org/cl/190659 and golang.org/cl/226218, minus the
regression tests in the latter.
The original work happened in golang.org/cl/151157, which was reverted
in golang.org/cl/190909 due to a crash found by fuzzing.
We tried a second time in golang.org/cl/190659, which shipped with Go
1.14. A bug was found, where strings would be mangled in certain edge
cases. The fix for that was golang.org/cl/226218, which was backported
into Go 1.14.4.
Unfortunately, a second regression was just reported in #39555, which is
a similar case of strings getting mangled when decoding under certain
conditions. It would be possible to come up with another small patch to
fix that edge case, but instead, let's just revert the entire
optimization, as it has proved to do more harm than good. Moreover, it's
hard to argue or prove that there will be no more such regressions.
However, all the work wasn't for nothing. First, we learned that the way
the decoder unquotes tokenized strings isn't simple; initially, we had
wrongly assumed that each string was unquoted exactly once and in order.
Second, we have gained a number of regression tests which will be useful
to prevent the same mistakes in the future, including the test cases we
add in this CL.
For #39555.
Fixes #39585.
Change-Id: I66a6919c2dd6d9789232482ba6cf3814eaa70f61
Reviewed-on: https://go-review.googlesource.com/c/go/+/237838
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
(cherry picked from commit 11389baf2ea0b5e920959b0aa8d406d8090a0a93)
Reviewed-on: https://go-review.googlesource.com/c/go/+/241081 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
This teaches vet to recognize %O in a fmt.Printf format string.
O has been supported since the 1.13 release, but vet would warn about it.
Fixes #39288.
For #29986.
Change-Id: Ia7817ee60ae6beac32cc402c0c68afa917e4ef0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/236199
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Dmitri Shuralyov [Fri, 12 Jun 2020 02:02:27 +0000 (22:02 -0400)]
[release-branch.go1.14] cmd/compile/internal/ssa: skip TestNexting with old Delve on linux/386
Support for linux/386 was added to Delve in version 1.4.1, but the
version of Delve currently installed on the linux-386-longtest
builder is 1.2.0. That isn't new enough, which causes the test
to fail. Skip it on that builder until it can be made to work.
The only reason it used to pass on the linux-386-longtest builder
before is because that builder was misconfigured to run tests for
linux/amd64. This was resolved in CL 234520.
Also improve internal documentation and the text of skip reasons.
For #39309.
Fixes #39562.
Change-Id: I395cb1f076e59dd3a3feb53e1dcdce5101e9a0f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/237619 Reviewed-by: David Chase <drchase@google.com>
Ivan Trubach [Thu, 12 Dec 2019 13:33:42 +0000 (13:33 +0000)]
[release-branch.go1.14] cmd/doc: fix merging comments in -src mode
These changes fix go doc -src mode that vomits comments from random files if
filesystem does not sort files by name. The issue was with parse.ParseDir
using the Readdir order of files, which varies between platforms and filesystem
implementations. Another option is to merge comments using token.FileSet.Iterate
order in cmd/doc, but since ParseDir is mostly used in go doc, I’ve opted for
smaller change because it’s unlikely to break other uses or cause any perfomance
issues.
Example (macOS APFS): `go doc -src net.ListenPacket`
Fixes #38993
Change-Id: I7f9f368c7d9ccd9a2cbc48665f2cb9798c7b3a3f
GitHub-Last-Rev: 654fb450421266a0bb64518016944db22bd681e3
GitHub-Pull-Request: golang/go#36104
Reviewed-on: https://go-review.googlesource.com/c/go/+/210999
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
(cherry picked from commit 585e31df63f6879c03b285711de6f9dcba1f2cb0)
Reviewed-on: https://go-review.googlesource.com/c/go/+/235579
Run-TryBot: Andrew Bonventre <andybons@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
Daniel Martí [Fri, 27 Mar 2020 23:56:09 +0000 (23:56 +0000)]
[release-branch.go1.14] encoding/json: don't mangle strings in an edge case when decoding
The added comment contains some context. The original optimization
assumed that each call to unquoteBytes (or unquote) followed its
corresponding call to rescanLiteral. Otherwise, unquoting a literal
might use d.safeUnquote from another re-scanned literal.
Unfortunately, this assumption is wrong. When decoding {"foo": "bar"}
into a map[T]string where T implements TextUnmarshaler, the sequence of
calls would be as follows:
Note that the call to UnmarshalText happens in literalStore, which
repeats the work to unquote the input string literal. But, since that
happens after we've re-scanned "bar", we're using the wrong safeUnquote
field value.
In the added test case, the second string had a non-zero number of safe
bytes, and the first string had none since it was all non-ASCII. Thus,
"safely" unquoting a number of the first string's bytes could cut a rune
in half, and thus mangle the runes.
A rather simple fix, without a full revert, is to only allow one use of
safeUnquote per call to unquoteBytes. Each call to rescanLiteral when
we have a string is soon followed by a call to unquoteBytes, so it's no
longer possible for us to use the wrong index.
Also add a test case from #38126, which is the same underlying bug, but
affecting the ",string" option.
Before the fix, the test would fail, just like in the original two issues:
--- FAIL: TestUnmarshalRescanLiteralMangledUnquote (0.00s)
decode_test.go:2443: Key "开源" does not exist in map: map[开���:12345开源]
decode_test.go:2458: Unmarshal unexpected error: json: invalid use of ,string struct tag, trying to unmarshal "\"aaa\tbbb\"" into string
Fixes #38106.
For #38105.
For #38126.
Change-Id: I761e54924e9a971a4f9eaa70bbf72014bb1476e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/226218
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
(cherry picked from commit 55361a26177b3faf151a1d35467db5d403b51f22)
Reviewed-on: https://go-review.googlesource.com/c/go/+/233057
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Daniel Martí [Tue, 31 Mar 2020 11:20:15 +0000 (12:20 +0100)]
[release-branch.go1.14] encoding/json: properly encode strings with ",string" again
golang.org/cl/193604 fixed one bug when one encodes a string with the
",string" option: if SetEscapeHTML(false) is used, we should not be
using HTML escaping for the inner string encoding. The CL correctly
fixed that.
The CL also tried to speed up this edge case. By avoiding an entire new
call to Marshal, the new Issue34127 benchmark reduced its time/op by
45%, and lowered the allocs/op from 3 to 2.
However, that last optimization wasn't correct:
Since Go 1.2 every string can be marshaled to JSON without error
even if it contains invalid UTF-8 byte sequences. Therefore
there is no need to use Marshal again for the only reason of
enclosing the string in double quotes.
JSON string encoding isn't just about adding quotes and taking care of
invalid UTF-8. We also need to escape some characters, like tabs and
newlines.
The new code failed to do that. The bug resulted in the added test case
failing to roundtrip properly; before our fix here, we'd see an error:
invalid use of ,string struct tag, trying to unmarshal "\"\b\f\n\r\t\"\\\"" into string
If you pay close attention, you'll notice that the special characters
like tab and newline are only encoded once, not twice. When decoding
with the ",string" option, the outer string decode works, but the inner
string decode fails, as we are now decoding a JSON string with unescaped
special characters.
The fix we apply here isn't to go back to Marshal, as that would
re-introduce the bug with SetEscapeHTML(false). Instead, we can use a
new encode state from the pool - it results in minimal performance
impact, and even reduces allocs/op further. The performance impact seems
fair, given that we need to check the entire string for characters that
need to be escaped.
name old time/op new time/op delta
Issue34127-8 89.7ns ± 2% 100.8ns ± 1% +12.27% (p=0.000 n=8+8)
name old alloc/op new alloc/op delta
Issue34127-8 40.0B ± 0% 32.0B ± 0% -20.00% (p=0.000 n=8+8)
name old allocs/op new allocs/op delta
Issue34127-8 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=8+8)
Instead of adding another standalone test, we convert an existing
"string tag" test to be table-based, and add another test case there.
One test case from the original CL also had to be amended, due to the
same problem - when escaping '<' due to SetEscapeHTML(true), we need to
end up with double escaping, since we're using ",string".
Fixes #38178.
For #38173.
Change-Id: I2b0df9e4f1d3452fff74fe910e189c930dde4b5b
Reviewed-on: https://go-review.googlesource.com/c/go/+/226498
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
(cherry picked from commit b1a48af7e8ee87cc46e1bbb07f81ac4853e0f27b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/233037 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Michael Pratt [Thu, 7 May 2020 22:13:21 +0000 (18:13 -0400)]
[release-branch.go1.14] runtime: disable preemption in startTemplateThread
When a locked M wants to start a new M, it hands off to the template
thread to actually call clone and start the thread. The template thread
is lazily created the first time a thread is locked (or if cgo is in
use).
stoplockedm will release the P (_Pidle), then call handoffp to give the
P to another M. In the case of a pending STW, one of two things can
happen:
1. handoffp starts an M, which does acquirep followed by schedule, which
will finally enter _Pgcstop.
2. handoffp immediately enters _Pgcstop. This only occurs if the P has
no local work, GC work, and no spinning M is required.
If handoffp starts an M, and must create a new M to do so, then newm
will simply queue the M on newmHandoff for the template thread to do the
clone.
When a stop-the-world is required, stopTheWorldWithSema will start the
stop and then wait for all Ps to enter _Pgcstop. If the template thread
is not fully created because startTemplateThread gets stopped, then
another stoplockedm may queue an M that will never get created, and the
handoff P will never leave _Pidle. Thus stopTheWorldWithSema will wait
forever.
A sequence to trigger this hang when STW occurs can be visualized with
two threads:
Note that the P in T2 is stuck sitting in _Pidle. Since the template
thread isn't running, the new M will not be started complete the
transition to _Pgcstop.
To resolve this, we disable preemption around the assignment of
haveTemplateThread and the creation of the template thread in order to
guarantee that if handTemplateThread is set then the template thread
will eventually exist, in the presence of stops.
Ian Lance Taylor [Wed, 20 May 2020 03:23:58 +0000 (20:23 -0700)]
[release-branch.go1.14] syscall: preserve Windows file permissions for O_CREAT|O_TRUNC
On Windows, calling syscall.Open(file, O_CREAT|O_TRUNC, 0) for a file
that already exists would change the file to be read-only.
That is not how the Unix syscall.Open behaves, so avoid it on
Windows by calling CreateFile twice if necessary.
For #38225
Fixes #39158
Change-Id: I70097fca8863df427cc8a97b9376a9ffc69c6318
Reviewed-on: https://go-review.googlesource.com/c/go/+/234534
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
(cherry picked from commit 567556d78657326c99b8fa84ec2a5ee511a0941b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/234686 Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
[release-branch.go1.14] go/doc: fix detection of whole file examples
After CL 211357 (commit 499dc1c),
hasTests and numDecl were not updated properly for function
declarations with parameters, which affected the whole file
example detection logic. This caused examples like
In CL 187657, I refactored constant conversion logic without realizing
that conversions between int/float and complex types are allowed for
constants (assuming the constant values are representable by the
destination type), but are never allowed for non-constant expressions.
This CL expands convertop to take an extra srcConstant parameter to
indicate whether the source expression is a constant; and if so, to
allow any numeric-to-numeric conversion. (Conversions of values that
cannot be represented in the destination type are rejected by
evconst.)
Fixes #38123.
For #38117.
Change-Id: Id7077d749a14c8fd910be38da170fa5254819f2b
Reviewed-on: https://go-review.googlesource.com/c/go/+/226197
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
(cherry picked from commit 34314280e46da1558bc7f9cd7e8a9ed610cf417b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/232719
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Michael Anthony Knyszek [Fri, 24 Apr 2020 14:46:28 +0000 (14:46 +0000)]
[release-branch.go1.14] runtime: make the scavenger's pacing logic more defensive
This change adds two bits of logic to the scavenger's pacing. Firstly,
it checks to make sure we scavenged at least one physical page, if we
released a non-zero amount of memory. If we try to release less than one
physical page, most systems will release the whole page, which could
lead to memory corruption down the road, and this is a signal we're in
this situation.
Secondly, the scavenger's pacing logic now checks to see if the time a
scavenging operation takes is measured to be exactly zero or negative.
The exact zero case can happen if time update granularity is too large
to effectively capture the time the scavenging operation took, like on
Windows where the OS timer frequency is generally 1ms. The negative case
should not happen, but we're being defensive (against kernel bugs, bugs
in the runtime, etc.). If either of these cases happen, we fall back to
Go 1.13 behavior: assume the scavenge operation took around 10µs per
physical page. We ignore huge pages in this case because we're in
unknown territory, so we choose to be conservative about pacing (huge
pages could only increase the rate of scavenging).
Currently, the scavenger is broken on Windows because the granularity of
time measurement is around 1 ms, which is too coarse to measure how fast
we're scavenging, so we often end up with a scavenging time of zero,
followed by NaNs and garbage values in the pacing logic, which usually
leads to the scavenger sleeping forever.
For #38617.
Fixes #38856.
Change-Id: Iaaa2a4cbb21338e1258d010f7362ed58b7db1af7
Reviewed-on: https://go-review.googlesource.com/c/go/+/229997
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Austin Clements <austin@google.com>
(cherry picked from commit c7915376ce3cdd172bf71ca4127c67f196b8e43e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/232743
Keith Randall [Thu, 2 Apr 2020 19:00:50 +0000 (12:00 -0700)]
[release-branch.go1.14] runtime/race: update some .syso files
Update race detector syso files for some platforms.
There's still 2 more to do, but they might take a while so I'm
mailing the ones I have now.
Note: some arm64 tests did not complete successfully due to out
of memory errors, but I suspect the .syso is correct.
For #14481
For #37485 (I think?)
For #37355
Fixes #38321
Change-Id: I7e7e707a1fd7574855a538ba89dc11acc999c760
Reviewed-on: https://go-review.googlesource.com/c/go/+/226981
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit 041bcb32b59b37ff55ad527083b7bd4ffa4c1a3d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/231222 Reviewed-by: Keith Randall <khr@golang.org>
Michael Anthony Knyszek [Wed, 22 Apr 2020 21:36:11 +0000 (21:36 +0000)]
[release-branch.go1.14] runtime: ensure allocToCache updates searchAddr in a valid way
Currently allocToCache assumes it can move the search address past the
block it allocated the cache from, which violates the property that
searchAddr should always point to mapped memory (i.e. memory represented
by pageAlloc.inUse).
This bug was already fixed once for pageAlloc.alloc in the Go 1.14
release via CL 216697, but that changed failed to take into account
allocToCache.
For #38605.
Fixes #38606.
Change-Id: Id08180aa10d19dc0f9f551a1d9e327a295560dff
Reviewed-on: https://go-review.googlesource.com/c/go/+/229577
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
(cherry picked from commit 287d1ec96c1271de532c6b1160cd9cbbe717ee34)
Reviewed-on: https://go-review.googlesource.com/c/go/+/230377 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Munday [Tue, 14 Apr 2020 14:46:26 +0000 (15:46 +0100)]
[release-branch.go1.14] cmd/compile: fix deallocation of live value copies in regalloc
When deallocating the input register to a phi so that the phi
itself could be allocated to that register the code was also
deallocating all copies of that phi input value. Those copies
of the value could still be live and if they were the register
allocator could reuse them incorrectly to hold speculative
copies of other phi inputs. This causes strange bugs.
No test because this is a very obscure scenario that is hard
to replicate but CL 228060 adds an assertion to the compiler
that does trigger when running the std tests on linux/s390x
without this CL applied. Hopefully that assertion will prevent
future regressions.
Fixes #38443.
Change-Id: Id975dadedd731c7bb21933b9ea6b17daaa5c9e1d
Reviewed-on: https://go-review.googlesource.com/c/go/+/228061
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit 382fe3e2498f2066400e7e7007aa9903440e339d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/230357
Rémy Oudompheng [Thu, 5 Mar 2020 06:59:00 +0000 (07:59 +0100)]
[release-branch.go1.14] math/big: correct off-by-one access in divBasic
The divBasic function computes the quotient of big nats u/v word by word.
It estimates each word qhat by performing a long division (top 2 words of u
divided by top word of v), looks at the next word to correct the estimate,
then perform a full multiplication (qhat*v) to catch any inaccuracy in the
estimate.
In the latter case, "negative" values appear temporarily and carries
must be carefully managed, and the recursive division refactoring
introduced a case where qhat*v has the same length as v, triggering an
out-of-bounds write in the case it happens when computing the top word
of the quotient.
Jay Conrod [Fri, 28 Feb 2020 21:31:19 +0000 (16:31 -0500)]
[release-branch.go1.14] cmd/go: make module zip extraction more robust
Currently, we extract module zip files to temporary directories, then
atomically rename them into place. On Windows, this can fail with
ERROR_ACCESS_DENIED if another process (antivirus) has files open
before the rename. In CL 220978, we repeated the rename operation in a
loop over 500 ms, but this didn't solve the problem for everyone.
A better solution will extract module zip files to their permanent
locations in the cache and will keep a ".partial" marker file,
indicating when a module hasn't been fully extracted (CL 221157).
This approach is not safe if current versions of Go access the module
cache concurrently, since the module directory is detected with a
single os.Stat.
In the interim, this CL makes two changes:
1. Flaky file system operations are repeated over 2000 ms to reduce
the chance of this error occurring.
2. cmd/go will now check for .partial files created by future
versions. If a .partial file is found, it will lock the lock file,
then remove the .partial file and directory if needed.
After some time has passed and Go versions lacking this CL are no
longer supported, we can start extracting module zip files in place.
Updates #37800
Change-Id: I467ee11aa59a90b63cf0e3e761c4fec89d57d3b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/221820
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit 093049b3709eda7537ece92a2991918cf53782d6)
Reviewed-on: https://go-review.googlesource.com/c/go/+/223147
Austin Clements [Wed, 26 Feb 2020 20:12:33 +0000 (15:12 -0500)]
[release-branch.go1.14] runtime: fix rounding in materializeGCProg
materializeGCProg allocates a temporary buffer for unrolling a GC
program. Unfortunately, when computing the size of the buffer, it
rounds *down* the number of bytes needed to store bitmap before
rounding up the number of pages needed to store those bytes. The fact
that it rounds up to pages usually mitigates the rounding down, but
the type from #37470 exists right on the boundary where this doesn't
work:
On 64-bit, this GC bitmap is exactly 8 KiB of zeros, followed by three
one bits. Hence, this needs 8193 bytes of storage, but the current
math in materializeGCProg rounds *down* the three one bits to 8192
bytes. Since this is exactly pageSize, the next step of rounding up to
the page size doesn't mitigate this error, and materializeGCProg
allocates a buffer that is one byte too small. runGCProg then writes
one byte past the end of this buffer, causing either a segfault (if
you're lucky!) or memory corruption.