Cherry Zhang [Fri, 27 Dec 2019 17:02:00 +0000 (12:02 -0500)]
[release-branch.go1.13] runtime: ensure memmove write pointer atomically on ARM64
If a pointer write is not atomic, if the GC is running
concurrently, it may observe a partially updated pointer, which
may point to unallocated or already dead memory. Most pointer
writes, like the store instructions generated by the compiler,
are already atomic. But we still need to be careful in places
like memmove. In memmove, we don't know which bits are pointers
(or too expensive to query), so we ensure that all aligned
pointer-sized units are written atomically.
Bryan C. Mills [Tue, 7 Jan 2020 17:03:28 +0000 (12:03 -0500)]
[release-branch.go1.13] net/http: avoid writing to Transport.ProxyConnectHeader
Previously, we accidentally wrote the Proxy-Authorization header for
the initial CONNECT request to the shared ProxyConnectHeader map when
it was non-nil.
Jason A. Donenfeld [Thu, 21 Nov 2019 15:16:56 +0000 (16:16 +0100)]
[release-branch.go1.13] runtime: do not use PowerRegisterSuspendResumeNotification on systems with "program time" timer
Systems where PowerRegisterSuspendResumeNotification returns ERROR_
FILE_NOT_FOUND are also systems where nanotime() is on "program time"
rather than "real time". The chain for this is:
Opening \\.\PdcPort fails with STATUS_OBJECT_NAME_NOT_FOUND when pdc.sys
hasn't been initialized. Pdc.sys also provides the various hooks for
sleep resumption events, which means if it's not loaded, then our "real
time" timer is actually on "program time". Finally STATUS_OBJECT_NAME_
NOT_FOUND is passed through RtlNtStatusToDosError, which returns ERROR_
FILE_NOT_FOUND. Therefore, in the case where the function returns ERROR_
FILE_NOT_FOUND, we don't mind, since the timer we're using will
correspond fine with the lack of sleep resumption notifications. This
applies, for example, to Docker users.
Updates #35447
Updates #35482
Fixes #35746
Change-Id: I9e1ce5bbc54b9da55ff7a3918b5da28112647eee
Reviewed-on: https://go-review.googlesource.com/c/go/+/211280
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Michael Anthony Knyszek [Mon, 14 Oct 2019 17:33:36 +0000 (17:33 +0000)]
[release-branch.go1.13] runtime: call goready in wakeScavenger instead of ready
This changes fixes an oversight in wakeScavenger which would cause ready
to be called off of the system stack. This change makes it so that
wakeScavenger calls goready, which switches to the system stack before
calling ready.
Fixes #36127.
Change-Id: Icb13f180b4d8fdd47c921eac1b896e3dd49e43b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/200999
Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit 2c87be436bddd9b49f11959adee1ae817cb48ee1)
Reviewed-on: https://go-review.googlesource.com/c/go/+/212103
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Bryan C. Mills [Tue, 5 Nov 2019 16:01:10 +0000 (11:01 -0500)]
[release-branch.go1.13] cmd/go/internal/modfetch: remove non-hermetic test
The test for gopkg.in/yaml.v2@v2 assumes that there are
no future upstream releases. That assumption empirically
does not hold. Backporting fixes to this test is annoying,
and other gopkg.in cases are already reasonably covered,
so remove the problematic test.
Updates #28856
Change-Id: I6455baa1816ac69e02d1ad5d03b82a93e1481a17
Reviewed-on: https://go-review.googlesource.com/c/go/+/205437
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(cherry picked from commit f0390ffc9d461cb84207b5a94c4b645c87673406)
Reviewed-on: https://go-review.googlesource.com/c/go/+/205438 Reviewed-by: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dmitri Shuralyov [Thu, 5 Dec 2019 14:13:21 +0000 (17:13 +0300)]
[release-branch.go1.13] doc: add CherryPickApproved filter to Release History links
Not all closed issues in a given minor milestone are included in that
release, only the ones that have been labeled as CherryPickApproved are.
Update the links to the GitHub issue tracker to include a filter on the
CherryPickApproved label, so that the default view shows only the
backports that were included in a given release. This should more useful
to most people than seeing all backports (considered and approved).
Do this only for Go 1.9.1 and newer releases, as that is when we started
using the CherryPickCandidate and CherryPickApproved labels.
Updates #35988
Fixes #36003
Change-Id: I51e07c1bc3ab9c4a5744e8f668c5470adf78bffe
Reviewed-on: https://go-review.googlesource.com/c/go/+/210117
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Than McIntosh [Thu, 10 Oct 2019 16:11:06 +0000 (12:11 -0400)]
[release-branch.go1.13] cmd/compile: fix spurious R_TLE_LE reloc on android/386
When compiling for GOARCH=386 GOOS=android, the compiler was attaching
R_TLS_LE relocations inappropriately -- as of Go 1.13 the TLS access
recipe for Android refers to a runtime symbol and no longer needs this
type of relocation (which was causing a crash when the linker tried to
process it).
Lynn Boger [Mon, 28 Oct 2019 13:29:40 +0000 (09:29 -0400)]
[release-branch.go1.13] runtime: fix textOff for multiple text sections
If a compilation has multiple text sections, code in
textOff must compare the offset argument against the range
for each text section to determine which one it is in.
The comparison looks like this:
if uintptr(off) >= sectaddr && uintptr(off) <= sectaddr+sectlen
If the off value being compared is equal to sectaddr+sectlen then it
is not within the range of the text section but after it. The
comparison should be just '<'.
Fixes #35211
Change-Id: I114633fd734563d38f4e842dd884c6c239f73c95
Reviewed-on: https://go-review.googlesource.com/c/go/+/203817
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
(cherry picked from commit 0ae9389609f23dc905c58fc2ad7bcc16b770f337)
Reviewed-on: https://go-review.googlesource.com/c/go/+/203819
Run-TryBot: Carlos Amedee <carlos@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Andrew [Wed, 20 Nov 2019 17:06:51 +0000 (12:06 -0500)]
[release-branch.go1.13] all: base64-encode binaries that will cause Apple notarization to fail
Starting with macOS 10.15 (Catalina), Apple now requires all software
distributed outside of the App Store to be notarized. Any binaries we
distribute must abide by a strict set of requirements like code-signing
and having a minimum target SDK of 10.9 (amongst others).
Apple’s notarization service will recursively inspect archives looking to
find notarization candidate binaries. If it finds a binary that does not
meet the requirements or is unable to decompress an archive, it will
reject the entire distribution. From cursory testing, it seems that the
service uses content sniffing to determine file types, so changing
the file extension will not work.
There are some binaries and archives included in our distribution that
are being detected by Apple’s service as potential candidates for
notarization or decompression. As these are files used by tests and some
are intentionally invalid, we don’t intend to ever make them compliant.
As a workaround for this, we base64-encode any binaries or archives that
Apple’s notarization service issues a warning for, as these warnings will
become errors in January 2020.
Updates #34986
Fixes #35748
Change-Id: I106fbb6227b61eb221755568f047ee11103c1680
Reviewed-on: https://go-review.googlesource.com/c/go/+/208118
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(cherry picked from commit 8bbfc51d9ac9ce9472e126cc3654c9a45eceb236)
Reviewed-on: https://go-review.googlesource.com/c/go/+/208219 Reviewed-by: Alexander Rakoczy <alex@golang.org>
Jason A. Donenfeld [Wed, 23 Oct 2019 11:08:46 +0000 (13:08 +0200)]
[release-branch.go1.13] syscall: reenable sysctl on iOS
This was disabled due to a report that the App Store rejects the symbol
__sysctl. However, we use the sysctl symbol, which is fine. The __sysctl
symbol is used by x/sys/unix, which needs fixing instead. So, this
commit reenables sysctl on iOS, so that things like net.InterfaceByName
can work again.
This reverts CL 193843, CL 193844, CL 193845, and CL 193846.
Agniva De Sarker [Sun, 13 Oct 2019 15:03:47 +0000 (20:33 +0530)]
[release-branch.go1.13] cmd/go/internal/work: fix error while passing custom vet tool
For GOROOT packages, we were adding -unsafeptr=false to prevent unsafe.Pointer
checks. But the flag also got passed to invocations of go vet with a custom
vet tool. To prevent this from happening, we add this flag only when no
tools are passed.
Updates #34053
Fixes #34922
Change-Id: I8bcd637fd8ec423d597fcdab2a0ceedd20786019
Reviewed-on: https://go-review.googlesource.com/c/go/+/200957
Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from commit 902d5aa84f8340752c20b93bfd450a6cefcf3952)
Reviewed-on: https://go-review.googlesource.com/c/go/+/201237
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Agniva De Sarker <agniva.quicksilver@gmail.com>
zdjones [Fri, 11 Oct 2019 15:04:47 +0000 (16:04 +0100)]
[release-branch.go1.13-security] cmd/compile: make poset use sufficient conditions for OrderedOrEqual
When assessing whether A <= B, the poset's OrderedOrEqual has a passing
condition which permits A <= B, but is not sufficient to infer that A <= B.
This CL removes that incorrect passing condition.
Having identified that A and B are in the poset, the method will report that
A <= B if any of these three conditions are true:
(1) A and B are the same node in the poset.
- This means we know that A == B.
(2) There is a directed path, strict or not, from A -> B
- This means we know that, at least, A <= B, but A < B is possible.
(3) There is a directed path from B -> A, AND that path has no strict edges.
- This means we know that B <= A, but do not know that B < A.
In condition (3), we do not have enough information to say that A <= B, rather
we only know that B == A (which satisfies A <= B) is possible. The way I
understand it, a strict edge shows a known, strictly-ordered relation (<) but
the lack of a strict edge does not show the lack of a strictly-ordered relation.
The difference is highlighted by the example in #34802, where a bounds check is
incorrectly removed by prove, such that negative indexes into a slice
succeed:
n := make([]int, 1)
for i := -1; i <= 0; i++ {
fmt.Printf("i is %d\n", i)
n[i] = 1 // No Bounds check, program runs, assignment to n[-1] succeeds!!
}
When prove is checking the negative/failed branch from the bounds check at n[i],
in the signed domain we learn (0 > i || i >= len(n)). Because prove can't learn
the OR condition, we check whether we know that i is non-negative so we can
learn something, namely that i >= len(n). Prove uses the poset to check whether
we know that i is non-negative. At this point the poset holds the following
relations as a directed graph:
-1 <= i <= 0
-1 < 0
In poset.OrderedOrEqual, we are testing for 0 <= i. In this case, condition (3)
above is true because there is a non-strict path from i -> 0, and that path
does NOT have any strict edges. Because this condition is true, the poset
reports to prove that i is known to be >= 0. Knowing, incorrectly, that i >= 0,
prove learns from the failed bounds check that i >= len(n) in the signed domain.
When the slice, n, was created, prove learned that len(n) == 1. Because i is
also the induction variable for the loop, upon entering the loop, prove previously
learned that i is in [-1,0]. So when prove attempts to learn from the failed
bounds check, it finds the new fact, i > len(n), unsatisfiable given that it
previously learned that i <= 0 and len(n) = 1.
Fixes #34807
Change-Id: I235f4224bef97700c3aa5c01edcc595eb9f13afc
Reviewed-on: https://go-review.googlesource.com/c/go/+/200759
Run-TryBot: Zach Jones <zachj1@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Giovanni Bajo <rasky@develer.com> Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/201060
Run-TryBot: Alexander Rakoczy <alex@golang.org>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/575398 Reviewed-by: Filippo Valsorda <valsorda@google.com>
zdjones [Fri, 30 Aug 2019 13:41:09 +0000 (14:41 +0100)]
[release-branch.go1.13-security] cmd/compile: rename poset method dominates to reaches
The partially ordered set uses a method named 'dominates' to determine whether
two nodes are partially ordered. Dominates does a depth-first search of the
DAG, beginning at the source node, and returns true as soon as it finds a path
to the target node. In the context of the forest-of-DAGs that makes up the
poset, dominates is not necessarily checking dominance, but is checking
reachability. See the issue tracker for a more detailed discussion of the
difference.
Fortunately, reachability is logically correct everywhere dominates is currently
used in poset.go. Reachability within a DAG is sufficient to establish the
partial ordering (source < target).
This CL changes the name of the method (dominates -> reaches) and updates
all the comments in the file accordingly.
Updates #34807
Change-Id: Ia3a34f7b14b363801d75b05099cfc686035f7d96
Reviewed-on: https://go-review.googlesource.com/c/go/+/192617 Reviewed-by: Giovanni Bajo <rasky@develer.com>
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/201059
Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/575397 Reviewed-by: Filippo Valsorda <valsorda@google.com>
Katie Hockman [Mon, 14 Oct 2019 20:42:21 +0000 (16:42 -0400)]
[release-branch.go1.13-security] crypto/dsa: prevent bad public keys from causing panic
dsa.Verify might currently use a nil s inverse in a
multiplication if the public key contains a non-prime Q,
causing a panic. Change this to check that the mod
inverse exists before using it.
This is a revert of CL 174437 and equivalent to CL 201360.
The size of the params block passed into the KDSA instruction is
incorrect and this appears to result in out-of-bounds writes
that cause a panic in the crypto/x509 tests when run on a machine
that supports KDSA.
Remove this assembly for now. We can revisit the use of the KDSA
instruction in a future release.
zdjones [Fri, 11 Oct 2019 15:04:47 +0000 (16:04 +0100)]
[release-branch.go1.13] cmd/compile: make poset use sufficient conditions for OrderedOrEqual
When assessing whether A <= B, the poset's OrderedOrEqual has a passing
condition which permits A <= B, but is not sufficient to infer that A <= B.
This CL removes that incorrect passing condition.
Having identified that A and B are in the poset, the method will report that
A <= B if any of these three conditions are true:
(1) A and B are the same node in the poset.
- This means we know that A == B.
(2) There is a directed path, strict or not, from A -> B
- This means we know that, at least, A <= B, but A < B is possible.
(3) There is a directed path from B -> A, AND that path has no strict edges.
- This means we know that B <= A, but do not know that B < A.
In condition (3), we do not have enough information to say that A <= B, rather
we only know that B == A (which satisfies A <= B) is possible. The way I
understand it, a strict edge shows a known, strictly-ordered relation (<) but
the lack of a strict edge does not show the lack of a strictly-ordered relation.
The difference is highlighted by the example in #34802, where a bounds check is
incorrectly removed by prove, such that negative indexes into a slice
succeed:
n := make([]int, 1)
for i := -1; i <= 0; i++ {
fmt.Printf("i is %d\n", i)
n[i] = 1 // No Bounds check, program runs, assignment to n[-1] succeeds!!
}
When prove is checking the negative/failed branch from the bounds check at n[i],
in the signed domain we learn (0 > i || i >= len(n)). Because prove can't learn
the OR condition, we check whether we know that i is non-negative so we can
learn something, namely that i >= len(n). Prove uses the poset to check whether
we know that i is non-negative. At this point the poset holds the following
relations as a directed graph:
-1 <= i <= 0
-1 < 0
In poset.OrderedOrEqual, we are testing for 0 <= i. In this case, condition (3)
above is true because there is a non-strict path from i -> 0, and that path
does NOT have any strict edges. Because this condition is true, the poset
reports to prove that i is known to be >= 0. Knowing, incorrectly, that i >= 0,
prove learns from the failed bounds check that i >= len(n) in the signed domain.
When the slice, n, was created, prove learned that len(n) == 1. Because i is
also the induction variable for the loop, upon entering the loop, prove previously
learned that i is in [-1,0]. So when prove attempts to learn from the failed
bounds check, it finds the new fact, i > len(n), unsatisfiable given that it
previously learned that i <= 0 and len(n) = 1.
Fixes #34807
Change-Id: I235f4224bef97700c3aa5c01edcc595eb9f13afc
Reviewed-on: https://go-review.googlesource.com/c/go/+/200759
Run-TryBot: Zach Jones <zachj1@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Giovanni Bajo <rasky@develer.com> Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/201060
Run-TryBot: Alexander Rakoczy <alex@golang.org>
zdjones [Fri, 30 Aug 2019 13:41:09 +0000 (14:41 +0100)]
[release-branch.go1.13] cmd/compile: rename poset method dominates to reaches
The partially ordered set uses a method named 'dominates' to determine whether
two nodes are partially ordered. Dominates does a depth-first search of the
DAG, beginning at the source node, and returns true as soon as it finds a path
to the target node. In the context of the forest-of-DAGs that makes up the
poset, dominates is not necessarily checking dominance, but is checking
reachability. See the issue tracker for a more detailed discussion of the
difference.
Fortunately, reachability is logically correct everywhere dominates is currently
used in poset.go. Reachability within a DAG is sufficient to establish the
partial ordering (source < target).
This CL changes the name of the method (dominates -> reaches) and updates
all the comments in the file accordingly.
Updates #34807
Change-Id: Ia3a34f7b14b363801d75b05099cfc686035f7d96
Reviewed-on: https://go-review.googlesource.com/c/go/+/192617 Reviewed-by: Giovanni Bajo <rasky@develer.com>
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/201059
Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Emmanuel T Odeke [Sun, 13 Oct 2019 22:07:06 +0000 (15:07 -0700)]
[release-branch.go1.13] net/http: fix Transport panic with nil Request.Header
For Go 1.13 we introduced Header.Clone and it returns
nil if a nil Header is cloned. Unfortunately, though,
this exported Header.Clone nil behavior differed from
the old Go 1.12 and earlier internal header clone
behavior which always returned non-nil Headers.
This CL fixes the places where that distinction mattered.
Shenghou Ma [Mon, 30 Sep 2019 13:44:37 +0000 (09:44 -0400)]
[release-branch.go1.13] runtime: fix darwin syscall performance regression
While understanding why syscall.Read is 2x slower on darwin/amd64, I found
out that, contrary to popular belief, the slowdown is not due to the migration
to use libSystem.dylib instead of direct SYSCALLs, i.e., CL 141639 (and #17490),
but due to a subtle change introduced in CL 141639.
Previously, syscall.Read used syscall.Syscall(SYS_READ), whose preamble called
runtime.entersyscall, but after CL 141639, syscall.Read changes to call
runtime.syscall_syscall instead, which in turn calls runtime.entersyscallblock
instead of runtime.entersyscall. And the entire 2x slow down can be attributed
to this change.
I think this is unnecessary as even though syscalls like Read might block, it
does not always block, so there is no need to handoff P proactively for each
Read. Additionally, we have been fine with not handing off P for each Read
prior to Go 1.12, so we probably don't need to change it. This changes restores
the pre-Go 1.12 behavior, where syscall preamble uses runtime.entersyscall,
and we rely on sysmon to take P back from g blocked in syscalls.
Updates #34712
Change-Id: If76e97b5a7040cf1c10380a567c4f5baec3121ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/197938
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit c1635ad8f0bb9fbe5bfbf0a633c78a03930758c4)
Reviewed-on: https://go-review.googlesource.com/c/go/+/200103
Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Bryan C. Mills [Tue, 8 Oct 2019 18:23:43 +0000 (14:23 -0400)]
[release-branch.go1.13] cmd/go: suppress more errors in package-to-module loading
In CL 197059, I suppressed errors if the target package was already found.
However, that does not cover the case of passing a '/v2' module path to
'go get' when the module does not contain a package at its root.
This CL is a minimal fix for that case, intended to be backportable to 1.13.
(Longer term, I intend to rework the version-validation check to treat
all mismatched paths as ErrNotExist.)
Updates #34746
Fixes #34747
Change-Id: Ia963c2ea00fae424812b8f46a4d6c2c668252147
Reviewed-on: https://go-review.googlesource.com/c/go/+/199839
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from commit 421d35cf69f4a18edf96004ba99c01e629a0f79f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/199997
Duco van Amstel [Fri, 4 Oct 2019 13:05:13 +0000 (13:05 +0000)]
[release-branch.go1.13] cmd/go: fix listing of ambiguous paths
Passing ambiguous patterns, ending in `.go`, to `go list` results in them
being interpreted as Go files despite potentially being package references.
This can then result in errors on other package references.
The parsing logic is modified to check for a locally present file
corresponding to any pattern ending in `.go`. If no such file is present
the pattern is considered to be a package reference.
We're also adding a variety of non-regression tests that fail with the
original parsing code but passes after applying the fix.
Updates #34653
Fixes #34694
Change-Id: I073871da0dfc5641a359643f95ac14608fdca09b
GitHub-Last-Rev: 5abc200103ffc122df05422d79cf30c3ba0ee646
GitHub-Pull-Request: golang/go#34663
Reviewed-on: https://go-review.googlesource.com/c/go/+/198459
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit 33683f1d64df0cef2c598a84b741abb5af8abe5e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198957 Reviewed-by: Jay Conrod <jayconrod@google.com>
Jason A. Donenfeld [Fri, 4 Oct 2019 14:48:17 +0000 (16:48 +0200)]
[release-branch.go1.13] syscall: replace mksyscall_windows.go with wrapper to new x/sys home
We replace the existing file with a thin wrapper around its target so
that we don't break anybody's workflow.
This combines CL 198977 and CL 199277.
Fixes #34388
Change-Id: I0d00371c483cb78f4be18fe987df33c79cd40f05
Reviewed-on: https://go-review.googlesource.com/c/go/+/199538
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Anthony Knyszek [Tue, 25 Jun 2019 19:06:57 +0000 (19:06 +0000)]
[release-branch.go1.13] runtime: scavenge on growth instead of inline with allocation
Inline scavenging causes significant performance regressions in tail
latency for k8s and has relatively little benefit for RSS footprint.
We disabled inline scavenging in Go 1.12.5 (CL 174102) as well, but
we thought other changes in Go 1.13 had mitigated the issues with
inline scavenging. Apparently we were wrong.
This CL switches back to only doing foreground scavenging on heap
growth, rather than doing it when allocation tries to allocate from
scavenged space.
Fixes #34556
Change-Id: I1f5df44046091f0b4f89fec73c2cde98bf9448cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/183857
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit eb96f8a57444d174bba500b3a5d2a8b21b7e6d1e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198486 Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
Michael Anthony Knyszek [Tue, 3 Sep 2019 19:54:32 +0000 (19:54 +0000)]
[release-branch.go1.13] runtime: redefine scavenge goal in terms of heap_inuse
This change makes it so that the scavenge goal is defined primarily in
terms of heap_inuse at the end of the last GC rather than next_gc. The
reason behind this change is that next_gc doesn't take into account
fragmentation, and we can fall into situation where the scavenger thinks
it should have work to do but there's no free and unscavenged memory
available.
In order to ensure the scavenge goal still tracks next_gc, we multiply
heap_inuse by the ratio between the current heap goal and the last heap
goal, which describes whether the heap is growing or shrinking, and by
how much.
Finally, this change updates the documentation for scavenging and
elaborates on why the scavenge goal is defined the way it is.
Fixes #34149
Change-Id: I8deaf87620b5dc12a40ab8a90bf27932868610da
Reviewed-on: https://go-review.googlesource.com/c/go/+/193040
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit 9b30811280427a6d50d2558f316d62210e948656)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198487
Run-TryBot: Andrew Bonventre <andybons@golang.org>
Austin Clements [Mon, 12 Aug 2019 18:54:28 +0000 (14:54 -0400)]
[release-branch.go1.13] runtime: grow the heap incrementally
Currently, we map and grow the heap a whole arena (64MB) at a time.
Unfortunately, in order to fix #32828, we need to switch from
scavenging inline with allocation back to scavenging on heap growth,
but heap-growth scavenging happens in large jumps because we grow the
heap in large jumps.
In order to prepare for better heap-growth scavenging, this CL
separates mapping more space for the heap from actually "growing" it
(tracking the new space with spans). Instead, growing the heap keeps
track of the "current arena" it's growing into. It track that with new
spans as needed, and only maps more arena space when the current arena
is inadequate. The effect to the user is the same, but this will let
us scavenge on much smaller increments of heap growth.
There are two slightly subtleties to this change:
1. If an allocation requires mapping a new arena and that new arena
isn't contiguous with the current arena, we don't want to lose the
unused space in the current arena, so we have to immediately track
that with a span.
2. The mapped space must be accounted as released and idle, even
though it isn't actually tracked in a span.
For #32828, since this makes heap-growth scavenging far more
effective, especially at small heap sizes. For example, this change is
necessary for TestPhysicalMemoryUtilization to pass once we remove
inline scavenging.
Updates #34556
Change-Id: I300e74a0534062467e4ce91cdc3508e5ef9aa73a
Reviewed-on: https://go-review.googlesource.com/c/go/+/189957
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit f18109d7e30c8d1a6e1c87ba3458499ac7ab2a79)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198485
Run-TryBot: Andrew Bonventre <andybons@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Jason A. Donenfeld [Tue, 27 Aug 2019 12:46:16 +0000 (06:46 -0600)]
[release-branch.go1.13] runtime: monitor for suspend/resume to kick timeouts
Starting in Windows 8, the wait functions don't take into account
suspend time, even though the monotonic counters do. This results in
timer buckets stalling on resume. Therefore, this commit makes it so
that on resume, we return from the wait functions and recalculate the
amount of time left to wait.
This is a cherry pick of CL 191957 and its cleanup, CL 198417.
Updates #31528
Fixes #34130
Change-Id: I0db02cc72188cb620954e87a0180e0a3c83f4a56
Reviewed-on: https://go-review.googlesource.com/c/go/+/193607
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Michael Hendricks [Wed, 2 Oct 2019 15:30:51 +0000 (09:30 -0600)]
[release-branch.go1.13] net: avoid an infinite loop in LookupAddr
If a request for a PTR record returned a response with a non-PTR
answer, goLookupPTR would loop forever. Skipping non-PTR answers
guarantees progress through the DNS response.
Michael Anthony Knyszek [Thu, 5 Sep 2019 16:34:00 +0000 (16:34 +0000)]
[release-branch.go1.13] runtime: fix lock acquire cycles related to scavenge.lock
There are currently two edges in the lock cycle graph caused by
scavenge.lock: with sched.lock and mheap_.lock. These edges appear
because of the call to ready() and stack growths respectively.
Furthermore, there's already an invariant in the code wherein
mheap_.lock must be acquired before scavenge.lock, hence the cycle.
The fix to this is to bring scavenge.lock higher in the lock cycle
graph, such that sched.lock and mheap_.lock are only acquired once
scavenge.lock is already held.
To faciliate this change, we move scavenger waking outside of
gcSetTriggerRatio such that it doesn't have to happen with the heap
locked. Furthermore, we check scavenge generation numbers with the heap
locked by using gopark instead of goparkunlock, and specify a function
which aborts the park should there be any skew in generation count.
Fixes #34150.
Change-Id: I3519119214bac66375e2b1262b36ce376c820d12
Reviewed-on: https://go-review.googlesource.com/c/go/+/191977
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit 62e415655238a3c0103c1b70e6805edf8193c543)
Reviewed-on: https://go-review.googlesource.com/c/go/+/197501 Reviewed-by: Austin Clements <austin@google.com>
Jay Conrod [Fri, 13 Sep 2019 17:58:58 +0000 (13:58 -0400)]
[release-branch.go1.13] cmd/go: don't include package dir in cache key when -trimpath is set
The '-trimpath' flag tells 'go build' to trim any paths from the
output files that are tied to the current workspace or toolchain. When
this flag is set, we do not need to include the package directory in
the text hashed to construct the action ID for each package.
Updates #33772
Fixes #34326
Change-Id: I20b902d2f58019709b15864ca79aa0d9255ae707
Reviewed-on: https://go-review.googlesource.com/c/go/+/195318
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit aa680c0c49b55722a72ad3772e590cd2f9af541d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198259
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
Also added a test to ensure that any interactions
between TimeoutHandler and Flusher result in the
correct status code and body, but also that we don't
get superfluous logs from stray writes as was seen
in the bug report.
[release-branch.go1.13-security] net/textproto: don't normalize headers with spaces before the colon
RFC 7230 is clear about headers with a space before the colon, like
X-Answer : 42
being invalid, but we've been accepting and normalizing them for compatibility
purposes since CL 5690059 in 2012.
On the client side, this is harmless and indeed most browsers behave the same
to this day. On the server side, this becomes a security issue when the
behavior doesn't match that of a reverse proxy sitting in front of the server.
For example, if a WAF accepts them without normalizing them, it might be
possible to bypass its filters, because the Go server would interpret the
header differently. Worse, if the reverse proxy coalesces requests onto a
single HTTP/1.1 connection to a Go server, the understanding of the request
boundaries can get out of sync between them, allowing an attacker to tack an
arbitrary method and path onto a request by other clients, including
authentication headers unknown to the attacker.
This was recently presented at multiple security conferences:
https://portswigger.net/blog/http-desync-attacks-request-smuggling-reborn
net/http servers already reject header keys with invalid characters.
Simply stop normalizing extra spaces in net/textproto, let it return them
unchanged like it does for other invalid headers, and let net/http enforce
RFC 7230, which is HTTP specific. This loses us normalization on the client
side, but there's no right answer on the client side anyway, and hiding the
issue sounds worse than letting the application decide.
Fixes CVE-2019-16276
Change-Id: I6d272de827e0870da85d93df770d6a0e161bbcf1
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/549719 Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>
(cherry picked from commit 1280b868e82bf173ea3e988be3092d160ee66082)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/558935 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Tom Thorogood [Sat, 14 Sep 2019 01:16:20 +0000 (01:16 +0000)]
[release-branch.go1.13] net/http: fix HTTP/2 idle pool tracing
CL 140357 caused HTTP/2 connections to be put in the idle pool, but
failed to properly guard the trace.GotConn call in getConn. dialConn
returns a minimal persistConn with conn == nil for HTTP/2 connections.
This persistConn was then returned from queueForIdleConn and caused the
httptrace.GotConnInfo passed into GotConn to have a nil Conn field.
HTTP/2 connections call GotConn themselves so leave it for HTTP/2 to call
GotConn as is done directly below.
Fixes #34285
Change-Id: If54bfaf6edb14f5391463f908efbef5bb8a5d78e
GitHub-Last-Rev: 2b7d66a1ce66b4424c4d0fca2b8e8b547d874136
GitHub-Pull-Request: golang/go#34283
Reviewed-on: https://go-review.googlesource.com/c/go/+/195237 Reviewed-by: Michael Fraenkel <michael.fraenkel@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 582d5194faec20c475ab93b45cf0520253dec4a9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/196579 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
[release-branch.go1.13] cmd/go/internal/modfetch/codehost: work around an apparent bug in 'git fetch --unshallow'
When 'git fetch' is passed the '--unshallow' flag, it assumes that the
local and remote refs are equal.¹ However, we were fetching an
expanded set of refs explicitly in the same command, violating that
assumption.
Now we first expand the set of refs, then unshallow the repo in a
separate fetch. Empirically, this seems to work, whereas the opposite
order does not.
Jay Conrod [Fri, 13 Sep 2019 15:10:36 +0000 (11:10 -0400)]
[release-branch.go1.13] cmd/go: fix link error for -coverpkg in GOPATH mode
If a generated test main package transitively depends on a main
package, the main package will now always be rebuilt as a library and
will not be compiled with '-p main'.
This expands the fix for #30907, which only applied to packages with
the BuildInfo set (main packages built in module mode). Linking
multiple packages with BuildInfo caused link errors, but it appears
these errors apply to some symbols in GOPATH mode.
Fixes #34223
Change-Id: Ic1e53437942269a950dd7e45d163707922c92edd
Reviewed-on: https://go-review.googlesource.com/c/go/+/195279
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 24781a1faf62ac5c9a553af6f46b787e972f5539)
Reviewed-on: https://go-review.googlesource.com/c/go/+/195281
[release-branch.go1.13] cmd/go: suppress errors in package-to-module queries if the package is already found
In CL 173017, I changed the package-to-module query logic to query all
possible module paths in parallel in order to reduce latency. (For
long package paths, most such paths will not exist and will fail with
little overhead.)
The module resolution algorithm treats various kinds of non-existence
as “soft errors”, to be reported only if package resolution fails, but
treats any remaining errors as hard errors that should fail the query.
Unfortunately, that interacted badly with the +incompatible version
validation added in CL 181881, causing a regression in the 'direct'
fetch path for modules using the “major branch” layout¹ with a post-v1
version on the repository's default branch. Because we did not
interpret a mismatched module path as “no such module”, a go.mod file
specifying the path 'example.com/foo/v2' would cause the search for
module 'example.com/foo' to error out. (That regression was not caught
ahead of time due to a lack of test coverage for 'go get' on a package
within a /vN module.)
The promotion of hard errors during parallel search also made the 'go'
command less tolerant of servers that advertise 'go-import' tags for
nonexistent repositories. CL 194561 mitigated that problem for HTTP
servers that return code 404 or 410 for a nonexistent repository, but
unfortunately a few servers in common use (notably GitLab and
pre-1.9.3 releases of Gitea) do not.
This change mitigates both of those failure modes by ignoring
“miscellaneous” errors from shorter module paths if the requested
package pattern was successfully matched against a module with a
longer path.
Rhys Hiltner [Thu, 29 Aug 2019 16:49:36 +0000 (09:49 -0700)]
[release-branch.go1.13] cmd/cover: skip go list when profile is empty
Only call "go list" when explicitly listing packages. An empty coverage
profile references no packages, and would otherwise lead to "go list"
implicitly looking at the package in "." (which might not exist).
Fixes #33984
Change-Id: I02d4e374405d86f03d105fe14648aa03b4d2284c
Reviewed-on: https://go-review.googlesource.com/c/go/+/192340 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 9d480edadc6144d9f9f5a896d729d1642e46083b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192722
Jay Conrod [Tue, 10 Sep 2019 20:06:12 +0000 (16:06 -0400)]
[release-branch.go1.13] cmd/go: strip trailing slash from versioned arguments
'go get' accepts arguments of the form path@version, and it passes
them through search.CleanPatterns before querying proxies. With this
change, CleanPatterns preserves text after '@' and will strip trailing
slashes from the patn.
Previously, we did not strip trailing slashes when a version was
present, which caused proxy base URL validation to fail. Module paths
that end with ".go" (for example, github.com/nats-io/nats.go) use
trailing slashes to prevent 'go build' and other commands from
interpreting packages as source file names, so this caused unnecessary
problems for them.
Fixes #34243
Change-Id: Id3730c52089e52f1cac446617c20132a3021a808
Reviewed-on: https://go-review.googlesource.com/c/go/+/194600
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit 8875fb97c5cadbc6f02e4ce89efa586023c0a777)
Reviewed-on: https://go-review.googlesource.com/c/go/+/194687
Kevin Gillette [Mon, 26 Aug 2019 03:30:21 +0000 (03:30 +0000)]
[release-branch.go1.13] net/http: make docs refer to Context.Value as a getter instead of context.WithValue
The doc comments of both ServerContextKey and LocalAddrContextKey both suggest that context.WithValue can be used to access (get) properties of the server or connection. This PR fixes those comments to refer to Context.Value instead.
Tom Thorogood [Fri, 2 Aug 2019 07:53:44 +0000 (07:53 +0000)]
[release-branch.go1.13] encoding/json: revert Compact HTML escaping documentation
This partly reverts CL 173417 as it incorrectly documented that Compact
performed HTML escaping and the output was safe to embed inside HTML
<script> tags. This has never been true.
Although Compact does escape U+2028 and U+2029, it doesn't escape <, >
or &. Compact is thus only performing a subset of HTML escaping and it's
output is not safe to embed inside HTML <script> tags.
A more complete fix would be for Compact to either never perform any
HTML escaping, as it was prior to CL 10883045, or to actually perform
the same HTML escaping as HTMLEscape. Neither change is likely safe
enough for go1.13.
Fixes #34006
Updates #30357
Change-Id: I912f0fe9611097d988048b28228c4a5b985080ba
GitHub-Last-Rev: aebababc9233c5705785b225377e80096d4bb8c4
GitHub-Pull-Request: golang/go#33427
Reviewed-on: https://go-review.googlesource.com/c/go/+/188717 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 79669dc705aa8fcddc8f9407c568e4317d619dfe)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192747 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
Dmitri Shuralyov [Wed, 28 Aug 2019 17:34:16 +0000 (13:34 -0400)]
[release-branch.go1.13] CONTRIBUTORS: second round of updates for Go 1.13
This update has been automatically generated using the updatecontrib
command:
cd gotip
go run golang.org/x/build/cmd/updatecontrib
With minor manual changes based on publicly available information
to canonicalize letter case for a few names.
Actions taken (relative to CONTRIBUTORS at origin/master):
Added Albert Teoh <albert.teoh@gmail.com>
Added Allen Li <ayatane@google.com>
Added Anderson Queiroz <contato@andersonq.eti.br>
Added Andrew Todd <andrew.todd@wework.com>
Added Artem Kolin <artemkaxboy@gmail.com>
Added Bharath Thiruveedula <tbharath91@gmail.com>
Added Christian Muehlhaeuser <muesli@gmail.com>
Added Darren McCleary <darren.rmc@gmail.com>
Added David Finkel <david.finkel@gmail.com>
Added Eddie Scholtz <escholtz@google.com>
Added GitHub User tatsumack (4510569) <tatsu.mack@gmail.com>
Added GitHub User utkarsh-extc (53217283) <53217283+utkarsh-extc@users.noreply.github.com>
Added GitHub User yuanhh (1298735) <yuan415030@gmail.com>
Added Illya Yalovyy <yalovoy@gmail.com>
Added James Eady <jmeady@google.com>
Added Jan Steinke <jan.steinke@gmail.com>
Added Javier Revillas <jrevillas@massivedynamic.io>
Added Jordi Martin <jordimartin@gmail.com>
Added Jorge Araya <jorgejavieran@yahoo.com.mx>
Added Kelly Heller <pestophagous@gmail.com>
Added Kevin Gillette <extemporalgenome@gmail.com>
Added Mark Glines <mark@glines.org>
Added Nao Yonashiro <owan.orisano@gmail.com>
Added Pascal Dierich <pascal@pascaldierich.com>
Added Pure White <wu.purewhite@gmail.com>
Added Sam Arnold <sarnold64@bloomberg.net>
Added Sander van Harmelen <sander@vanharmelen.nl>
Added Sergei Zagurskii <gvozdoder@gmail.com>
Added Shivani Singhal <shivani.singhal2804@gmail.com>
Added Stefan Baebler <sbaebler@outbrain.com>
Added Tianon Gravi <admwiggin@gmail.com>
Added Toshihiro Shiino <shiino.toshihiro@gmail.com>
Added Wagner Riffel <wgrriffel@gmail.com>
Used GitHub User tatsumack (4510569) form for tatsumack <tatsu.mack@gmail.com> https://github.com/golang/website/commit/7eeb7ef [website]
Used GitHub User utkarsh-extc (53217283) form for utkarsh-extc <53217283+utkarsh-extc@users.noreply.github.com> https://github.com/golang/sys/commit/51ab0e2 [sys]
Used GitHub User yuanhh (1298735) form for yuanhh <yuan415030@gmail.com> https://github.com/golang/crypto/commit/60c769a [crypto]
Used GitHub name "Akhil Indurti" for smasher164 <aindurti@gmail.com> https://github.com/golang/go/commit/5ca44dc403 [build exp go sys]
Used GitHub name "Artem Kolin" for artemkaxboy <artemkaxboy@gmail.com> https://github.com/golang/go/commit/e881604d1c [go]
Used GitHub name "Ivan Markin" for nogoegst <nogoegst@users.noreply.github.com> https://github.com/golang/go/commit/a1addf15df [go]
Used GitHub name "Keiji Yoshida" for yosssi <yoshida.keiji.84@gmail.com> https://github.com/golang/lint/commit/ac6833c [lint]
Used GitHub name "Michalis Kargakis" for kargakis <mkargaki@redhat.com> https://github.com/golang/go/commit/e243d242d7 [go]
Used GitHub name "Roberto Clapis" for Roberto <empijei@users.noreply.github.com> https://github.com/golang/go/commit/963776e689 [go]
Used GitHub name "Robin Eklind" for mewmew <rnd0x00@gmail.com> https://github.com/golang/go/commit/b8620afb8d [blog go proposal.git]
Updates #12042
Change-Id: I1b21a18138849c537048558dd4324a823ba12a14
Reviewed-on: https://go-review.googlesource.com/c/go/+/192099 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit d0eaec79f95c2ad04d58e42c8e7cb123200cdfd5)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192178 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Bryan C. Mills [Tue, 27 Aug 2019 18:18:48 +0000 (14:18 -0400)]
[release-branch.go1.13] cmd/go/internal/get: remove '--' separator from 'git ls-remote' command
'git ls-remote' started recognizing the '--' separator at some point
after 2.7.4, but git defaults to version 2.7.4 on Ubuntu 16.04 LTS,
which remains supported by Ubuntu until April 2021.
We added '--' tokens to most VCS commands as a defensive measure in
CL 181237, but it isn't strictly necessary here because the 'scheme'
argument to our template is chosen from a predefined list: we can
safely drop it to retain compatibility.
Bryan C. Mills [Mon, 26 Aug 2019 20:44:36 +0000 (16:44 -0400)]
[release-branch.go1.13] net/http: fix wantConnQueue memory leaks in Transport
I'm trying to keep the code changes minimal for backporting to Go 1.13,
so it is still possible for a handful of entries to leak,
but the leaks are now O(1) instead of O(N) in the steady state.
Longer-term, I think it would be a good idea to coalesce idleMu with
connsPerHostMu and clear entries out of both queues as soon as their
goroutines are done waiting.
Cherry Zhang [Fri, 23 Aug 2019 18:40:20 +0000 (14:40 -0400)]
[release-branch.go1.13] cmd/vendor: update vendored x/arch repo to 8a70ba74b3a1
Update vendored x/arch repo to pick up the fix of issue #33802.
This is done with the following commands:
$ cd $GOROOT/src/cmd
$ go get -d golang.org/x/arch@latest
go: finding golang.org/x/arch latest
go: downloading golang.org/x/arch v0.0.0-20190815191158-8a70ba74b3a1
go: extracting golang.org/x/arch v0.0.0-20190815191158-8a70ba74b3a1
$ go mod tidy
$ go mod vendor
Updates #33802.
Fixes #33811.
Change-Id: I0a44f1d83d6f573124cea1f099378b1c851f3feb
Reviewed-on: https://go-review.googlesource.com/c/go/+/191619
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from commit 66ff373911a87140319a6550e4e1b2c6043b1329)
Reviewed-on: https://go-review.googlesource.com/c/go/+/191737
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Emmanuel T Odeke [Wed, 21 Aug 2019 05:09:31 +0000 (23:09 -0600)]
[release-branch.go1.13] doc/go1.13: document _ between digits for math/big, strconv
Document that:
* math/big.Float.Parse
* math/big.Int.SetString
* strconv.ParseFloat
* strconv.ParseInt
* strconv.ParseUint
now accept underscores to group digits only if base = 0,
as per the Go 2 language changes.
Updates #32815
Change-Id: Id45bd803a18442436419739297e8aed0d32ca56c
Reviewed-on: https://go-review.googlesource.com/c/go/+/191077 Reviewed-by: Robert Griesemer <gri@golang.org>
(cherry picked from commit e764432605cb26478067890b49f4f16019c9a9b9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/191477
Run-TryBot: Andrew Bonventre <andybons@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
http2: limit number of control frames in server send queue
An attacker could cause servers to queue an unlimited number of PING
ACKs or RST_STREAM frames by soliciting them and not reading them, until
the program runs out of memory.
Limit control frames in the queue to a few thousands (matching the limit
imposed by other vendors) by counting as they enter and exit the scheduler,
so the protection will work with any WriteScheduler.
Once the limit is exceeded, close the connection, as we have no way to
communicate with the peer.
Change-Id: I842968fc6ed3eac654b497ade8cea86f7267886b
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/525552 Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>
This change was generated with cmd/go and cmd/bundle:
$ go get -u golang.org/x/net
$ go mod tidy
$ go mod vendor
$ go generate net/http
Fixes CVE-2019-9512 and CVE-2019-9514
Fixes #33606
Filippo Valsorda [Tue, 20 Aug 2019 21:29:04 +0000 (17:29 -0400)]
Revert "encoding/json: avoid work when unquoting strings"
This reverts CL 151157.
CL 151157 introduced a crash when decoding into ",string" fields. It
came with a moderate speedup, so at this stage of the release cycle
let's just revert it, and reapply it in Go 1.14 with the fix in CL 190659.
Also applied the test cases from CL 190659.
Updates #33728
Change-Id: Ie46e2bc15224b251888580daf6b79d5865f3878e
Reviewed-on: https://go-review.googlesource.com/c/go/+/190909
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>