Chris Broadfoot [Thu, 19 Jan 2017 20:36:22 +0000 (12:36 -0800)]
[release-branch.go1.8] all: merge master into release-branch.go1.8
6593d8650d go/ast: fix Object's doc comment about Data c1730ae424 runtime: force workers out before checking mark roots d10eddcba3 testing: make parallel t.Run safe again 2c8b70eacf crypto/x509: revert SystemCertPool implementation for Windows fcfd91858b doc/go1.8: document Plan 9 requirements 81a61a96c9 runtime: for plugins, don't add duplicate itabs f674537cc9 README.md: update and simplify d8711919db cmd/go: fix bug help message 48d8edb5b2 crypto/tls: disable CBC cipher suites with SHA-256 by default 92ecd78933 cmd/compile: add ZeroWB case in writebarrier 787125abab doc: 2017 is the Year of the Gopher 5b708a6b6a cmd/compile: lvalues are only required for == when calling runtime fns e83d506714 vendor/golang_org/x/crypto/poly1305: revendor to pick up fix for #18673 76f981c8d8 net/http: skip TestServerHijackGetsBackgroundByte on Plan 9 e395e3246a net/http: skip TestServerHijackGetsBackgroundByte_big on Plan 9 6a3c6c0de8 net/http: add another hijack-after-background-read test 467109bf56 all: test adjustments for the iOS builder b2a3b54b95 net/http: make sure Hijack's bufio.Reader includes pre-read background byte 593ea3b360 cmd/go, misc: rework cwd handling for iOS tests 0642b8a2f1 syscall: export Fsid.X__val on s390x 4601eae6ba doc/gdb: mention GOTRACEBACK=crash 4c4c5fc7a3 misc/cgo/testplugin: test that types and itabs are unique 22689c4450 reflect: keep makeFuncImpl live across makeFuncStub 9cf06ed6cd cmd/link: only exclude C-only symbols on darwin 9c3630f578 compress/flate: avoid large stack growth in fillDeflate 4f0aac52d9 cmd/go: add comment about SIGUSR2 on iOS 333f764df3 cmd/go, misc: switch from breakpoint to SIGUSR2 39e31d5ec0 doc/go1.8: update timezone database version 08da8201ca misc/cgo/testshared: test that types and itabs are unique fdde7ba2a2 runtime: avoid clobbering C callee-save register in cgoSigtramp f65abf6ddc cmd/compile: hide testdclstack behind debug flag 641ef2a733 compress/gzip: skip TestGZIPFilesHaveZeroMTimes on non-builders 0724aa813f crypto/dsa: gofmt ac05542985 net/http: deflake TestRetryIdempotentRequestsOnError b842c9aac7 doc: remove inline styles
Austin Clements [Wed, 18 Jan 2017 02:58:10 +0000 (21:58 -0500)]
runtime: force workers out before checking mark roots
Currently we check that all roots are marked as soon as gcMarkDone
decides to transition from mark 1 to mark 2. However, issue #16083
indicates that there may be a race where we try to complete mark 1
while a worker is still scanning a stack, causing the root mark check
to fail.
We don't yet understand this race, but as a simple mitigation, move
the root check to after gcMarkDone performs a ragged barrier, which
will force any remaining workers to finish their current job.
Updates #16083. This may "fix" it, but it would be better to
understand and fix the underlying race.
Keith Randall [Wed, 11 Jan 2017 23:14:06 +0000 (15:14 -0800)]
runtime: for plugins, don't add duplicate itabs
We already do this for shared libraries. Do it for plugins also.
Suggestions on how to test this would be welcome.
I'd like to get this in for 1.8. It could lead to mysterious
hangs when using plugins.
Fixes #18676
Change-Id: I03209b096149090b9ba171c834c5e59087ed0f92
Reviewed-on: https://go-review.googlesource.com/35117 Reviewed-by: David Crawshaw <crawshaw@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Filippo Valsorda [Mon, 16 Jan 2017 23:54:45 +0000 (23:54 +0000)]
crypto/tls: disable CBC cipher suites with SHA-256 by default
As is, they were fully vulnerable to the Lucky13 attack. The SHA1
variants implement limited countermeasures (see f28cf8346c4) but the
SHA256 ones are apparently used rarely enough (see 8741504888b) that
it's not worth the extra code.
Instead, disable them by default and update the warning.
Updates #13385
Updates #15487
Change-Id: I45b8b716001e2fa0811b17e25be76e2512e5abb2
Reviewed-on: https://go-review.googlesource.com/35290 Reviewed-by: Adam Langley <alangley@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Matt Layher <mdlayher@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Cherry Zhang [Fri, 13 Jan 2017 20:42:50 +0000 (15:42 -0500)]
cmd/compile: add ZeroWB case in writebarrier
It looks like it should be there, although I couldn't find a test
case that fails without it. ZeroWB is probably never generated now:
zeroing an initialized heap object is done by making an autotmp on
stack, zeroing it, and copying (typedmemmove) to heap.
Brad Fitzpatrick [Mon, 16 Jan 2017 01:48:42 +0000 (01:48 +0000)]
doc: 2017 is the Year of the Gopher
Change-Id: Iac713ae1f322f893c92b3fc47fe9b5719052f9eb
Reviewed-on: https://go-review.googlesource.com/35240 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: David Symonds <dsymonds@golang.org> Reviewed-by: Minux Ma <minux@golang.org>
David Crawshaw [Sat, 14 Jan 2017 03:08:27 +0000 (22:08 -0500)]
all: test adjustments for the iOS builder
The working directory is now adjusted to match the typical Go test
working directory in main, as the old trick for adjusting earlier
stopped working with the latest version of LLDB bugs.
That means the small number of places where testdata files are
read before main is called no longer work. This CL adjusts those
reads to happen after main is called. (This has the bonus effect of
not reading some benchmark testdata files in all.bash.)
Fixes compress/bzip2, go/doc, go/parser, os, and time package
tests on the iOS builder.
Brad Fitzpatrick [Fri, 13 Jan 2017 21:43:56 +0000 (21:43 +0000)]
net/http: make sure Hijack's bufio.Reader includes pre-read background byte
Previously, if the Hijack called stopped the background read call
which read a byte, that byte was sitting in memory, buffered, ready to
be Read by Hijack's returned bufio.Reader, but it wasn't yet in the
bufio.Reader's buffer itself, so bufio.Reader.Buffered() reported 1
byte fewer.
This matters for callers who wanted to stitch together any buffered
data (with bufio.Reader.Peek(bufio.Reader.Buffered())) with Hijack's
returned net.Conn. Otherwise there was no way for callers to know a
byte was read.
David Crawshaw [Fri, 13 Jan 2017 12:18:40 +0000 (07:18 -0500)]
cmd/go, misc: rework cwd handling for iOS tests
Another change in behvaior (bug) in LLDB. Despite the fact that
LLDB can dump the symtab of our test binaries and show the function
addresses, it can no longer call the functions. This means the chdir
trick on signal is failing.
This CL uses a new trick. For iOS, the exec script passes the change
in directory as an argument, and it is processed early by the test
harness generated by cmd/go.
For the iOS builders.
Change-Id: I8f5d0f831fe18de99f097761f89c5184d5bf2afb
Reviewed-on: https://go-review.googlesource.com/35152 Reviewed-by: Elias Naur <elias.naur@gmail.com>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Munday [Fri, 30 Sep 2016 15:12:01 +0000 (11:12 -0400)]
syscall: export Fsid.X__val on s390x
mkpost.go replaces all variables prefixed with 'X_' with '_' on s390x
because most of them do not need to be exposed. X__val is being used
by a third party library so it turns out we do need to expose it on
s390x (it is already exposed on all other Linux architectures).
Alberto Donizetti [Mon, 9 Jan 2017 18:11:58 +0000 (19:11 +0100)]
doc/gdb: mention GOTRACEBACK=crash
Also fix a couple of other errors.
Fixes #6877
Change-Id: I94c81c5847cc7b0adab19418e71687bc2ee7fe94
Reviewed-on: https://go-review.googlesource.com/34960 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Austin Clements [Thu, 12 Jan 2017 21:54:42 +0000 (16:54 -0500)]
reflect: keep makeFuncImpl live across makeFuncStub
When traceback sees reflect.makeFuncStub (or reflect.methodValueCall)
on the stack, it expects to be able to get the *reflect.makeFuncImpl
(or *reflect.methodValue) for that call from the first outgoing
argument slot of makeFuncStub/methodValueCall.
However, currently this object isn't necessarily kept live across
makeFuncStub. This means it may get garbage collected while in a
reflect call and reused for something else. If we then try to
traceback, the runtime will see a corrupted makeFuncImpl object and
panic. This was not a problem in previous releases because we always
kept arguments live across the whole function. This became a problem
when we stopped doing this.
Fix this by using reflect.KeepAlive to keep the
makeFuncImpl/methodValue live across all of callReflect/callMethod,
which in turn keeps it live as long as makeFuncStub/methodValueCall
are on the stack.
David Crawshaw [Thu, 12 Jan 2017 22:35:53 +0000 (17:35 -0500)]
cmd/link: only exclude C-only symbols on darwin
C-only symbols are excluded from pclntab because of a quirk of darwin,
where functions are referred to by an exported symbol so dynamic
relocations de-duplicate to the host binary module and break unwinding.
This doesn't happen on ELF systems because the linker always refers to
unexported module-local symbols, so we don't need this condition.
And the current logic for excluding some functions breaks the module
verification code in moduledataverify1. So disable this for plugins
on linux.
(In 1.9, it will probably be necessary to introduce a module-local
symbol reference system on darwin to fix a different bug, so all of
this onlycsymbol code made be short-lived.)
With this CL, the tests in CL 35116 pass.
Change-Id: I517d7ca4427241fa0a91276c462827efb9383be9
Reviewed-on: https://go-review.googlesource.com/35190 Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Joe Tsai [Thu, 12 Jan 2017 07:03:15 +0000 (23:03 -0800)]
compress/flate: avoid large stack growth in fillDeflate
Ranging over an array causes the array to be copied over to the
stack, which cause large re-growths. Instead, we should iterate
over slices of the array.
Also, assigning a large struct literal uses the stack even
though the actual fields being populated are small in comparison
to the entirety of the struct (see #18636).
Fixing the stack growth does not alter CPU-time performance much
since the stack-growth and copying was such a tiny portion of the
compression work:
David Crawshaw [Sun, 8 Jan 2017 01:41:00 +0000 (17:41 -0800)]
cmd/go, misc: switch from breakpoint to SIGUSR2
The iOS test harness has set a breakpoint early in the life of Go
programs so that it can change the current working directory using
information only available from the host debugger. Somewhere in the
upgrade to iOS 10 / XCode 8.2, breakpoints stopped working. This
may be an LLDB bug, or a bug in the ios-deploy LLDB scripts, it's
not clear.
Work around the problem by giving up on breakpoints. Instead, early
in the life of every test binary built for iOS, send (and ignore) a
SIGUSR2 signal. The debugger will catch this, giving the script
go_darwin_arm_exec a chance to change the working directory.
For the iOS builders.
Change-Id: I7476531985217d0c76bc176904c48379210576c2
Reviewed-on: https://go-review.googlesource.com/34926 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Shenghou Ma [Thu, 12 Jan 2017 03:48:34 +0000 (22:48 -0500)]
doc/go1.8: update timezone database version
Fixes #18623.
Change-Id: Ic965f5f7088c3270adbca7162226be486d1b9b4e
Reviewed-on: https://go-review.googlesource.com/35130 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Wed, 11 Jan 2017 22:39:41 +0000 (17:39 -0500)]
runtime: avoid clobbering C callee-save register in cgoSigtramp
Use R11 (a caller-saved temp register) instead of RBX (a callee-saved
register).
I believe this only affects linux/amd64, since it is the only platform
with a non-trivial cgoSigtramp implementation.
Updates #18328.
Change-Id: I3d35c4512624184d5a8ece653fa09ddf50e079a2
Reviewed-on: https://go-review.googlesource.com/35068 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
David Chase [Tue, 10 Jan 2017 13:29:34 +0000 (08:29 -0500)]
cmd/compile: disable flaky test
The test is inherently racy and vulnerable to starvation,
and within all.bash on some platforms that means it flakes.
Test is kept because it can be useful standalone to verify
behavior of GOEXPERIMENT=preeemptibleloops, and there is
likely to be further development of this feature in the
future.
There's also some question as to why it is flaking, because
though technically this is permitted, it's very odd in this
simple case.
David Chase [Thu, 10 Nov 2016 21:03:47 +0000 (16:03 -0500)]
cmd/compile: insert scheduling checks on loop backedges
Loop breaking with a counter. Benchmarked (see comments),
eyeball checked for sanity on popular loops. This code
ought to handle loops in general, and properly inserts phi
functions in cases where the earlier version might not have.
Includes test, plus modifications to test/run.go to deal with
timeout and killing looping test. Tests broken by the addition
of extra code (branch frequency and live vars) for added
checks turn the check insertion off.
If GOEXPERIMENT=preemptibleloops, the compiler inserts reschedule
checks on every backedge of every reducible loop. Alternately,
specifying GO_GCFLAGS=-d=ssa/insert_resched_checks/on will
enable it for a single compilation, but because the core Go
libraries contain some loops that may run long, this is less
likely to have the desired effect.
This is intended as a tool to help in the study and diagnosis
of GC and other latency problems, now that goal STW GC latency
is on the order of 100 microseconds or less.
Updates #17831.
Updates #10958.
Change-Id: I6206c163a5b0248e3f21eb4fc65f73a179e1f639
Reviewed-on: https://go-review.googlesource.com/33910
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Joe Tsai [Mon, 9 Jan 2017 09:00:36 +0000 (01:00 -0800)]
net/http: preserve original HTTP method when possible
In Go1.7, a 301, 302, or 303 redirect on a HEAD method, would still
cause the following redirects to still use a HEAD.
In CL/29852 this behavior was changed such that those codes always
caused a redirect with the GET method. Fix this such that both
GET and HEAD will preserve the method.
shawnps [Sat, 7 Jan 2017 16:23:11 +0000 (08:23 -0800)]
all: fix misspellings
Change-Id: I429637ca91f7db4144f17621de851a548dc1ce76
Reviewed-on: https://go-review.googlesource.com/34923 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Matthew Dempsky [Fri, 6 Jan 2017 20:25:32 +0000 (12:25 -0800)]
net: disable RFC 6724 Rule 9 for IPv4 addresses
Rule 9 arguably doesn't make sense for IPv4 addresses, and so far it
has only caused problems (#13283, #18518). Disable it until we hear
from users that actually want/need it.
Russ Cox [Fri, 6 Jan 2017 05:54:24 +0000 (00:54 -0500)]
runtime: fix corruption crash/race between select and stack growth
To implement the blocking of a select, a goroutine builds a list of
offers to communicate (pseudo-g's, aka sudog), one for each case,
queues them on the corresponding channels, and waits for another
goroutine to complete one of those cases and wake it up. Obviously it
is not OK for two other goroutines to complete multiple cases and both
wake the goroutine blocked in select. To make sure that only one
branch of the select is chosen, all the sudogs contain a pointer to a
shared (single) 'done uint32', which is atomically cas'ed by any
interested goroutines. The goroutine that wins the cas race gets to
wake up the select. A complication is that 'done uint32' is stored on
the stack of the goroutine running the select, and that stack can move
during the select due to stack growth or stack shrinking.
The relevant ordering to block and unblock in select is:
1. Lock all channels.
2. Create list of sudogs and queue sudogs on all channels.
3. Switch to system stack, mark goroutine as asleep,
unlock all channels.
4. Sleep until woken.
5. Wake up on goroutine stack.
6. Lock all channels.
7. Dequeue sudogs from all channels.
8. Free list of sudogs.
9. Unlock all channels.
There are two kinds of stack moves: stack growth and stack shrinking.
Stack growth happens while the original goroutine is running.
Stack shrinking happens asynchronously, during garbage collection.
While a channel listing a sudog is locked by select in this process,
no other goroutine can attempt to complete communication on that
channel, because that other goroutine doesn't hold the lock and can't
find the sudog. If the stack moves while all the channel locks are
held or when the sudogs are not yet or no longer queued in the
channels, no problem, because no goroutine can get to the sudogs and
therefore to selectdone. We only need to worry about the stack (and
'done uint32') moving with the sudogs queued in unlocked channels.
Stack shrinking can happen any time the goroutine is stopped.
That code already acquires all the channel locks before doing the
stack move, so it avoids this problem.
Stack growth can happen essentially any time the original goroutine is
running on its own stack (not the system stack). In the first half of
the select, all the channels are locked before any sudogs are queued,
and the channels are not unlocked until the goroutine has stopped
executing on its own stack and is asleep, so that part is OK. In the
second half of the select, the goroutine wakes up on its own goroutine
stack and immediately locks all channels. But the actual call to lock
might grow the stack, before acquiring any locks. In that case, the
stack is moving with the sudogs queued in unlocked channels. Not good.
One goroutine has already won a cas on the old stack (that goroutine
woke up the selecting goroutine, moving it out of step 4), and the
fact that done = 1 now should prevent any other goroutines from
completing any other select cases. During the stack move, however,
sudog.selectdone is moved from pointing to the old done variable on
the old stack to a new memory location on the new stack. Another
goroutine might observe the moved pointer before the new memory
location has been initialized. If the new memory word happens to be
zero, that goroutine might win a cas on the new location, thinking it
can now complete the select (again). It will then complete a second
communication (reading from or writing to the goroutine stack
incorrectly) and then attempt to wake up the selecting goroutine,
which is already awake.
The scribbling over the goroutine stack unexpectedly is already bad,
but likely to go unnoticed, at least immediately. As for the second
wakeup, there are a variety of ways it might play out.
* The goroutine might not be asleep.
That will produce a runtime crash (throw) like in #17007:
runtime: gp: gp=0xc0422dcb60, goid=2299, gp->atomicstatus=8
runtime: g: g=0xa5cfe0, goid=0, g->atomicstatus=0
fatal error: bad g->status in ready
Here, atomicstatus=8 is copystack; the second, incorrect wakeup is
observing that the selecting goroutine is in state "Gcopystack"
instead of "Gwaiting".
* The goroutine might be sleeping in a send on a nil chan.
If it wakes up, it will crash with 'fatal error: unreachable'.
* The goroutine might be sleeping in a send on a non-nil chan.
If it wakes up, it will crash with 'fatal error: chansend:
spurious wakeup'.
* The goroutine might be sleeping in a receive on a nil chan.
If it wakes up, it will crash with 'fatal error: unreachable'.
* The goroutine might be sleeping in a receive on a non-nil chan.
If it wakes up, it will silently (incorrectly!) continue as if it
received a zero value from a closed channel, leaving a sudog queued on
the channel pointing at that zero vaue on the goroutine's stack; that
space will be reused as the goroutine executes, and when some other
goroutine finally completes the receive, it will do a stray write into
the goroutine's stack memory, which may cause problems. Then it will
attempt the real wakeup of the goroutine, leading recursively to any
of the cases in this list.
* The goroutine might have been running a select in a finalizer
(I hope not!) and might now be sleeping waiting for more things to
finalize. If it wakes up, as long as it goes back to sleep quickly
(before the real GC code tries to wake it), the spurious wakeup does
no harm (but the stack was still scribbled on).
* The goroutine might be sleeping in gcParkAssist.
If it wakes up, that will let the goroutine continue executing a bit
earlier than we would have liked. Eventually the GC will attempt the
real wakeup of the goroutine, leading recursively to any of the cases
in this list.
* The goroutine cannot be sleeping in bgsweep, because the background
sweepers never use select.
* The goroutine might be sleeping in netpollblock.
If it wakes up, it will crash with 'fatal error: netpollblock:
corrupted state'.
* The goroutine might be sleeping in main as another thread crashes.
If it wakes up, it will exit(0) instead of letting the other thread
crash with a non-zero exit status.
* The goroutine cannot be sleeping in forcegchelper,
because forcegchelper never uses select.
* The goroutine might be sleeping in an empty select - select {}.
If it wakes up, it will return to the next line in the program!
* The goroutine might be sleeping in a non-empty select (again).
In this case, it will wake up spuriously, with gp.param == nil (no
reason for wakeup), but that was fortuitously overloaded for handling
wakeup due to a closing channel and the way it is handled is to rerun
the select, which (accidentally) handles the spurious wakeup
correctly:
if cas == nil {
// This can happen if we were woken up by a close().
// TODO: figure that out explicitly so we don't need this loop.
goto loop
}
Before looping, it will dequeue all the sudogs on all the channels
involved, so that no other goroutine will attempt to wake it.
Since the goroutine was blocked in select before, being blocked in
select again when the spurious wakeup arrives may be quite likely.
In this case, the spurious wakeup does no harm (but the stack was
still scribbled on).
* The goroutine might be sleeping in semacquire (mutex slow path).
If it wakes up, that is taken as a signal to try for the semaphore
again, not a signal that the semaphore is now held, but the next
iteration around the loop will queue the sudog a second time, causing
a cycle in the wakeup list for the given address. If that sudog is the
only one in the list, when it is eventually dequeued, it will
(due to the precise way the code is written) leave the sudog on the
queue inactive with the sudog broken. But the sudog will also be in
the free list, and that will eventually cause confusion.
* The goroutine might be sleeping in notifyListWait, for sync.Cond.
If it wakes up, (*Cond).Wait returns. The docs say "Unlike in other
systems, Wait cannot return unless awoken by Broadcast or Signal,"
so the spurious wakeup is incorrect behavior, but most callers do not
depend on that fact. Eventually the condition will happen, attempting
the real wakeup of the goroutine and leading recursively to any of the
cases in this list.
* The goroutine might be sleeping in timeSleep aka time.Sleep.
If it wakes up, it will continue running, leaving a timer ticking.
When that time bomb goes off, it will try to ready the goroutine
again, leading to any one of the cases in this list.
* The goroutine cannot be sleeping in timerproc,
because timerproc never uses select.
* The goroutine might be sleeping in ReadTrace.
If it wakes up, it will print 'runtime: spurious wakeup of trace
reader' and return nil. All future calls to ReadTrace will print
'runtime: ReadTrace called from multiple goroutines simultaneously'.
Eventually, when trace data is available, a true wakeup will be
attempted, leading to any one of the cases in this list.
None of these fatal errors appear in any of the trybot or dashboard
logs. The 'bad g->status in ready' that happens if the goroutine is
running (the most likely scenario anyway) has happened once on the
dashboard and eight times in trybot logs. Of the eight, five were
atomicstatus=8 during net/http tests, so almost certainly this bug.
The other three were atomicstatus=2, all near code in select,
but in a draft CL by Dmitry that was rewriting select and may or may
not have had its own bugs.
This bug has existed since Go 1.4. Until then the select code was
implemented in C, 'done uint32' was a C stack variable 'uint32 done',
and C stacks never moved. I believe it has become more common recently
because of Brad's work to run more and more tests in net/http in
parallel, which lengthens race windows.
The fix is to run step 6 on the system stack,
avoiding possibility of stack growth.
Fixes #17007 and possibly other mysterious failures.
Austin Clements [Fri, 23 Dec 2016 00:30:23 +0000 (17:30 -0700)]
runtime: update big mgc.go comment
The comment describing the overall GC algorithm at the top of mgc.go
has gotten woefully out-of-date (and was possibly never
correct/complete). Update it to reflect the current workings of the
GC and the set of phases that we now divide it into.
Change-Id: I02143c0ebefe9d4cd7753349dab8045f0973bf95
Reviewed-on: https://go-review.googlesource.com/34711 Reviewed-by: Rick Hudson <rlh@golang.org>
Russ Cox [Fri, 6 Jan 2017 06:17:24 +0000 (01:17 -0500)]
net/http: better failure in TestTransportPersistConnLeak
If one of the c.Get(ts.URL) results in an error, the child goroutine
calls t.Errorf, but the test goroutine gets stuck waiting for <-gotReqCh,
so the test hangs and the program is eventually killed (after 10 minutes!).
Whatever might have been printed to t.Errorf is never seen.
Adjust test so that the test fails cleanly in this case.
Still trying to debug why c.Get might fail.
It seems to have something to do with occasional connection
failures on macOS Sierra.
Austin Clements [Fri, 6 Jan 2017 14:44:41 +0000 (09:44 -0500)]
runtime: use 4K as the boundary of legal pointers
Currently, the check for legal pointers in stack copying uses
_PageSize (8K) as the minimum legal pointer. By default, Linux won't
let you map under 64K, but
1) it's less clear what other OSes allow or will allow in the future;
2) while mapping the first page is a terrible idea, mapping anywhere
above that is arguably more justifiable;
3) the compiler only assumes the first physical page (4K) is never
mapped.
Make the runtime consistent with the compiler and more robust by
changing the bad pointer check to use 4K as the minimum legal pointer.
This came out of discussions on CLs 34663 and 34719.
Lion Yang [Wed, 4 Jan 2017 22:25:59 +0000 (06:25 +0800)]
x/crypto/chacha20poly1305: fix detection of BMI on AMD64
This change uses runtime.support_bmi2 as an additional condition
to examine the usability of AVX2 version algorithm, fixes
the crash on the platfrom which supports AVX2 but not support BMI2.
Fixes #18512
Change-Id: I408c0844ae2eb242dacf70cb9e8cec1b8f3bd941
Reviewed-on: https://go-review.googlesource.com/34851 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Lion Yang [Wed, 4 Jan 2017 21:13:53 +0000 (05:13 +0800)]
crypto: detect BMI usability on AMD64 for sha1 and sha256
The existing implementations on AMD64 only detects AVX2 usability,
when they also contains BMI (bit-manipulation instructions).
These instructions crash the running program as 'unknown instructions'
on the architecture, e.g. i3-4000M, which supports AVX2 but not
support BMI.
This change added the detections for BMI1 and BMI2 to AMD64 runtime with
two flags as the result, `support_bmi1` and `support_bmi2`,
in runtime/runtime2.go. It also completed the condition to run AVX2 version
in packages crypto/sha1 and crypto/sha256.
Fixes #18512
Change-Id: I917bf0de365237740999de3e049d2e8f2a4385ad
Reviewed-on: https://go-review.googlesource.com/34850 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Russ Cox [Wed, 4 Jan 2017 19:12:56 +0000 (14:12 -0500)]
.gitignore: fix attempt at rooted paths
When I wrote the lines
bin/
pkg/
I was trying to match just the top-level bin and pkg directories, and I put the
final slash in because 'git help gitignore' says:
o If the pattern does not contain a slash /, Git treats it as a shell
glob pattern and checks for a match against the pathname relative
to the location of the .gitignore file (relative to the toplevel of
the work tree if not from a .gitignore file).
o Otherwise, Git treats the pattern as a shell glob suitable for
consumption by fnmatch(3) with the FNM_PATHNAME flag: wildcards in
the pattern will not match a / in the pathname. For example,
"Documentation/*.html" matches "Documentation/git.html" but not
"Documentation/ppc/ppc.html" or
"tools/perf/Documentation/perf.html".
Putting a trailing slash was my way of opting in to the "rooted path" semantics
without looking different from the surrounding rooted paths like "src/go/build/zcgo.go".
But HA HA GIT FOOLED YOU! above those two bullets the docs say:
o If the pattern ends with a slash, it is removed for the purpose of
the following description, ...
Change all the patterns to use a leading slash for "rooted" behavior.
This bit me earlier today because I had a perfectly reasonable source
code directory go/src/cmd/go/testdata/src/empty/pkg that was
not added by 'git add empty'.
Change-Id: I6f8685b3c5be22029c33de9ccd735487089a1c03
Reviewed-on: https://go-review.googlesource.com/34832 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Brad Fitzpatrick [Thu, 5 Jan 2017 00:52:04 +0000 (00:52 +0000)]
lib/time: update tzdata to 2016j
Fixes #18500
Change-Id: I4dddd1b99aecf86b9431b0c14f452152dff9b95a
Reviewed-on: https://go-review.googlesource.com/34816 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Brad Fitzpatrick [Wed, 4 Jan 2017 21:23:00 +0000 (21:23 +0000)]
net/http/httputil: make DumpRequest and DumpRequestOut recognize http.NoBody
Fixes #18506
Change-Id: I6b0b107296311178938609e878e1ef47a30a463f
Reviewed-on: https://go-review.googlesource.com/34814 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Brad Fitzpatrick [Wed, 4 Jan 2017 21:03:24 +0000 (21:03 +0000)]
net/http: make Server cancel its ReadTimeout between requests
Fixes #18447
Change-Id: I5d60c3632a5ce625d3bac9d85533ce689e301707
Reviewed-on: https://go-review.googlesource.com/34813 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Marineau [Tue, 3 Jan 2017 08:15:05 +0000 (00:15 -0800)]
runtime: check sched_getaffinity return value
Android on ChromeOS uses a restrictive seccomp filter that blocks
sched_getaffinity, leading this code to index a slice by -errno.
Change-Id: Iec09a4f79dfbc17884e24f39bcfdad305de75b37
Reviewed-on: https://go-review.googlesource.com/34794 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
This will make diagnosis easier when there are multiple function calls per line.
Change-Id: I9881713c5671b847b84a0df0115f57e7cba17d72
Reviewed-on: https://go-review.googlesource.com/34730 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Vladimir Stefanovic [Tue, 27 Dec 2016 15:26:41 +0000 (16:26 +0100)]
runtime: fix SP alignment in mips{,le} sigfwd
Fixes misc/cgo/testsigfwd, enabled for mips{,le} with the next commit
(https://golang.org/cl/34646).
Change-Id: I2bec894b0492fd4d84dd73a4faa19eafca760107
Reviewed-on: https://go-review.googlesource.com/34645 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Daniel Theophanes [Mon, 26 Dec 2016 19:33:46 +0000 (11:33 -0800)]
database/sql: prevent Tx.rollback from racing Tx.close
Previously Tx.done was being set in close, but in a Tx
rollback and Commit are the real closing methods,
and Tx.close is just a helper common to both. Prior to this
change a multiple rollback statements could be called, one
would enter close and begin closing it while the other was
still in rollback breaking it. Fix that by setting done
in rollback and Commit, not in Tx.close.
Austin Clements [Fri, 23 Dec 2016 01:17:32 +0000 (18:17 -0700)]
runtime: document MemStats.BySize fields
Change-Id: Iae8cdcd84e9b5f5d7c698abc6da3fc2af0ef839a
Reviewed-on: https://go-review.googlesource.com/34710 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Emmanuel Odeke [Fri, 23 Dec 2016 11:23:10 +0000 (03:23 -0800)]
test: lock in test for _ assignment evaluation/zerodivide panic
Fixes #5790.
Fixes #18421.
* Lock in _ = x1/x2 divide by zero runtime panics since
it is actually evaluated and not discarded as in previous
versions before Go1.8.
* Update a test that was skipping over zerodivide tests
that expected runtime panics, enabling us to check for
the expected panics.
Change-Id: I0af0a6ecc19345fa9763ab2e35b275fb2d9d0194
Reviewed-on: https://go-review.googlesource.com/34712 Reviewed-by: Keith Randall <khr@golang.org>
Raul Silvera [Wed, 21 Dec 2016 18:16:15 +0000 (10:16 -0800)]
cmd/pprof: Re-enable weblist and disasm
Previous changes started using the full filename for object files
on graph nodes, instead of just the file basename. The basename
was still being used when selecting mappings to disassemble for
weblist and disasm commands, causing a mismatch.
This fixes #18385. It was already fixed on the upstream pprof.
Brad Fitzpatrick [Thu, 1 Dec 2016 21:45:59 +0000 (21:45 +0000)]
net/http: restore Transport's Request.Body byte sniff in limited cases
In Go 1.8, we'd removed the Transport's Request.Body
one-byte-Read-sniffing to disambiguate between non-nil Request.Body
with a ContentLength of 0 or -1. Previously, we tried to see whether a
ContentLength of 0 meant actually zero, or just an unset by reading a
single byte of the Request.Body and then stitching any read byte back
together with the original Request.Body.
That historically has caused many problems due to either data races,
blocking forever (#17480), or losing bytes (#17071). Thus, we removed
it in both HTTP/1 and HTTP/2 in Go 1.8. Unfortunately, during the Go
1.8 beta, we've found that a few people have gotten bitten by the
behavior change on requests with methods typically not containing
request bodies (e.g. GET, HEAD, DELETE). The most popular example is
the aws-go SDK, which always set http.Request.Body to a non-nil value,
even on such request methods. That was causing Go 1.8 to send such
requests with Transfer-Encoding chunked bodies, with zero bytes,
confusing popular servers (including but limited to AWS).
This CL partially reverts the no-byte-sniffing behavior and restores
it only for GET/HEAD/DELETE/etc requests, and only when there's no
Transfer-Encoding set, and the Content-Length is 0 or -1.
Updates #18257 (aws-go) bug
And also private bug reports about non-AWS issues.
Updates #18407 also, but haven't yet audited things enough to declare
it fixed.
Change-Id: Ie5284d3e067c181839b31faf637eee56e5738a6a
Reviewed-on: https://go-review.googlesource.com/34668
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Elias Naur [Thu, 22 Dec 2016 17:31:17 +0000 (18:31 +0100)]
runtime: skip floating point hardware check on Android
CL 33652 removed the fake auxv for Android, and replaced it with
a /proc/self/auxv fallback. When /proc/self/auxv is unreadable,
however, hardware capabilities detection won't work and the runtime
will mistakenly think that floating point hardware is unavailable.
Fix this by always assuming floating point hardware on Android.
Manually tested on a Nexus 5 running Android 6.0.1. I suspect the
android/arm builder has a readable /proc/self/auxv and therefore
does not trigger the failure mode.
Change-Id: I95c3873803f9e17333c6cb8b9ff2016723104085
Reviewed-on: https://go-review.googlesource.com/34641 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Austin Clements [Tue, 20 Dec 2016 20:37:56 +0000 (15:37 -0500)]
runtime: avoid CreateThread panic when exiting process
On Windows, CreateThread occasionally fails with ERROR_ACCESS_DENIED.
We're not sure why this is, but the Wine source code suggests that
this can happen when there's a concurrent CreateThread and ExitProcess
in the same process.
Fix this by setting a flag right before calling ExitProcess and
halting if CreateThread fails and this flag is set.
Updates #18253 (might fix it, but we're not sure this is the issue and
can't reproduce it on demand).
Change-Id: I1945b989e73a16cf28a35bf2613ffab07577ed4e
Reviewed-on: https://go-review.googlesource.com/34616
TryBot-Result: Gobot Gobot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Kirill Smelkov [Thu, 1 Dec 2016 18:40:57 +0000 (21:40 +0300)]
cmd/compile/internal/ssa: add tests for BSWAP on stores on AMD64
Commit 10f75748 (CL 32222) taught AMD64 backend to rewrite series of
byte loads or stores with corresponding shifts into a single long or
quad load or store + appropriate BSWAP. However it did not added test
for stores - only loads were tested.
Fix it.
NOTE Tests for indexed stores are not added because 10f75748 did not add
support for indexed stores - only indexed loads were handled then.