runtime: don't free large spans until heapBitsSweepSpan returns
This fixes a race between 1) sweeping and freeing an unmarked large
span and 2) reusing that span and allocating from it. This race arises
because mSpan_Sweep returns spans for large objects to the heap
*before* heapBitsSweepSpan clears the mark bit on the object in the
span.
Specifically, the following sequence of events can lead to an
incorrectly zeroed bitmap byte, which causes the garbage collector to
not trace any pointers in that object (the pointer bits for the first
four words are cleared, and the scan bits are also cleared, so it
looks like a no-scan object).
1) P0 calls mSpan_Sweep on a large span S0 with an unmarked object on it.
2) mSpan_Sweep calls heapBitsSweepSpan, which invokes the callback for
the one (unmarked) object on the span.
3) The callback calls mHeap_Free, which makes span S0 available for
allocation, but this is too early.
4) P1 grabs this S0 from the heap to use for allocation.
5) P1 allocates an object on this span and writes that object's type
bits to the bitmap.
6) P0 returns from the callback to heapBitsSweepSpan.
heapBitsSweepSpan clears the byte containing the mark, even though
this span is now owned by P1 and this byte contains important
bitmap information.
This fixes this problem by simply delaying the mHeap_Free until after
the heapBitsSweepSpan. I think the overall logic of mSpan_Sweep could
be simplified now, but this seems like the minimal change.
Fixes #11617.
Change-Id: I6b1382c7e7cc35f81984467c0772fe9848b7522a
Reviewed-on: https://go-review.googlesource.com/12320
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Rob Pike <r@golang.org>
Change-Id: Ia7cc1d52211b32a2eb2b3888d621b28d6932aca9
Reviewed-on: https://go-review.googlesource.com/12290 Reviewed-by: Ian Lance Taylor <iant@golang.org>
David Crawshaw [Thu, 16 Jul 2015 17:14:56 +0000 (13:14 -0400)]
crypto/x509: iOS build tag
The iOS simulator compiles with GOOS=darwin GOARCH=386, and x509
sets the inappropriate flag -mmacosx-version-min=10.6. Condition
its compilation on the absence of an "ios" build tag.
net/http: don't reuse conns after incomplete 100-continue requests
If we receive an HTTP request with "Expect: 100-continue" and the
Handler never read to EOF, the conn is in an unknown state.
Don't reuse that connection.
Andrew Gerrand [Wed, 15 Jul 2015 08:36:49 +0000 (18:36 +1000)]
cmd/go: fix parsing of Git SCP-like remotes
Now that we care about the protocol of Git remotes (for the -insecure
flag), we need to recognize and parse the SCP-like remote format.
Fixes golang/go#11457
Change-Id: Ia26132274fafb1cbfefe2475f7ac5f17ccd6da40
Reviewed-on: https://go-review.googlesource.com/12226 Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtime: fix saved PC/SP after safe-point function in syscall
Running a safe-point function on syscall entry uses systemstack() and
hence clobbers g.sched.pc and g.sched.sp. Fix this by re-saving them
after the systemstack, just like in the other uses of systemstack in
reentersyscall.
runtime: run safe-point function before entering _Psyscall
Currently, we run a P's safe-point function immediately after entering
_Psyscall state. This is unsafe, since as soon as we put the P in
_Psyscall, we no longer control the P and another M may claim it.
We'll still run the safe-point function only once (because doing so
races on an atomic), but the P may no longer be at a safe-point when
we do so.
In particular, this means that the use of forEachP to dispose all P's
gcw caches is unsafe. A P may enter a syscall, run the safe-point
function, and dispose the P's gcw cache concurrently with another M
claiming the P and attempting to use its gcw cache. If this happens,
we may empty the gcw's workbuf after putting it on
work.{full,partial}, or add pointers to it after putting it in
work.empty. This will cause an assertion failure when we later pop the
workbuf from the list and its object count is inconsistent with the
list we got it from.
Fix this by running the safe-point function just before putting the P
in _Psyscall.
Related to #11640. This probably fixes this issue, but while I'm able
to show that we can enter a bad safe-point state as a result of this,
I can't reproduce that specific failure.
Ian Lance Taylor [Wed, 15 Jul 2015 05:17:41 +0000 (22:17 -0700)]
cmd/compile: recognize embedded field in inlined function
There was already special code to recognize "?" in hidden_structdcl,
which is used for inlined types and variables. This recognizes "?" in
structdcl as well, a case that arises when a struct type appears
within an inlined function body.
Fixes #10219.
Change-Id: Ic5257ae54f817e0d4a189c2294dcd633c9f2101a
Reviewed-on: https://go-review.googlesource.com/12241
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
cmd/compile: fix PtrTo(t) for unnamed t with embedded fields
Fixes #8427.
Change-Id: I826a3bc4519845ad30d6dbaf058fe7ed7bee8db0
Reviewed-on: https://go-review.googlesource.com/12233 Reviewed-by: Ian Lance Taylor <iant@golang.org>
- Make Log2 exact for powers of two.
- Fix error tolerance function to make tolerance
a function of the correct (expected) value.
Fixes #9066.
Change-Id: I0320a93ce4130deed1c7b7685627d51acb7bc56d
Reviewed-on: https://go-review.googlesource.com/12230 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: I3b85053262cac3c30358f8e03a5aca65dbc67623
Reviewed-on: https://go-review.googlesource.com/12231 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Originally 'go test -h' printed the output of 'go help test'.
Then issue #6576 was filed, because that output didn't list (for example) -bench.
CL 14502065 changed 'go test -h' to print the output of 'go help testflag'.
Then issue #9209 was filed, because that output didn't list (for example) -c.
To print all the relevant flags, parts of both 'go help test' and 'go help testflag'
are needed. Refactor the help messages to make those parts available
and print them.
Fixes #9209.
Change-Id: Ie8205b8fb37d00c10d25b3fc98f14286ec46c4e3
Reviewed-on: https://go-review.googlesource.com/12173 Reviewed-by: Rob Pike <r@golang.org>
Nigel Tao [Wed, 15 Jul 2015 04:55:02 +0000 (14:55 +1000)]
image/color: tweak the YCbCr to RGBA conversion formula.
Before, calling the RGBA method of YCbCr color would return red values
in the range [0x0080, 0xff80]. After, the range is [0x0000, 0xffff] and
is consistent with what Gray colors' RGBA method returns. In particular,
pure black, pure white and every Gray color in between are now exactly
representable as a YCbCr color.
This fixes a regression from Go 1.4 (where YCbCr{0x00, 0x80, 0x80} was
no longer equivalent to pure black), introduced by golang.org/cl/8073 in
the Go 1.5 development cycle. In Go 1.4, the +0x80 rounding was not
noticable when Cb == 0x80 && Cr == 0x80, because the YCbCr to RGBA
conversion truncated to 8 bits before multiplying by 0x101, so the
output range was [0x0000, 0xffff].
The TestYCbCrRoundtrip fuzzy-match tolerance grows from 1 to 2 because
the YCbCr to RGB conversion now maps to an ever-so-slightly larger
range, along with the usual imprecision of accumulating rounding errors.
Also s/int/int32/ in ycbcr.go. The conversion shouldn't overflow either
way, as int is always at least 32 bits, but it does make it clearer that
the computation doesn't depend on sizeof(int).
Fixes #11691
Change-Id: I538ca0adf7e040fa96c5bc8b3aef4454535126b9
Reviewed-on: https://go-review.googlesource.com/12220 Reviewed-by: Rob Pike <r@golang.org>
Rob Pike [Wed, 15 Jul 2015 05:02:38 +0000 (15:02 +1000)]
time: make it clearer how to format a fractional second
Fixes #10963.
Change-Id: I8d769b4d25b306f2df41f882ec01d97bbd63171d
Reviewed-on: https://go-review.googlesource.com/12221 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
os/exec: document that Cmd.Wait waits for stdin I/O
Fixes #10338.
Change-Id: Ib86cb9a6c694b1e442a9957153c7ca38a7d11c3e
Reviewed-on: https://go-review.googlesource.com/12232 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/go: fix go get -u handling of changing dependencies
Fixes #9224.
Change-Id: Ie0f4f14407099e4fa7ebe361a95b6492012928a2
Reviewed-on: https://go-review.googlesource.com/12192 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
There appears to be no portable way to fix the earlier versions.
We already run git with stdin closed, but on Unix git calls getpass,
which opens /dev/tty itself. We could do package syscall-specific
things to get /dev/tty invalidated during the exec, but I'd really
rather not. And on Windows, Git opens "CONIN$" and "CONOUT$"
itself, and I have no idea how to invalidate those.
Fix the problem for newish Git versions and wait for people to update.
Best we can do.
Fixes #9341.
Change-Id: I576579b106764029853e0f74d411e19108deecf5
Reviewed-on: https://go-review.googlesource.com/12175 Reviewed-by: Rob Pike <r@golang.org>
cmd/go: fix TestVendorRun when $GOROOT is inside a symlinked path
Fixes #11305.
Change-Id: Icaa3a009aa4ab214c9aaf74f52c3e622fa266a9d
Reviewed-on: https://go-review.googlesource.com/12194 Reviewed-by: David Crawshaw <crawshaw@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/internal/obj: fix pc/sp information for prologue
When the prologue call to morestack was moved down to the
bottom of the function, the pc/sp tables were not updated.
If a traceback through a call to morestack is needed, it would
get confused at and stop at morestack.
Confirmed the fix by adding //go:systemstack (which calls
morestackc, but same issue) where it did not belong
and inspecting the crash.
Change-Id: Id0294bb9dba51ef1a49154637228fb57f1086a94
Reviewed-on: https://go-review.googlesource.com/12144 Reviewed-by: Rob Pike <r@golang.org>
Didier Spezia [Sat, 27 Jun 2015 13:07:22 +0000 (13:07 +0000)]
encoding/xml: improve marshaller sanity checks of directives
When building a directive, the current sanity check prevents
a '>' to be used, which makes a DOCTYPE directive with an
internal subset be rejected. It is accepted by the parser
though, so what can be parsed cannot be encoded.
Improved the corresponding sanity check to mirror the behavior
of the parser (in the way it handles angle brackets, quotes,
and comments).
Larz Conwell [Sun, 17 May 2015 03:01:39 +0000 (23:01 -0400)]
encoding/json: Only allow string option for valid types
The "string" option only applies for strings, floats, integers, and
booleans as per the documentation. So when decoding ignore the "string"
option if the value is not of one of the types mentioned. This matches
the Marshal step which also ignores the "string" option for invalid
types.
Nigel Tao [Tue, 14 Jul 2015 05:46:18 +0000 (15:46 +1000)]
image/jpeg: don't unread a byte if we've already taken bits from it.
This rolls back most of golang.org/cl/8841, aka 2f98bac310, and makes a
different fix. It keeps the TestTruncatedSOSDataDoesntPanic test
introduced by that other CL, which obviously still passes after this CL.
Fixes #11650, a regression (introduced by cl/8841) from Go 1.4.
The original cl/8841 changed the image/jpeg not to panic on an input
given in #10387. We still do not panic on that input, after this CL.
I have a corpus of over 160,000 JPEG images, a sample of a web crawl.
The image/jpeg code ran happily over that whole corpus both before and
after this CL, although that corpus clearly didn't catch the regression
in the first place.
This code was otherwise tested manually. I don't think that it's trivial
to synthesize a JPEG input that happens to run out of Huffman data at
just the right place. The test image attached to #11650 obviously has
that property, but I don't think we can simply add that test image to
the repository: it's 227KiB, and I don't know its copyright status.
I also looked back over the issue tracker for problematic JPEGs that
people have filed. The Go code, after this CL, is still happy on these
files in my directory:
issue2362a.jpeg
issue3916.jpeg
issue3976.jpeg
issue4084.jpeg
issue4259.jpeg
issue4291.jpeg
issue4337.jpeg
issue4500.jpeg
issue4705.jpeg
issue4975.jpeg
issue5112.jpeg
issue6767.jpeg
issue9888.jpeg
issue10133.jpeg
issue10357.jpeg
issue10447.jpeg
issue11648.jpeg
issue11650.jpeg
There were other images attached in the issue tracker that aren't
actually valid JPEGs. They failed both before and after this CL:
broken-issue2362b.jpeg
broken-issue6450.jpeg
broken-issue8693.jpeg
broken-issue10154.jpeg
broken-issue10387.jpeg
broken-issue10388.jpeg
broken-issue10389.jpeg
broken-issue10413.jpeg
In summary, this CL fixes #11650 and, after some automated and manual
testing, I don't think introduces new regressions.
Change-Id: I30b67036e9b087f3051d57dac7ea05fb4fa36f66
Reviewed-on: https://go-review.googlesource.com/12163 Reviewed-by: Rob Pike <r@golang.org>
os/exec: ignore pipe write errors when command completes successfully
Fixes #9173
Change-Id: I83530533db84b07cb88dbf6ec690be48a06a9d7d
Reviewed-on: https://go-review.googlesource.com/12152 Reviewed-by: Ian Lance Taylor <iant@golang.org>
These memstats are currently being computed by gcMark, which was
appropriate in Go 1.4, but gcMark is now just one part of a bigger
picture. In particular, it can't account for the sweep termination
pause time, it can't account for all of the mark termination pause
time, and the reported "pause end" and "last GC" times will be
slightly earlier than they really are.
Lift computing of these statistics into func gc, which has the
appropriate visibility into the process to compute them correctly.
Fixes one of the issues in #10323. This does not add new statistics
appropriate to the concurrent collector; it simply fixes existing
statistics that are being misreported.
Austin Clements [Tue, 30 Jun 2015 22:20:13 +0000 (18:20 -0400)]
runtime: report MemStats.PauseEnd in UNIX time
Currently we report MemStats.PauseEnd in nanoseconds, but with no
particular 0 time. On Linux, the 0 time is when the host started. On
Darwin, it's the UNIX epoch. This is also inconsistent with the other
absolute time in MemStats, LastGC, which is always reported in
nanoseconds since 1970.
Fix PauseEnd so it's always reported in nanoseconds since 1970, like
LastGC.
Mikio Hara [Thu, 11 Jun 2015 03:46:01 +0000 (12:46 +0900)]
net: clean up builtin DNS stub resolver, fix tests
This change does clean up as preparation for fixing #11081.
- renames cfg to resolvConf for clarification
- adds a new type resolverConfig and its methods: init, update,
tryAcquireSema, releaseSema for mutual exclusion of resolv.conf data
- deflakes, simplifies tests for resolv.conf data; previously the tests
sometimes left some garbage in the data
Change-Id: I8d6b04cb509e62e27d6935b91ffe35fdaea4ebcd
Reviewed-on: https://go-review.googlesource.com/12028 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
Alex Brainman [Sat, 11 Jul 2015 07:19:39 +0000 (17:19 +1000)]
cmd/dist: disable misc/scgo/testsovar on netbsd
Update #11654
Change-Id: Ia199b8dd349542ad8b92b463dd2f3734dd7e66a4
Reviewed-on: https://go-review.googlesource.com/12060
Run-TryBot: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
The one in misc/makerelease/makerelease.go is particularly bad and
probably warrants rotating our keys.
I didn't update old weekly notes, and reverted some changes involving
test code for now, since we're late in the Go 1.5 freeze. Otherwise,
the rest are all auto-generated changes, and all manually reviewed.
Change-Id: Ia2753576ab5d64826a167d259f48a2f50508792d
Reviewed-on: https://go-review.googlesource.com/12048 Reviewed-by: Rob Pike <r@golang.org>
Elias Naur [Sat, 11 Jul 2015 10:59:00 +0000 (12:59 +0200)]
runtime: abort on fatal errors and panics in c-shared and c-archive modes
The default behaviour for fatal errors and runtime panics is to dump
the goroutine stack traces and exit with code 2. However, when the process is
owned by foreign code, it is suprising and inappropriate to suddenly exit
the whole process, even on fatal errors. Instead, re-use the crash behaviour
from GOTRACEBACK=crash and abort.
The motivating use case is issue #11382, where an Android crash reporter
is confused by an exiting process, but I believe the aborting behaviour
is appropriate for all cases where Go does not own the process.
The change is simple and contained and will enable reliable crash reporting
for Android apps in Go 1.5, but I'll leave it to others to judge whether it
is too late for Go 1.5.
Fixes #11382
Change-Id: I477328e1092f483591c99da1fbb8bc4411911785
Reviewed-on: https://go-review.googlesource.com/12032 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>