Russ Cox [Thu, 30 Oct 2014 00:37:44 +0000 (20:37 -0400)]
runtime: change top-most return PC from goexit to goexit+PCQuantum
If you get a stack of PCs from Callers, it would be expected
that every PC is immediately after a call instruction, so to find
the line of the call, you look up the line for PC-1.
CL 163550043 now explicitly documents that.
The most common exception to this is the top-most return PC
on the stack, which is the entry address of the runtime.goexit
function. Subtracting 1 from that PC will end up in a different
function entirely.
To remove this special case, make the top-most return PC
goexit+PCQuantum and then implement goexit in assembly
so that the first instruction can be skipped.
Russ Cox [Wed, 29 Oct 2014 22:07:24 +0000 (18:07 -0400)]
cmd/objdump: use cmd/internal/objfile
This removes a bunch of ugly duplicate code.
The end goal is to factor the disassembly code
into cmd/internal/objfile too, so that pprof can use it,
but one step at a time.
Russ Cox [Wed, 29 Oct 2014 19:14:24 +0000 (15:14 -0400)]
runtime: fix line number in first stack frame in printed stack trace
Originally traceback was only used for printing the stack
when an unexpected signal came in. In that case, the
initial PC is taken from the signal and should be used
unaltered. For the callers, the PC is the return address,
which might be on the line after the call; we subtract 1
to get to the CALL instruction.
Traceback is now used for a variety of things, and for
almost all of those the initial PC is a return address,
whether from getcallerpc, or gp->sched.pc, or gp->syscallpc.
In those cases, we need to subtract 1 from this initial PC,
but the traceback code had a hard rule "never subtract 1
from the initial PC", left over from the signal handling days.
Change gentraceback to take a flag that specifies whether
we are tracing a trap.
Change traceback to default to "starting with a return PC",
which is the overwhelmingly common case.
Add tracebacktrap, like traceback but starting with a trap PC.
Use tracebacktrap in signal handlers.
Fixes #7690.
LGTM=iant, r
R=r, iant
CC=golang-codereviews
https://golang.org/cl/167810044
David du Colombier [Tue, 28 Oct 2014 21:44:59 +0000 (22:44 +0100)]
os: fix write on Plan 9
In CL 160670043 the write function was changed
so a zero-length write is now allowed. This leads
the ExampleWriter_Init test to fail.
The reason is that Plan 9 preserves message
boundaries, while the os library expects systems
that don't preserve them. We have to ignore
zero-length writes so they will never turn into EOF.
This issue was previously discussed in CL 7406046.
Russ Cox [Tue, 28 Oct 2014 15:35:00 +0000 (11:35 -0400)]
syscall: fix ParseRoutingSockaddr with unexpected submessages
No easy way to test (would have to actually trigger some routing
events from kernel) but the code is clearly wrong as written.
If the header says there is a submessage, we need to at least
skip over its bytes, not just continue to the next iteration.
Fixes #8203.
LGTM=r
R=r
CC=golang-codereviews, mikioh.mikioh, p
https://golang.org/cl/164140044
Russ Cox [Tue, 28 Oct 2014 15:14:25 +0000 (11:14 -0400)]
cmd/go: add get -f flag
get -u now checks that remote repo paths match the
ones predicted by the import paths: if you are get -u'ing
rsc.io/pdf, it has to be checked out from the right location.
This is important in case the rsc.io/pdf redirect changes.
In some cases, people have good reasons to use
non-standard remote repos. Add -f flag to allow that.
The f can stand for force or fork, as you see fit.
Mikio Hara [Tue, 28 Oct 2014 07:20:49 +0000 (16:20 +0900)]
net: add test for lookupIPDeadline
Just to confirm the fix, by typing the follwing:
go test -run=TestLookupIPDeadline -dnsflood or
go test -run=TestLookupIPDeadline -dnsflood -tags netgo
Rob Pike [Tue, 28 Oct 2014 00:08:50 +0000 (17:08 -0700)]
doc/go_mem.html: don't be clever
Add a short introductory section saying what most Go
programmers really need to know, which is that you
shouldn't have to read this document to understand
the behavior of your program.
Robert Griesemer [Mon, 27 Oct 2014 23:31:15 +0000 (16:31 -0700)]
spec: permit parentheses around builtin function names
Not a language change.
This is simply documenting the status quo which permits
builtin function names to be parenthesized in calls; e.g.,
both
len(s)
and
(((len)))(s)
are accepted by all compilers and go/types.
Changed the grammar by merging the details of BuiltinCall
with ordinary Calls. Also renamed the Call production to
Arguments which more clearly identifies that part of the
grammar and also matches better with its counterpart on
the declaration side (Parameters).
The fact that the first argument can be a type (for builtins)
or cannot be a type (for regular function calls) is expressed
in the prose, no need to make the grammar more complicated.
Fixes #9001.
LGTM=iant, r, rsc
R=r, rsc, iant, ken, dave
CC=golang-codereviews
https://golang.org/cl/160570043
Russ Cox [Mon, 27 Oct 2014 22:59:02 +0000 (18:59 -0400)]
test: make maplinear more robust
The test just doubled a certain number of times
and then gave up. On a mostly fast but occasionally
slow machine this may never make the test run
long enough to see the linear growth.
Change test to keep doubling until the first round
takes at least a full second, to reduce the effect of
occasional scheduling or other jitter.
The failure we saw had a time for the first round
of around 100ms.
Note that this test still passes once it sees a linear
effect, even with a very small total time.
The timeout here only applies to how long the execution
must be to support a reported failure.
Austin Clements [Mon, 27 Oct 2014 21:12:48 +0000 (17:12 -0400)]
runtime: fix endianness assumption when decoding ftab
The ftab ends with a half functab record consisting only of
the 'entry' field followed by a uint32 giving the offset of
the next table. Previously, symtabinit assumed it could read
this uint32 as a uintptr. Since this is unsafe on big endian,
explicitly read the offset as a uint32.
Ian Lance Taylor [Mon, 27 Oct 2014 15:46:18 +0000 (08:46 -0700)]
net: if a DNS lookup times out, forget that it is in flight
Before this CL, if the system resolver does a very slow DNS
lookup for a particular host, all subsequent requests for that
host will hang waiting for that lookup to complete. That is
more or less expected when Dial is called with no deadline.
When Dial has a deadline, though, we can accumulate a large
number of goroutines waiting for that slow DNS lookup. Try to
avoid this problem by restarting the DNS lookup when it is
redone after a deadline is passed.
This CL also avoids creating an extra goroutine purely to
handle the deadline.
No test because we would have to simulate a slow DNS lookup
followed by a fast DNS lookup.
Emil Hessman [Mon, 27 Oct 2014 01:43:14 +0000 (12:43 +1100)]
misc/makerelease/windows: fix 404 help URL in installer
ARPHELPLINK yields 404; update the URL.
While here, also prefix the ARPREADME and ARPURLINFOABOUT URL's with the HTTP scheme to make 'em clickable links in the Add or Remove Programs listing.
Rob Pike [Fri, 24 Oct 2014 16:37:25 +0000 (09:37 -0700)]
unsafe: document that unsafe programs are not protected
The compatibility guideline needs to be clear about this even
though it means adding a clause that was not there from the
beginning. It has always been understood, so this isn't really
a change in policy, just in its expression.
Russ Cox [Fri, 24 Oct 2014 14:27:39 +0000 (10:27 -0400)]
cmd/gc: synthesize zeroed value for non-assignment context
CL 157910047 introduced code to turn a node representing
a zeroed composite literal into N, the nil Node* pointer
(which represents any zero, not the Go literal nil).
That's great for assignments like x = T{}, but it doesn't work
when T{} is used in a value context like T{}.v or x == T{}.
Fix those.
Should have no effect on performance; confirmed.
The deltas below are noise (compare ns/op):
Russ Cox [Fri, 24 Oct 2014 03:44:47 +0000 (23:44 -0400)]
encoding/csv: for Postgres, unquote empty strings, quote \.
In theory both of these lines encode the same three fields:
a,,c
a,"",c
However, Postgres defines that when importing CSV, the unquoted
version is treated as NULL (missing), while the quoted version is
treated as a string value (empty string). If the middle field is supposed to
be an integer value, the first line can be imported (NULL is okay), but
the second line cannot (empty string is not).
Postgres's import command (COPY FROM) has an option to force
the unquoted empty to be interpreted as a string but it does not
have an option to force the quoted empty to be interpreted as a NULL.
From http://www.postgresql.org/docs/9.0/static/sql-copy.html:
The CSV format has no standard way to distinguish a NULL
value from an empty string. PostgreSQL's COPY handles this
by quoting. A NULL is output as the NULL parameter string
and is not quoted, while a non-NULL value matching the NULL
parameter string is quoted. For example, with the default
settings, a NULL is written as an unquoted empty string,
while an empty string data value is written with double
quotes (""). Reading values follows similar rules. You can
use FORCE_NOT_NULL to prevent NULL input comparisons for
specific columns.
Therefore printing the unquoted empty is more flexible for
imports into Postgres than printing the quoted empty.
In addition to making the output more useful with Postgres, not
quoting empty strings makes the output smaller and easier to read.
It also matches the behavior of Microsoft Excel and Google Drive.
Since we are here and making concessions for Postgres, handle this
case too (again quoting the Postgres docs):
Because backslash is not a special character in the CSV
format, \., the end-of-data marker, could also appear as a
data value. To avoid any misinterpretation, a \. data value
appearing as a lone entry on a line is automatically quoted
on output, and on input, if quoted, is not interpreted as
the end-of-data marker. If you are loading a file created by
another application that has a single unquoted column and
might have a value of \., you might need to quote that value
in the input file.
Dmitriy Vyukov [Wed, 22 Oct 2014 16:23:49 +0000 (20:23 +0400)]
sync: release Pool memory during second and later GCs
Pool memory was only being released during the first GC after the first Put.
Put assumes that p.local != nil means p is on the allPools list.
poolCleanup (called during each GC) removed each pool from allPools
but did not clear p.local, so each pool was cleared by exactly one GC
and then never cleared again.
This bug was introduced late in the Go 1.3 release cycle.
Dave Cheney [Wed, 22 Oct 2014 01:30:15 +0000 (12:30 +1100)]
runtime/cgo: encode BLX directly, fixes one clang build error on arm
Fixes #8348.
Trying to work around clang's dodgy support for .arch by reverting to the external assembler didn't work out so well. Minux had a much better solution to encode the instructions we need as .word directives which avoids .arch altogether.
I've confirmed with gdb that this form produces the expected machine code
Jens Frederich [Tue, 21 Oct 2014 23:56:26 +0000 (10:56 +1100)]
misc/makerelease: set version number in Windows installer
Set correct version number at Windows installer based on
Go's Mercurial tag.
Name | Version
------------------------------------------------
Go Programming Language amd64 go1.3.3 | 1.3.3
Go Programming Language amd64 go1.2rc3 | 1.2
Go Programming Language amd64 go1.2beta1 | 1.2
Some platforms are still not working even with the integrated assembler disabled, will have to find another solution.
««« original CL description
cmd/cgo: disable clang's integrated assembler
Fixes #8348.
Clang's internal assembler (introduced by default in clang 3.4) understands the .arch directive, but doesn't change the default value of -march. This causes the build to fail when we use BLX (armv5 and above) when clang is compiled for the default armv4t architecture (which appears to be the default on all the distros I've used).
This is probably a clang bug, so work around it for the time being by disabling the integrated assembler when compiling the cgo assembly shim.
This CL also includes a small change to ldelf.c which was required as clang 3.4 and above generate more weird symtab entries.
Dave Cheney [Mon, 20 Oct 2014 23:28:39 +0000 (23:28 +0000)]
cmd/cgo: disable clang's integrated assembler
Fixes #8348.
Clang's internal assembler (introduced by default in clang 3.4) understands the .arch directive, but doesn't change the default value of -march. This causes the build to fail when we use BLX (armv5 and above) when clang is compiled for the default armv4t architecture (which appears to be the default on all the distros I've used).
This is probably a clang bug, so work around it for the time being by disabling the integrated assembler when compiling the cgo assembly shim.
This CL also includes a small change to ldelf.c which was required as clang 3.4 and above generate more weird symtab entries.
Keith Randall [Mon, 20 Oct 2014 22:48:42 +0000 (15:48 -0700)]
runtime: fix flaky TestBlockProfile test
It has been failing periodically on Solaris/x64.
Change blockevent so it always records an event if we called
SetBlockProfileRate(1), even if the time delta is negative or zero.
Hopefully this will fix the test on Solaris.
Caveat: I don't actually know what the Solaris problem is, this
is just an educated guess.
LGTM=dave
R=dvyukov, dave
CC=golang-codereviews
https://golang.org/cl/159150043
Ian Lance Taylor [Mon, 20 Oct 2014 17:43:43 +0000 (10:43 -0700)]
reflect: allocate correct type in assignTo and cvtT2I
I came across this while debugging a GC problem in gccgo.
There is code in assignTo and cvtT2I that handles assignment
to all interface values. It allocates an empty interface even
if the real type is a non-empty interface. The fields are
then set for a non-empty interface, but the memory is recorded
as holding an empty interface. This means that the GC has
incorrect information.
This is extremely unlikely to fail, because the code in the GC
that handles empty interfaces looks like this:
In the current runtime the condition is always true--if
KindDirectIface is set, then KindNoPointers is clear--and we
always want to set obj = eface->data. So the question is what
happens when we incorrectly store a non-empty interface value
in memory marked as an empty interface. In that case
eface->type will not be a *rtype as we expect, but will
instead be a pointer to an Itab. We are going to use this
pointer to look at a *rtype kind field. The *rtype struct
starts out like this:
type rtype struct {
size uintptr
hash uint32 // hash of type; avoids computation in hash tables
_ uint8 // unused/padding
align uint8 // alignment of variable with this type
fieldAlign uint8 // alignment of struct field with this type
kind uint8 // enumeration for C
An Itab always has at least two pointers, so on a
little-endian 64-bit system the kind field will be the high
byte of the second pointer. This will normally be zero, so
the test of typ->kind will succeed, which is what we want.
On a 32-bit system it might be possible to construct a failing
case by somehow getting the Itab for an interface with one
method to be immediately followed by a word that is all ones.
The effect would be that the test would sometimes fail and the
GC would not mark obj, leading to an invalid dangling
pointer. I have not tried to construct this test.
I noticed this in gccgo, where this error is much more likely
to cause trouble for a rather random reason: gccgo uses a
different layout of rtype, and in gccgo the kind field happens
to be the low byte of a pointer, not the high byte.
Daniel Morsing [Mon, 20 Oct 2014 14:59:10 +0000 (15:59 +0100)]
cmd/gc: emit code for extern = <N>
https://golang.org/cl/152700045/ made it possible for struct literals assigned to globals to use <N> as the RHS. Normally, this is to zero out variables on first use. Because globals are already zero (or their linker initialized value), we just ignored this.
Now that <N> can occur from non-initialization code, we need to emit this code. We don't use <N> for initialization of globals any more, so this shouldn't cause any excessive zeroing.
Rob Pike [Mon, 20 Oct 2014 14:33:08 +0000 (07:33 -0700)]
encoding/gob: add custom decoder buffer for performance
As we did with encoding, provide a trivial byte reader for
faster decoding. We can also reduce some of the copying
by doing the allocation all at once using a slightly different
interface from byte buffers.
James Robinson [Mon, 20 Oct 2014 01:58:02 +0000 (12:58 +1100)]
compress/flate: add Reset() to allow reusing large buffers to compress multiple buffers
This adds a Reset() to compress/flate's decompressor and plumbs that through
to compress/zlib and compress/gzip's Readers so callers can avoid large
allocations when performing many inflate operations. In particular this
preserves the allocation of the decompressor.hist buffer, which is 32kb and
overwritten as needed while inflating.
On the benchmark described in issue 6317, produces the following speedup on
my 2.3ghz Intel Core i7 MBP with go version devel +6b696a34e0af Sun Aug 03
15:14:59 2014 -0700 darwin/amd64:
blocked.text w/out patch vs blocked.text w/ patch:
benchmark old ns/op new ns/op delta
BenchmarkGunzip 83715775337927917687 -5.30%
benchmark old allocs new allocs delta
BenchmarkGunzip 176818 148519 -16.00%
benchmark old bytes new bytes delta
BenchmarkGunzip 29218493612739528 -95.64%
flat.text vs blocked.text w/patch:
benchmark old ns/op new ns/op delta
BenchmarkGunzip 79394478277927917687 -0.15%
benchmark old allocs new allocs delta
BenchmarkGunzip 90702 148519 +63.74%
benchmark old bytes new bytes delta
BenchmarkGunzip 995952812739528 +27.91%
Similar speedups to those bradfitz saw in https://golang.org/cl/13416045.
Rob Pike [Sun, 19 Oct 2014 17:33:22 +0000 (10:33 -0700)]
flag: disallow setting flags multiple times
This is a day 1 error in the flag package: It did not check
that a flag was set at most once on the command line.
Because user-defined flags may have more general
properties, the check applies only to the standard flag
types in this package: bool, string, etc.
Keith Randall [Sun, 19 Oct 2014 04:02:49 +0000 (21:02 -0700)]
runtime: dequeue the correct SudoG
select {
case <- c:
case <- c:
}
In this case, c.recvq lists two SudoGs which have the same G.
So we can't use the G as the key to dequeue the correct SudoG,
as that key is ambiguous. Dequeueing the wrong SudoG ends up
freeing a SudoG that is still in c.recvq.
The fix is to use the actual SudoG pointer as the key.
Rob Pike [Sat, 18 Oct 2014 18:22:05 +0000 (11:22 -0700)]
text/template: fix bug in pipelined variadics
Simple bug in argument processing: The final arg may
be the pipeline value, in which case it gets bound to the
fixed argument section. The code got that wrong. Easy
to fix.
Rob Pike [Sat, 18 Oct 2014 03:51:15 +0000 (20:51 -0700)]
encoding/gob: use simple append-only buffer for encoding
Bytes buffers have more API and are a little slower. Since appending
is a key part of the path in encode, using a faster implementation
speeds things up measurably.
The couple of positive swings are likely garbage-collection related
since memory allocation looks different in the benchmark now.
I am not concerned by them.
Rob Pike [Fri, 17 Oct 2014 19:37:41 +0000 (12:37 -0700)]
encoding/gob: custom array/slice decoders
Use go generate to write better loops for decoding arrays,
just as we did for encoding. It doesn't help as much,
relatively speaking, but it's still noticeable.
Dmitriy Vyukov [Fri, 17 Oct 2014 17:28:47 +0000 (21:28 +0400)]
runtime/pprof: fix memory profiler test
Don't use cmd/pprof as it is not necessary installed
and does not work on nacl and plan9.
Instead just look at the raw profile.
Russ Cox [Fri, 17 Oct 2014 17:10:42 +0000 (13:10 -0400)]
cmd/gc: don't use static init to initialize small structs, fields
Better to avoid the memory loads and just use immediate constants.
This especially applies to zeroing, which was being done by
copying zeros from elsewhere in the binary, even if the value
was going to be completely initialized with non-zero values.
The zero writes were optimized away but the zero loads from
the data segment were not.
Russ Cox [Fri, 17 Oct 2014 16:54:31 +0000 (12:54 -0400)]
reflect: a few microoptimizations
Replace i < 0 || i >= x with uint(i) >= uint(x).
Shorten a few other code sequences.
Move the kind bits to the bottom of the flag word, to avoid shifts.
Rob Pike [Fri, 17 Oct 2014 16:00:07 +0000 (09:00 -0700)]
encoding/gob: speed up encoding of arrays and slices
We borrow a trick from the fmt package and avoid reflection
to walk the elements when possible. We could push further with
unsafe (and we may) but this is a good start.
Decode can benefit similarly; it will be done separately.
Use go generate (engen.go) to produce the helper functions
(enc_helpers.go).