Russ Cox [Wed, 29 Oct 2014 01:50:16 +0000 (21:50 -0400)]
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped
I removed support for jumping between functions years ago,
as part of doing the instruction layout for each function separately.
Given that, it makes sense to treat labels as function-scoped.
This lets each function have its own 'loop' label, for example.
Makes the assembly much cleaner and removes the last
reason anyone would reach for the 123(PC) form instead.
Note that this is on the dev.power64 branch, but it changes all
the assemblers. The change will ship in Go 1.5 (perhaps after
being ported into the new assembler).
David du Colombier [Tue, 28 Oct 2014 21:44:59 +0000 (22:44 +0100)]
os: fix write on Plan 9
In CL 160670043 the write function was changed
so a zero-length write is now allowed. This leads
the ExampleWriter_Init test to fail.
The reason is that Plan 9 preserves message
boundaries, while the os library expects systems
that don't preserve them. We have to ignore
zero-length writes so they will never turn into EOF.
This issue was previously discussed in CL 7406046.
Austin Clements [Tue, 28 Oct 2014 19:57:33 +0000 (15:57 -0400)]
[dev.power64] runtime: fix atomicor8 for power64x
Power64 servers do not currently support sub-word size atomic
memory access, so atomicor8 uses word size atomic access.
However, previously atomicor8 made no attempt to align this
access, resulting in errors. Fix this by aligning the pointer
to a word boundary and shifting the value appropriately.
Since atomicor8 is used in GC, add a test to runtime·check to
make sure this doesn't break in the future.
This also fixes an incorrect branch label, an incorrectly
sized argument move, and adds argument names to help go vet.
LGTM=rsc
R=rsc, dave
CC=golang-codereviews
https://golang.org/cl/165820043
Austin Clements [Tue, 28 Oct 2014 19:08:09 +0000 (15:08 -0400)]
[dev.power64] 9a: correct generation of four argument ops
The "to" field was the penultimate argument to outgcode,
instead of the last argument, which swapped the third and
fourth operands. The argument order was correct in a.y, so
just swap the meaning of the arguments in outgcode. This
hadn't come up because we hadn't used these more obscure
operations in any hand-written assembly until now.
LGTM=rsc, dave
R=rsc, dave
CC=golang-codereviews
https://golang.org/cl/160690043
Russ Cox [Tue, 28 Oct 2014 15:35:00 +0000 (11:35 -0400)]
syscall: fix ParseRoutingSockaddr with unexpected submessages
No easy way to test (would have to actually trigger some routing
events from kernel) but the code is clearly wrong as written.
If the header says there is a submessage, we need to at least
skip over its bytes, not just continue to the next iteration.
Fixes #8203.
LGTM=r
R=r
CC=golang-codereviews, mikioh.mikioh, p
https://golang.org/cl/164140044
Russ Cox [Tue, 28 Oct 2014 15:14:25 +0000 (11:14 -0400)]
cmd/go: add get -f flag
get -u now checks that remote repo paths match the
ones predicted by the import paths: if you are get -u'ing
rsc.io/pdf, it has to be checked out from the right location.
This is important in case the rsc.io/pdf redirect changes.
In some cases, people have good reasons to use
non-standard remote repos. Add -f flag to allow that.
The f can stand for force or fork, as you see fit.
Mikio Hara [Tue, 28 Oct 2014 07:20:49 +0000 (16:20 +0900)]
net: add test for lookupIPDeadline
Just to confirm the fix, by typing the follwing:
go test -run=TestLookupIPDeadline -dnsflood or
go test -run=TestLookupIPDeadline -dnsflood -tags netgo
Rob Pike [Tue, 28 Oct 2014 00:08:50 +0000 (17:08 -0700)]
doc/go_mem.html: don't be clever
Add a short introductory section saying what most Go
programmers really need to know, which is that you
shouldn't have to read this document to understand
the behavior of your program.
Robert Griesemer [Mon, 27 Oct 2014 23:31:15 +0000 (16:31 -0700)]
spec: permit parentheses around builtin function names
Not a language change.
This is simply documenting the status quo which permits
builtin function names to be parenthesized in calls; e.g.,
both
len(s)
and
(((len)))(s)
are accepted by all compilers and go/types.
Changed the grammar by merging the details of BuiltinCall
with ordinary Calls. Also renamed the Call production to
Arguments which more clearly identifies that part of the
grammar and also matches better with its counterpart on
the declaration side (Parameters).
The fact that the first argument can be a type (for builtins)
or cannot be a type (for regular function calls) is expressed
in the prose, no need to make the grammar more complicated.
Fixes #9001.
LGTM=iant, r, rsc
R=r, rsc, iant, ken, dave
CC=golang-codereviews
https://golang.org/cl/160570043
Russ Cox [Mon, 27 Oct 2014 22:59:02 +0000 (18:59 -0400)]
test: make maplinear more robust
The test just doubled a certain number of times
and then gave up. On a mostly fast but occasionally
slow machine this may never make the test run
long enough to see the linear growth.
Change test to keep doubling until the first round
takes at least a full second, to reduce the effect of
occasional scheduling or other jitter.
The failure we saw had a time for the first round
of around 100ms.
Note that this test still passes once it sees a linear
effect, even with a very small total time.
The timeout here only applies to how long the execution
must be to support a reported failure.
Austin Clements [Mon, 27 Oct 2014 21:27:03 +0000 (17:27 -0400)]
[dev.power64] runtime: power64 fixes and ports of changes
Fix include paths that got moved in the great pkg/ rename. Add
missing runtime/arch_* files for power64. Port changes that
happened on default since branching to
runtime/{asm,atomic,sys_linux}_power64x.s (precise stacks,
calling convention change, various new and deleted functions.
Port struct renaming and fix some bugs in
runtime/defs_linux_power64.h.
LGTM=rsc
R=rsc, dave
CC=golang-codereviews
https://golang.org/cl/161450043
Austin Clements [Mon, 27 Oct 2014 21:19:41 +0000 (17:19 -0400)]
[dev.power64] liblink: fix lost branch target
A recent commit lost the branch target in the really-big-stack
case of splitstack, causing an infinite loop stack preempt
case. Revive the branch target.
Austin Clements [Mon, 27 Oct 2014 21:12:48 +0000 (17:12 -0400)]
runtime: fix endianness assumption when decoding ftab
The ftab ends with a half functab record consisting only of
the 'entry' field followed by a uint32 giving the offset of
the next table. Previously, symtabinit assumed it could read
this uint32 as a uintptr. Since this is unsafe on big endian,
explicitly read the offset as a uint32.
Austin Clements [Mon, 27 Oct 2014 19:25:40 +0000 (15:25 -0400)]
[dev.power64] liblink: power64 fixes and ports of changes
Ports of platform-specific changes that happened on default
after dev.power64 forked (fixes for c2go, wrapper math fixes,
moved stackguard field, stackguard1 support, precise stacks).
Bug fixes (missing AMOVW in instruction table, correct
unsigned 32-bit remainder).
Ian Lance Taylor [Mon, 27 Oct 2014 15:46:18 +0000 (08:46 -0700)]
net: if a DNS lookup times out, forget that it is in flight
Before this CL, if the system resolver does a very slow DNS
lookup for a particular host, all subsequent requests for that
host will hang waiting for that lookup to complete. That is
more or less expected when Dial is called with no deadline.
When Dial has a deadline, though, we can accumulate a large
number of goroutines waiting for that slow DNS lookup. Try to
avoid this problem by restarting the DNS lookup when it is
redone after a deadline is passed.
This CL also avoids creating an extra goroutine purely to
handle the deadline.
No test because we would have to simulate a slow DNS lookup
followed by a fast DNS lookup.
Emil Hessman [Mon, 27 Oct 2014 01:43:14 +0000 (12:43 +1100)]
misc/makerelease/windows: fix 404 help URL in installer
ARPHELPLINK yields 404; update the URL.
While here, also prefix the ARPREADME and ARPURLINFOABOUT URL's with the HTTP scheme to make 'em clickable links in the Add or Remove Programs listing.
Rob Pike [Fri, 24 Oct 2014 16:37:25 +0000 (09:37 -0700)]
unsafe: document that unsafe programs are not protected
The compatibility guideline needs to be clear about this even
though it means adding a clause that was not there from the
beginning. It has always been understood, so this isn't really
a change in policy, just in its expression.
Russ Cox [Fri, 24 Oct 2014 14:27:39 +0000 (10:27 -0400)]
cmd/gc: synthesize zeroed value for non-assignment context
CL 157910047 introduced code to turn a node representing
a zeroed composite literal into N, the nil Node* pointer
(which represents any zero, not the Go literal nil).
That's great for assignments like x = T{}, but it doesn't work
when T{} is used in a value context like T{}.v or x == T{}.
Fix those.
Should have no effect on performance; confirmed.
The deltas below are noise (compare ns/op):
Russ Cox [Fri, 24 Oct 2014 03:44:47 +0000 (23:44 -0400)]
encoding/csv: for Postgres, unquote empty strings, quote \.
In theory both of these lines encode the same three fields:
a,,c
a,"",c
However, Postgres defines that when importing CSV, the unquoted
version is treated as NULL (missing), while the quoted version is
treated as a string value (empty string). If the middle field is supposed to
be an integer value, the first line can be imported (NULL is okay), but
the second line cannot (empty string is not).
Postgres's import command (COPY FROM) has an option to force
the unquoted empty to be interpreted as a string but it does not
have an option to force the quoted empty to be interpreted as a NULL.
From http://www.postgresql.org/docs/9.0/static/sql-copy.html:
The CSV format has no standard way to distinguish a NULL
value from an empty string. PostgreSQL's COPY handles this
by quoting. A NULL is output as the NULL parameter string
and is not quoted, while a non-NULL value matching the NULL
parameter string is quoted. For example, with the default
settings, a NULL is written as an unquoted empty string,
while an empty string data value is written with double
quotes (""). Reading values follows similar rules. You can
use FORCE_NOT_NULL to prevent NULL input comparisons for
specific columns.
Therefore printing the unquoted empty is more flexible for
imports into Postgres than printing the quoted empty.
In addition to making the output more useful with Postgres, not
quoting empty strings makes the output smaller and easier to read.
It also matches the behavior of Microsoft Excel and Google Drive.
Since we are here and making concessions for Postgres, handle this
case too (again quoting the Postgres docs):
Because backslash is not a special character in the CSV
format, \., the end-of-data marker, could also appear as a
data value. To avoid any misinterpretation, a \. data value
appearing as a lone entry on a line is automatically quoted
on output, and on input, if quoted, is not interpreted as
the end-of-data marker. If you are loading a file created by
another application that has a single unquoted column and
might have a value of \., you might need to quote that value
in the input file.
Austin Clements [Wed, 22 Oct 2014 20:39:31 +0000 (16:39 -0400)]
[dev.power64] runtime: fix early GC of Defer objects
go_bootstrap was panicking during runtime initialization
(under runtime.main) because Defer objects were being
prematurely GC'd. This happened because of an incorrect
change to runtime·unrollgcprog_m to make it endian-agnostic
during the conversion of runtime bitmaps to byte arrays.
Austin Clements [Wed, 22 Oct 2014 19:51:54 +0000 (15:51 -0400)]
[dev.power64] all: merge default into dev.power64
This brings dev.power64 up-to-date with the current tip of
default. go_bootstrap is still panicking with a bad defer
when initializing the runtime (even on amd64).
Austin Clements [Wed, 22 Oct 2014 18:02:04 +0000 (14:02 -0400)]
[dev.power64] runtime: Fix broken merge of noasm.go
The earlier dev.power64 merge missed the port of
runtime/noasm.goc to runtime/noasm_arm.go. This CL fixes this
by moving noasm_arm.go to noasm.go and adding a +build to
share the file between arm and power64.
Dmitriy Vyukov [Wed, 22 Oct 2014 16:23:49 +0000 (20:23 +0400)]
sync: release Pool memory during second and later GCs
Pool memory was only being released during the first GC after the first Put.
Put assumes that p.local != nil means p is on the allPools list.
poolCleanup (called during each GC) removed each pool from allPools
but did not clear p.local, so each pool was cleared by exactly one GC
and then never cleared again.
This bug was introduced late in the Go 1.3 release cycle.
Dave Cheney [Wed, 22 Oct 2014 01:30:15 +0000 (12:30 +1100)]
runtime/cgo: encode BLX directly, fixes one clang build error on arm
Fixes #8348.
Trying to work around clang's dodgy support for .arch by reverting to the external assembler didn't work out so well. Minux had a much better solution to encode the instructions we need as .word directives which avoids .arch altogether.
I've confirmed with gdb that this form produces the expected machine code
Jens Frederich [Tue, 21 Oct 2014 23:56:26 +0000 (10:56 +1100)]
misc/makerelease: set version number in Windows installer
Set correct version number at Windows installer based on
Go's Mercurial tag.
Name | Version
------------------------------------------------
Go Programming Language amd64 go1.3.3 | 1.3.3
Go Programming Language amd64 go1.2rc3 | 1.2
Go Programming Language amd64 go1.2beta1 | 1.2
Some platforms are still not working even with the integrated assembler disabled, will have to find another solution.
««« original CL description
cmd/cgo: disable clang's integrated assembler
Fixes #8348.
Clang's internal assembler (introduced by default in clang 3.4) understands the .arch directive, but doesn't change the default value of -march. This causes the build to fail when we use BLX (armv5 and above) when clang is compiled for the default armv4t architecture (which appears to be the default on all the distros I've used).
This is probably a clang bug, so work around it for the time being by disabling the integrated assembler when compiling the cgo assembly shim.
This CL also includes a small change to ldelf.c which was required as clang 3.4 and above generate more weird symtab entries.
Dave Cheney [Mon, 20 Oct 2014 23:28:39 +0000 (23:28 +0000)]
cmd/cgo: disable clang's integrated assembler
Fixes #8348.
Clang's internal assembler (introduced by default in clang 3.4) understands the .arch directive, but doesn't change the default value of -march. This causes the build to fail when we use BLX (armv5 and above) when clang is compiled for the default armv4t architecture (which appears to be the default on all the distros I've used).
This is probably a clang bug, so work around it for the time being by disabling the integrated assembler when compiling the cgo assembly shim.
This CL also includes a small change to ldelf.c which was required as clang 3.4 and above generate more weird symtab entries.
Keith Randall [Mon, 20 Oct 2014 22:48:42 +0000 (15:48 -0700)]
runtime: fix flaky TestBlockProfile test
It has been failing periodically on Solaris/x64.
Change blockevent so it always records an event if we called
SetBlockProfileRate(1), even if the time delta is negative or zero.
Hopefully this will fix the test on Solaris.
Caveat: I don't actually know what the Solaris problem is, this
is just an educated guess.
LGTM=dave
R=dvyukov, dave
CC=golang-codereviews
https://golang.org/cl/159150043
Ian Lance Taylor [Mon, 20 Oct 2014 17:43:43 +0000 (10:43 -0700)]
reflect: allocate correct type in assignTo and cvtT2I
I came across this while debugging a GC problem in gccgo.
There is code in assignTo and cvtT2I that handles assignment
to all interface values. It allocates an empty interface even
if the real type is a non-empty interface. The fields are
then set for a non-empty interface, but the memory is recorded
as holding an empty interface. This means that the GC has
incorrect information.
This is extremely unlikely to fail, because the code in the GC
that handles empty interfaces looks like this:
In the current runtime the condition is always true--if
KindDirectIface is set, then KindNoPointers is clear--and we
always want to set obj = eface->data. So the question is what
happens when we incorrectly store a non-empty interface value
in memory marked as an empty interface. In that case
eface->type will not be a *rtype as we expect, but will
instead be a pointer to an Itab. We are going to use this
pointer to look at a *rtype kind field. The *rtype struct
starts out like this:
type rtype struct {
size uintptr
hash uint32 // hash of type; avoids computation in hash tables
_ uint8 // unused/padding
align uint8 // alignment of variable with this type
fieldAlign uint8 // alignment of struct field with this type
kind uint8 // enumeration for C
An Itab always has at least two pointers, so on a
little-endian 64-bit system the kind field will be the high
byte of the second pointer. This will normally be zero, so
the test of typ->kind will succeed, which is what we want.
On a 32-bit system it might be possible to construct a failing
case by somehow getting the Itab for an interface with one
method to be immediately followed by a word that is all ones.
The effect would be that the test would sometimes fail and the
GC would not mark obj, leading to an invalid dangling
pointer. I have not tried to construct this test.
I noticed this in gccgo, where this error is much more likely
to cause trouble for a rather random reason: gccgo uses a
different layout of rtype, and in gccgo the kind field happens
to be the low byte of a pointer, not the high byte.
Daniel Morsing [Mon, 20 Oct 2014 14:59:10 +0000 (15:59 +0100)]
cmd/gc: emit code for extern = <N>
https://golang.org/cl/152700045/ made it possible for struct literals assigned to globals to use <N> as the RHS. Normally, this is to zero out variables on first use. Because globals are already zero (or their linker initialized value), we just ignored this.
Now that <N> can occur from non-initialization code, we need to emit this code. We don't use <N> for initialization of globals any more, so this shouldn't cause any excessive zeroing.
Rob Pike [Mon, 20 Oct 2014 14:33:08 +0000 (07:33 -0700)]
encoding/gob: add custom decoder buffer for performance
As we did with encoding, provide a trivial byte reader for
faster decoding. We can also reduce some of the copying
by doing the allocation all at once using a slightly different
interface from byte buffers.
James Robinson [Mon, 20 Oct 2014 01:58:02 +0000 (12:58 +1100)]
compress/flate: add Reset() to allow reusing large buffers to compress multiple buffers
This adds a Reset() to compress/flate's decompressor and plumbs that through
to compress/zlib and compress/gzip's Readers so callers can avoid large
allocations when performing many inflate operations. In particular this
preserves the allocation of the decompressor.hist buffer, which is 32kb and
overwritten as needed while inflating.
On the benchmark described in issue 6317, produces the following speedup on
my 2.3ghz Intel Core i7 MBP with go version devel +6b696a34e0af Sun Aug 03
15:14:59 2014 -0700 darwin/amd64:
blocked.text w/out patch vs blocked.text w/ patch:
benchmark old ns/op new ns/op delta
BenchmarkGunzip 83715775337927917687 -5.30%
benchmark old allocs new allocs delta
BenchmarkGunzip 176818 148519 -16.00%
benchmark old bytes new bytes delta
BenchmarkGunzip 29218493612739528 -95.64%
flat.text vs blocked.text w/patch:
benchmark old ns/op new ns/op delta
BenchmarkGunzip 79394478277927917687 -0.15%
benchmark old allocs new allocs delta
BenchmarkGunzip 90702 148519 +63.74%
benchmark old bytes new bytes delta
BenchmarkGunzip 995952812739528 +27.91%
Similar speedups to those bradfitz saw in https://golang.org/cl/13416045.
Rob Pike [Sun, 19 Oct 2014 17:33:22 +0000 (10:33 -0700)]
flag: disallow setting flags multiple times
This is a day 1 error in the flag package: It did not check
that a flag was set at most once on the command line.
Because user-defined flags may have more general
properties, the check applies only to the standard flag
types in this package: bool, string, etc.
Keith Randall [Sun, 19 Oct 2014 04:02:49 +0000 (21:02 -0700)]
runtime: dequeue the correct SudoG
select {
case <- c:
case <- c:
}
In this case, c.recvq lists two SudoGs which have the same G.
So we can't use the G as the key to dequeue the correct SudoG,
as that key is ambiguous. Dequeueing the wrong SudoG ends up
freeing a SudoG that is still in c.recvq.
The fix is to use the actual SudoG pointer as the key.
Rob Pike [Sat, 18 Oct 2014 18:22:05 +0000 (11:22 -0700)]
text/template: fix bug in pipelined variadics
Simple bug in argument processing: The final arg may
be the pipeline value, in which case it gets bound to the
fixed argument section. The code got that wrong. Easy
to fix.
Rob Pike [Sat, 18 Oct 2014 03:51:15 +0000 (20:51 -0700)]
encoding/gob: use simple append-only buffer for encoding
Bytes buffers have more API and are a little slower. Since appending
is a key part of the path in encode, using a faster implementation
speeds things up measurably.
The couple of positive swings are likely garbage-collection related
since memory allocation looks different in the benchmark now.
I am not concerned by them.