With this change, default now contains Go 1.5 work.
Any future bug fixes for Go 1.4 in the compilers or
the runtime will have to be made directly to the
release branch.
Austin Clements [Fri, 5 Dec 2014 14:24:01 +0000 (09:24 -0500)]
[dev.cc] liblink: don't patch jumps to jumps to symbols
When liblink sees something like
JMP x
...
x: JMP y
it rewrites the first jump to jump directly to y. This is
fine if y is a resolved label. However, it *also* does this
if y is a function symbol, but fails to carry over the
relocation that would later patch in that symbol's value. As
a result, the original jump becomes either a self-jump (if
relative) or a jump to PC 0 (if absolute).
Fix this by disabling this optimization if the jump being
patched in is a jump to a symbol.
Shenghou Ma [Fri, 5 Dec 2014 07:22:20 +0000 (02:22 -0500)]
[dev.cc] cmd/ld: finalize linkmode before determining whether to import runtime/cgo
Frankly, I don't understand how the current code could possibly work except
when every android program is using cgo. Discovered this while working on
the iOS port.
Rob Pike [Fri, 5 Dec 2014 00:15:38 +0000 (09:15 +0900)]
cmd/go: avoid use of bufio.Scanner in generate
Scanner can't handle stupid long lines and there are
reports of stupid long lines in production.
Note the issue isn't long "//go:generate" lines, but
any long line in any Go source file.
To be fair, if you're going to have a stupid long line
it's not a bad bet you'll want to run it through go
generate, because it's some embeddable asset that
has been machine generated. (One could ask why
that generation process didn't add a newline or two,
but we should cope anyway.)
Rewrite the file scanner in "go generate" so it can
handle arbitrarily long lines, and only stores in memory
those lines that start "//go:generate".
Also: Adjust the documentation to make clear that it
does not parse the file.
Russ Cox [Mon, 1 Dec 2014 21:32:06 +0000 (16:32 -0500)]
runtime: fix hang in GC due to shrinkstack vs netpoll race
During garbage collection, after scanning a stack, we think about
shrinking it to reclaim some memory. The shrinking code (called
while the world is stopped) checked that the status was Gwaiting
or Grunnable and then changed the state to Gcopystack, to essentially
lock the stack so that no other GC thread is scanning it.
The same locking happens for stack growth (and is more necessary there).
oldstatus = runtime·readgstatus(gp);
oldstatus &= ~Gscan;
if(oldstatus == Gwaiting || oldstatus == Grunnable)
runtime·casgstatus(gp, oldstatus, Gcopystack); // oldstatus is Gwaiting or Grunnable
else
runtime·throw("copystack: bad status, not Gwaiting or Grunnable");
Unfortunately, "stop the world" doesn't stop everything. It stops all
normal goroutine execution, but the network polling thread is still
blocked in epoll and may wake up. If it does, and it chooses a goroutine
to mark runnable, and that goroutine is the one whose stack is shrinking,
then it can happen that between readgstatus and casgstatus, the status
changes from Gwaiting to Grunnable.
casgstatus assumes that if the status is not what is expected, it is a
transient change (like from Gwaiting to Gscanwaiting and back, or like
from Gwaiting to Gcopystack and back), and it loops until the status
has been restored to the expected value. In this case, the status has
changed semi-permanently from Gwaiting to Grunnable - it won't
change again until the GC is done and the world can continue, but the
GC is waiting for the status to change back. This wedges the program.
To fix, call a special variant of casgstatus that accepts either Gwaiting
or Grunnable as valid statuses.
Without the fix bug with the extra check+throw in casgstatus, the
program below dies in a few seconds (2-10) with GOMAXPROCS=8
on a 2012 Retina MacBook Pro. With the fix, it runs for minutes
and minutes.
package main
import (
"io"
"log"
"net"
"runtime"
)
func main() {
const N = 100
for i := 0; i < N; i++ {
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
log.Fatal(err)
}
ch := make(chan net.Conn, 1)
go func() {
var err error
c1, err := net.Dial("tcp", l.Addr().String())
if err != nil {
log.Fatal(err)
}
ch <- c1
}()
c2, err := l.Accept()
if err != nil {
log.Fatal(err)
}
c1 := <-ch
l.Close()
go netguy(c1, c2)
go netguy(c2, c1)
c1.Write(make([]byte, 100))
}
for {
runtime.GC()
}
}
func netguy(r, w net.Conn) {
buf := make([]byte, 100)
for {
bigstack(1000)
_, err := io.ReadFull(r, buf)
if err != nil {
log.Fatal(err)
}
w.Write(buf)
}
}
var g int
func bigstack(n int) {
var buf [100]byte
if n > 0 {
bigstack(n - 1)
}
g = int(buf[0]) + int(buf[99])
}
Fixes #9186.
LGTM=rlh
R=austin, rlh
CC=dvyukov, golang-codereviews, iant, khr, r
https://golang.org/cl/179680043
Austin Clements [Tue, 25 Nov 2014 21:00:25 +0000 (16:00 -0500)]
[dev.cc] 9l: make R_CALLPOWER like ELF's R_PPC64_REL24
These accomplished the same thing, but R_CALLPOWER expected
the whole instruction to be in the addend (and completely
overwrote what was in the text section), while R_PPC64_REL24
overwrites only bits 6 through 24 of whatever was in the text
section. Make R_CALLPOWER work like R_PPC64_REL24 to ease the
implementation of dynamic linking.
David du Colombier [Tue, 25 Nov 2014 07:42:00 +0000 (08:42 +0100)]
[dev.cc] cmd/5g,cmd/6g,cmd/9g: fix warnings on Plan 9
warning: src/cmd/5g/reg.c:461 format mismatch d VLONG, arg 5
warning: src/cmd/6g/reg.c:396 format mismatch d VLONG, arg 5
warning: src/cmd/9g/reg.c:440 format mismatch d VLONG, arg 5
Austin Clements [Mon, 24 Nov 2014 16:40:36 +0000 (11:40 -0500)]
[dev.cc] 9g: fill progtable for CC, V, and VCC instruction variants
This adds some utilities for converting between the CC, V, and
VCC variants of operations and uses these to derive the
ProgInfo entries for these variants (which are identical to
the ProgInfo for the base operations).
The 9g peephole optimizer will also use these conversion
utilities.
David du Colombier [Fri, 21 Nov 2014 19:56:33 +0000 (20:56 +0100)]
[dev.cc] liblink: fix warnings on Plan 9
warning: src/liblink/list6.c:94 set and not used: s
warning: src/liblink/list6.c:157 format mismatch ld VLONG, arg 3
warning: src/liblink/list6.c:157 format mismatch E UINT, arg 4
warning: src/liblink/list6.c:157 format mismatch d VLONG, arg 5
warning: src/liblink/list6.c:163 set and not used: s
warning: src/liblink/list9.c:105 set and not used: s
warning: src/liblink/list9.c:185 format mismatch ld VLONG, arg 3
warning: src/liblink/list9.c:185 format mismatch E UINT, arg 4
warning: src/liblink/list9.c:185 format mismatch d VLONG, arg 5
warning: src/liblink/list9.c:193 set and not used: s
Dmitriy Vyukov [Thu, 20 Nov 2014 14:51:02 +0000 (09:51 -0500)]
runtime: fix atomic operations on non-heap addresses
Race detector runtime does not tolerate operations on addresses
that was not previously declared with __tsan_map_shadow
(namely, data, bss and heap). The corresponding address
checks for atomic operations were removed in
https://golang.org/cl/111310044
Restore these checks.
It's tricker than just not calling into race runtime,
because it is the race runtime that makes the atomic
operations themselves (if we do not call into race runtime
we skip the atomic operation itself as well). So instead we call
__tsan_go_ignore_sync_start/end around the atomic operation.
This forces race runtime to skip all other processing
except than doing the atomic operation itself.
Fixes #9136.
Russ Cox [Wed, 19 Nov 2014 20:25:33 +0000 (15:25 -0500)]
runtime: remove assumption that noptrdata data bss noptrbss are ordered and contiguous
The assumption can be violated by external linkers reordering them or
inserting non-Go sections in between them. I looked briefly at trying
to write out the _go_.o in external linking mode in a way that forced
the ordering, but no matter what there's no way to force Go's data
and Go's bss to be next to each other. If there is any data or bss from
non-Go objects, it's very likely to get stuck in between them.
Instead, rewrite the two places we know about that make the assumption.
I grepped for noptrdata to look for more and didn't find any.
The added race test (os/exec in external linking mode) fails without
the changes in the runtime. It crashes with an invalid pointer dereference.
Austin Clements [Wed, 19 Nov 2014 19:56:49 +0000 (14:56 -0500)]
[dev.cc] runtime: add explicit siginfo.si_addr field
struct siginfo_t's si_addr field is part of a union.
Previously, we represented this union in Go using an opaque
byte array and accessed the si_addr field using unsafe (and
wrong on 386 and arm!) pointer arithmetic. Since si_addr is
the only field we use from this union, this replaces the
opaque byte array with an explicit declaration of the si_addr
field and accesses it directly.
Austin Clements [Wed, 19 Nov 2014 19:24:41 +0000 (14:24 -0500)]
[dev.cc] runtime: decode power64 branch instructions the way the CPU does
Previously, this used the top 8 bits of an instruction as a
sort-of opcode and ignored the top two bits of the relative
PC. This worked because these jumps are always negative and
never big enough for the top two bits of the relative PC (also
the bottom 2 bits of the sort-of opcode) to be anything other
than 0b11, but the code is confusing because it doesn't match
the actual structure of the instruction.
Instead, use the real 6 bit opcode and use all 24 bits of
relative PC.
LGTM=rsc
R=rsc, dave
CC=golang-codereviews
https://golang.org/cl/179960043
In go1.3, I type enter at the prompt and the program exits.
With the CL being rolled back, the program wedges at the
prompt.
««« original CL description
syscall: SysProcAttr job control changes
Making the child's process group the foreground process group and
placing the child in a specific process group involves co-ordination
between the parent and child that must be done post-fork but pre-exec.
Austin Clements [Wed, 19 Nov 2014 16:30:58 +0000 (11:30 -0500)]
[dev.cc] runtime: allow more address bits in lfstack on Power64
Previously, lfstack assumed Linux limited user space addresses
to 43 bits on Power64 based on a paper from 2001. It turns
out the limit is now 46 bits, so lfstack was truncating
pointers.
Raise the limit to 48 bits (for some future proofing and to
make it match amd64) and add a self-test that will fail in a
useful way if ever unpack(pack(x)) != x.
With this change, dev.cc passes all.bash on power64le.
Austin Clements [Tue, 18 Nov 2014 20:18:52 +0000 (15:18 -0500)]
[dev.cc] 9a: make RET a synonym for RETURN; use "g" instead of "R30"
Previously, 9a was the only assembler that had a different
name for RET, causing unnecessary friction in simple files
that otherwise assembled on all architectures. Add RET so
these work on 9a.
This also renames "R30" to "g" to avoid unintentionally
clobbering g in assembly code. This parallels a change made
to 5a.
Russ Cox [Tue, 18 Nov 2014 17:07:50 +0000 (12:07 -0500)]
[dev.cc] runtime: generate GOOS- and GOARCH-specific files with go generate
Eventually I'd like almost everything cmd/dist generates
to be done with 'go generate' and checked in, to simplify
the bootstrap process. The only thing cmd/dist really needs
to do is write things like the current experiment info and
the current version.
This is a first step toward that. It replaces the _NaCl etc
constants with generated ones goos_nacl, goos_darwin,
goarch_386, and so on.
LGTM=dave, austin
R=austin, dave, bradfitz
CC=golang-codereviews, iant, r
https://golang.org/cl/174290043
Austin Clements [Mon, 17 Nov 2014 19:44:41 +0000 (14:44 -0500)]
cmd/pprof: fix EOF handling when getting function source
getFunctionSource gathers five lines of "margin" around every
requested sample line. However, if this margin went past the
end of the source file, getFunctionSource would encounter an
io.EOF error and abort with this error, resulting in listings
like
(pprof) list main.main
ROUTINE ======================== main.main in ...
0 8.33s (flat, cum) 99.17% of Total
Error: EOF
(pprof)
Modify the error handling in getFunctionSource so io.EOF is
always considered non-fatal. If it reaches EOF, it simply
returns the lines it has.
Russ Cox [Mon, 17 Nov 2014 01:52:45 +0000 (20:52 -0500)]
debug/goobj: move to cmd/internal/goobj
debug/goobj is not ready to be published but it is
needed for the various binary-reading commands.
Move to cmd/internal/goobj.
(The Go 1.3 release branch deleted it, but that's not
an option anymore due to the command dependencies.
The API is still not vetted nor terribly well designed.)
David du Colombier [Sun, 16 Nov 2014 21:55:07 +0000 (22:55 +0100)]
[dev.cc] cmd/8g: work around "out of fixed registers" on Plan 9
This change works around the "out of fixed registers"
issue with the Plan 9 C compiler on 386, introduced by
the Bits change to uint64 in CL 169060043.
The purpose of this CL is to be able to properly
follow the conversion of the Plan 9 runtime to Go
on the Plan 9 builders.
This CL could be reverted once the Go compilers will
be converted to Go.
Thanks to Nick Owens for investigating this issue.
Russ Cox [Sun, 16 Nov 2014 21:44:45 +0000 (16:44 -0500)]
runtime: fix sudog leak
The SudoG used to sit on the stack, so it was cheap to allocated
and didn't need to be cleaned up when finished.
For the conversion to Go, we had to move sudog off the stack
for a few reasons, so we added a cache of recently used sudogs
to keep allocation cheap. But we didn't add any of the necessary
cleanup before adding a SudoG to the new cache, and so the cached
SudoGs had stale pointers inside them that have caused all sorts
of awful, hard to debug problems.
CL 155760043 made sure SudoG.elem is cleaned up.
CL 150520043 made sure SudoG.selectdone is cleaned up.
This CL makes sure SudoG.next, SudoG.prev, and SudoG.waitlink
are cleaned up. I should have done this when I did the other two
fields; instead I wasted a week tracking down a leak they caused.
A dangling SudoG.waitlink can point into a sudogcache list that
has been "forgotten" in order to let the GC collect it, but that
dangling .waitlink keeps the list from being collected.
And then the list holding the SudoG with the dangling waitlink
can find itself in the same situation, and so on. We end up
with lists of lists of unusable SudoGs that are still linked into
the object graph and never collected (given the right mix of
non-trivial selects and non-channel synchronization).
Dave Cheney [Sat, 15 Nov 2014 02:27:05 +0000 (13:27 +1100)]
[dev.cc] runtime: fix _sfloat thunk
* _sfloat dispatches to runtime._sfloat2 with the Go calling convention, so the seecond argument is a [15]uint32, not a *[15]uint32.
* adjust _sfloat2 to return the new pc in 68(R13) as expected.
David du Colombier [Fri, 14 Nov 2014 21:57:33 +0000 (22:57 +0100)]
[dev.cc] liblink: fix warnings on Plan 9
warning: src/liblink/asm9.c:501 set and not used: bflag
warning: src/liblink/list9.c:259 format mismatch .5lux INT, arg 4
warning: src/liblink/list9.c:261 format mismatch .5lux INT, arg 3
warning: src/liblink/list9.c:319 more arguments than format VLONG
warning: src/liblink/obj9.c:222 set and not used: autoffset
Austin Clements [Fri, 14 Nov 2014 18:58:31 +0000 (13:58 -0500)]
[dev.power64] 6g,9g: formatters for Prog and Addr details
The pretty printers for these make it hard to understand
what's actually in the fields of these structures. These
"ugly printers" show exactly what's in each field, which can
be useful for understanding and debugging code.
Russ Cox [Fri, 14 Nov 2014 17:09:42 +0000 (12:09 -0500)]
[dev.garbage] all: merge dev.power64 (7667e41f3ced) into dev.garbage
Now the only difference between dev.cc and dev.garbage
is the runtime conversion on the one side and the
garbage collection on the other. They both have the
same set of changes from default and dev.power64.
Austin Clements [Fri, 14 Nov 2014 17:08:46 +0000 (12:08 -0500)]
[dev.power64] liblink: generate dnames[5689] for D_* constants
This is more complicated than the other enums because the D_*
enums are full of explicit initializers and repeated values.
This tries its best. (This will get much cleaner once we
tease these constants apart better.)
Austin Clements [Fri, 14 Nov 2014 16:56:31 +0000 (11:56 -0500)]
[dev.power64] 5g,6g,8g,9g: debug prints for regopt pass 6 and paint2
Theses were very helpful in understanding the regions and
register selection when porting regopt to 9g. Add them to the
other compilers (and improve 9g's successor debug print).