cmd/compile: build compiler with PGO

Reapples CL 495596, which was reverted at CL 496185. The x/tools
failure, #60263, has been resolved. The ppc64 failures, #60368, have
_not_ been resolved, but are believed to be specific to that port. This
CL will make ppc64 flaky while the issue is investigated, but give more
soak time on primary ports.

Build the compiler with PGO. As go build -pgo=auto is enabled by
default, we just need to store a profile in the compiler's
directory.

The profile is collected from building all std and cmd packages on
Linux/AMD64 machine, using profile.sh.

This improves the compiler speed. On Linux/AMD64,
name        old time/op       new time/op       delta
Template          138ms ± 5%        136ms ± 4%  -1.44%  (p=0.005 n=36+39)
Unicode           147ms ± 4%        140ms ± 4%  -4.99%  (p=0.000 n=40+39)
GoTypes           780ms ± 3%        778ms ± 4%    ~     (p=0.172 n=39+39)
Compiler          105ms ± 5%         99ms ± 7%  -5.64%  (p=0.000 n=40+40)
SSA               5.83s ± 6%        5.80s ± 6%    ~     (p=0.556 n=40+40)
Flate            89.0ms ± 5%       87.0ms ± 6%  -2.18%  (p=0.000 n=40+40)
GoParser          172ms ± 4%        167ms ± 4%  -2.72%  (p=0.000 n=39+40)
Reflect           333ms ± 4%        333ms ± 3%    ~     (p=0.426 n=40+39)
Tar               128ms ± 4%        126ms ± 4%  -1.82%  (p=0.000 n=39+39)
XML               173ms ± 4%        170ms ± 4%  -1.39%  (p=0.000 n=39+40)
[Geo mean]        253ms             248ms       -2.13%

The profile is pretty transferable. Using the same profile, we
see a bigger win on Darwin/ARM64,
name        old time/op       new time/op       delta
Template         71.0ms ± 2%       68.3ms ± 2%  -3.90%  (p=0.000 n=20+20)
Unicode          71.8ms ± 2%       66.8ms ± 2%  -6.90%  (p=0.000 n=20+20)
GoTypes           444ms ± 1%        428ms ± 1%  -3.53%  (p=0.000 n=19+20)
Compiler         48.9ms ± 3%       45.6ms ± 3%  -6.81%  (p=0.000 n=20+20)
SSA               3.25s ± 2%        3.09s ± 1%  -5.03%  (p=0.000 n=19+20)
Flate            44.0ms ± 2%       42.3ms ± 2%  -3.72%  (p=0.000 n=19+20)
GoParser         76.7ms ± 1%       73.5ms ± 1%  -4.15%  (p=0.000 n=18+19)
Reflect           172ms ± 1%        165ms ± 1%  -4.13%  (p=0.000 n=20+19)
Tar              63.1ms ± 1%       60.4ms ± 2%  -4.24%  (p=0.000 n=19+20)
XML              83.2ms ± 2%       79.2ms ± 2%  -4.79%  (p=0.000 n=20+20)
[Geo mean]        127ms             121ms       -4.73%

For #60368.

Change-Id: I2cec0fc85e21c38d57ba6f0e5e90cde5d443ebd2
Reviewed-on: https://go-review.googlesource.com/c/go/+/497455
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>

diff --git a/src/cmd/compile/default.pgo b/src/cmd/compile/default.pgo
new file mode 100644 (file)
index 0000000..2ba7968
Binary files /dev/null and b/src/cmd/compile/default.pgo differ

diff --git a/src/cmd/compile/profile.sh b/src/cmd/compile/profile.sh
new file mode 100644 (file)
index 0000000..37d65d8
--- /dev/null
+++ b/src/cmd/compile/profile.sh
@@ -0,0 +1,21 @@
+# Copyright 2023 The Go Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style
+# license that can be found in the LICENSE file.
+
+# This script collects a CPU profile of the compiler
+# for building all targets in std and cmd, and puts
+# the profile at cmd/compile/default.pgo.
+
+dir=$(mktemp -d)
+cd $dir
+seed=$(date)
+
+for p in $(go list std cmd); do
+       h=$(echo $seed $p | md5sum | cut -d ' ' -f 1)
+       echo $p $h
+       go build -o /dev/null -gcflags=-cpuprofile=$PWD/prof.$h $p
+done
+
+go tool pprof -proto prof.* > $(go env GOROOT)/src/cmd/compile/default.pgo
+
+rm -r $dir

diff --git a/src/cmd/dist/buildtool.go b/src/cmd/dist/buildtool.go
index af3db5f59023bd9e99fdef6884cac49ca2c3de8d..7ecfb3a811b0e83896accae9bc51ddd832ff909b 100644 (file)
--- a/src/cmd/dist/buildtool.go
+++ b/src/cmd/dist/buildtool.go
@@ -100,6 +100,10 @@ var ignorePrefixes = []string{
 var ignoreSuffixes = []string{
        "_test.s",
        "_test.go",
+       // Skip PGO profile. No need to build toolchain1 compiler
+       // with PGO. And as it is not a text file the import path
+       // rewrite will break it.
+       ".pgo",
 }
 
 var tryDirs = []string{