cmd/compile: fix deadlock in (*Named).load

For lazy import resolution, there's reentrancy issue with (*Named).load
method, when "t.resolve(t)" can lead us to the same named type, thus
(*Named).load is called recursively, causing the deadlock.

The main problem is that when instantinate a type, we calculate the type
hashing, including TParams. Calling t.TParams().Len() triggers the
reentrancy call to "(*Named).load".

To fix this, just not checking TParams().Len() if we are hashing.

Updates #48185

Change-Id: Ie34842d7b10fad5d11fbcf75bb1c64a89deac6b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/347534
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>

diff --git a/src/cmd/compile/internal/types2/typestring.go b/src/cmd/compile/internal/types2/typestring.go
index da5de08758d93cbe34938703bcf2abd3ed654800..6083955306bde4e53e01648824758d77933fdbf7 100644 (file)
--- a/src/cmd/compile/internal/types2/typestring.go
+++ b/src/cmd/compile/internal/types2/typestring.go
@@ -234,7 +234,7 @@ func (w *typeWriter) typ(typ Type) {
                if t.targs != nil {
                        // instantiated type
                        w.typeList(t.targs.list())
-               } else if t.TParams().Len() != 0 {
+               } else if !w.hash && t.TParams().Len() != 0 { // For type hashing, don't need to format the TParams
                        // parameterized type
                        w.tParamList(t.TParams().list())
                }

diff --git a/test/typeparam/issue48185a.dir/p.go b/test/typeparam/issue48185a.dir/p.go
new file mode 100644 (file)
index 0000000..176c7f4
--- /dev/null
+++ b/test/typeparam/issue48185a.dir/p.go
@@ -0,0 +1,19 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+type MarshalOptions struct {
+       Marshalers *Marshalers
+}
+
+type Encoder struct {}
+
+type Marshalers = marshalers[MarshalOptions, Encoder]
+
+type marshalers[Options, Coder any] struct{}
+
+func MarshalFuncV1[T any](fn func(T) ([]byte, error)) *Marshalers {
+       return &Marshalers{}
+}

diff --git a/test/typeparam/issue48185a.dir/p_test.go b/test/typeparam/issue48185a.dir/p_test.go
new file mode 100644 (file)
index 0000000..52c87a7
--- /dev/null
+++ b/test/typeparam/issue48185a.dir/p_test.go
@@ -0,0 +1,11 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+
+import "p"
+
+func main() {
+       _ = p.MarshalFuncV1[int](func(int) ([]byte, error) { return nil, nil })
+}

diff --git a/test/typeparam/issue48185a.go b/test/typeparam/issue48185a.go
new file mode 100644 (file)
index 0000000..40df49f
--- /dev/null
+++ b/test/typeparam/issue48185a.go
@@ -0,0 +1,7 @@
+// rundir
+
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package ignored