go/doc/comment: add heuristics for common badly formatted comments
In a set of 55M Go doc comments drawn from the latest version of
all public Go modules known to the module proxy in spring 2020,
the current Go 1.19 gofmt reformats about 1.57M of them.
Out of those 1.57M comments, inspection of random samples
shows that around 5% of the changed comments contain
unindented code snippets, multiline shell commands, or lists.
For example:
// Here is a greeting:
//
// func main() {
// fmt.Println("hello, world")
// }
// Run this command:
//
// path/to/your/program -flag1=longargument1 \
// -flag2=longargument2 \
// -flag3
// There are three possibilities:
//
// - Unindented code snippets (or JSON objects)
// in which the first and last line are unindented
// but end in { and start with }, respectively.
// - Unindented multiline shell commands
// in which the lines end in \
// - Unindented lists, in which wrapped lines are indented.
All three of these cases involve unindented lines next to indented
lines that would according to the usual rules begin a pre block.
Before this CL, they'd be reformatted to:
// Here is a greeting:
//
// func main() {
//
// fmt.Println("hello, world")
//
// }
// Run this command:
//
// path/to/your/program -flag1=longargument1 \
//
// -flag2=longargument2 \
// -flag3
// There are three possibilities:
//
// - Unindented code snippets (or JSON objects)
//
// in which the first and last line are unindented
// but end in { and start with }, respectively.
//
// - Unindented multiline shell commands
//
// in which the lines end in \
//
// - Unindented lists, in which wrapped lines are indented.
The fact that they are not already in canonical format gives us
a signal that they might not mean what the usual rules would say.
This CL takes advantage of that opening to apply a few heuristics
to better handle these cases:
1. If an indented code block immediately follows (without a blank line)
an unindented line ending in { or \, include the unindented line
in the code block.
2. If an indented code block immediately precedes (without a blank line)
an unindented line beginning with }, include the unindented line
in the code block.
3. If an indented line immediately follows (without a blank line)
an unindented line that starts with a list marker, assume this is
an unindented list with a wrapped indented line, and treat all
adjacent unindented lines starting with list markers as part of
the list, stopping at any surrounding blank lines.
This raises the fraction of “correctly” reformatted doc comments
in the corpus from approximately 87% to approximately 93%.
Change-Id: I7ac542eb085032d607a7caf3ba9020787b2978b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/410360
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org> Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>