This change doesn't modify any functionality.
It also doesn't update all of the comments and
variable names of the internal code, but everything
user facing should be correct.
Updates #49185
Change-Id: Ia8b2c94b89ba45897c4085ea0c17a3d8896f7ec7
Reviewed-on: https://go-review.googlesource.com/c/go/+/362794
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
// 'Go test' recompiles each package along with any files with names matching
// the file pattern "*_test.go".
// These additional files can contain test functions, benchmark functions, fuzz
-// targets and example functions. See 'go help testfunc' for more.
+// tests and example functions. See 'go help testfunc' for more.
// Each listed package causes the execution of a separate test binary.
// Files whose names begin with "_" (including "_test.go") or "." are ignored.
//
// Run each test, benchmark, and fuzz seed n times (default 1).
// If -cpu is set, run n times for each GOMAXPROCS value.
// Examples are always run once. -count does not apply to
-// fuzz targets matched by -fuzz.
+// fuzz tests matched by -fuzz.
//
// -cover
// Enable coverage analysis.
//
// -cpu 1,2,4
// Specify a list of GOMAXPROCS values for which the tests, benchmarks or
-// fuzz targets should be executed. The default is the current value
-// of GOMAXPROCS. -cpu does not apply to fuzz targets matched by -fuzz.
+// fuzz tests should be executed. The default is the current value
+// of GOMAXPROCS. -cpu does not apply to fuzz tests matched by -fuzz.
//
// -failfast
// Do not start new tests after the first test failure.
//
// -fuzz regexp
-// Run the fuzz target matching the regular expression. When specified,
+// Run the fuzz test matching the regular expression. When specified,
// the command line argument must match exactly one package within the
-// main module, and regexp must match exactly one fuzz target within
-// that package. After tests, benchmarks, seed corpora of other fuzz
-// targets, and examples have completed, the matching target will be
-// fuzzed. See the Fuzzing section of the testing package documentation
-// for details.
+// main module, and regexp must match exactly one fuzz test within
+// that package. Fuzzing will occur after tests, benchmarks, seed corpora
+// of other fuzz tests, and examples have completed. See the Fuzzing
+// section of the testing package documentation for details.
//
// -fuzztime t
// Run enough iterations of the fuzz test to take t, specified as a
// same information as the -v flag in a machine-readable format.
//
// -list regexp
-// List tests, benchmarks, fuzz targets, or examples matching the regular
-// expression. No tests, benchmarks, fuzz targets, or examples will be run.
+// List tests, benchmarks, fuzz tests, or examples matching the regular
+// expression. No tests, benchmarks, fuzz tests, or examples will be run.
// This will only list top-level tests. No subtest or subbenchmarks will be
// shown.
//
// -parallel n
// Allow parallel execution of test functions that call t.Parallel, and
-// f.Fuzz functions that call t.Parallel when running the seed corpus.
+// fuzz targets that call t.Parallel when running the seed corpus.
// The value of this flag is the maximum number of tests to run
// simultaneously.
// While fuzzing, the value of this flag is the maximum number of
// (see 'go help build').
//
// -run regexp
-// Run only those tests, examples, and fuzz targets matching the regular
+// Run only those tests, examples, and fuzz tests matching the regular
// expression. For tests, the regular expression is split by unbracketed
// slash (/) characters into a sequence of regular expressions, and each
// part of a test's identifier must match the corresponding element in
//
// func BenchmarkXxx(b *testing.B) { ... }
//
-// A fuzz target is one named FuzzXxx and should have the signature,
+// A fuzz test is one named FuzzXxx and should have the signature,
//
// func FuzzXxx(f *testing.F) { ... }
//
//
// The entire test file is presented as the example when it contains a single
// example function, at least one other function, type, variable, or constant
-// declaration, and no fuzz targets or test or benchmark functions.
+// declaration, and no tests, benchmarks, or fuzz tests.
//
// See the documentation of the testing package for more information.
//
'Go test' recompiles each package along with any files with names matching
the file pattern "*_test.go".
These additional files can contain test functions, benchmark functions, fuzz
-targets and example functions. See 'go help testfunc' for more.
+tests and example functions. See 'go help testfunc' for more.
Each listed package causes the execution of a separate test binary.
Files whose names begin with "_" (including "_test.go") or "." are ignored.
Run each test, benchmark, and fuzz seed n times (default 1).
If -cpu is set, run n times for each GOMAXPROCS value.
Examples are always run once. -count does not apply to
- fuzz targets matched by -fuzz.
+ fuzz tests matched by -fuzz.
-cover
Enable coverage analysis.
-cpu 1,2,4
Specify a list of GOMAXPROCS values for which the tests, benchmarks or
- fuzz targets should be executed. The default is the current value
- of GOMAXPROCS. -cpu does not apply to fuzz targets matched by -fuzz.
+ fuzz tests should be executed. The default is the current value
+ of GOMAXPROCS. -cpu does not apply to fuzz tests matched by -fuzz.
-failfast
Do not start new tests after the first test failure.
-fuzz regexp
- Run the fuzz target matching the regular expression. When specified,
+ Run the fuzz test matching the regular expression. When specified,
the command line argument must match exactly one package within the
- main module, and regexp must match exactly one fuzz target within
- that package. After tests, benchmarks, seed corpora of other fuzz
- targets, and examples have completed, the matching target will be
- fuzzed. See the Fuzzing section of the testing package documentation
- for details.
+ main module, and regexp must match exactly one fuzz test within
+ that package. Fuzzing will occur after tests, benchmarks, seed corpora
+ of other fuzz tests, and examples have completed. See the Fuzzing
+ section of the testing package documentation for details.
-fuzztime t
Run enough iterations of the fuzz test to take t, specified as a
same information as the -v flag in a machine-readable format.
-list regexp
- List tests, benchmarks, fuzz targets, or examples matching the regular
- expression. No tests, benchmarks, fuzz targets, or examples will be run.
+ List tests, benchmarks, fuzz tests, or examples matching the regular
+ expression. No tests, benchmarks, fuzz tests, or examples will be run.
This will only list top-level tests. No subtest or subbenchmarks will be
shown.
-parallel n
Allow parallel execution of test functions that call t.Parallel, and
- f.Fuzz functions that call t.Parallel when running the seed corpus.
+ fuzz targets that call t.Parallel when running the seed corpus.
The value of this flag is the maximum number of tests to run
simultaneously.
While fuzzing, the value of this flag is the maximum number of
(see 'go help build').
-run regexp
- Run only those tests, examples, and fuzz targets matching the regular
+ Run only those tests, examples, and fuzz tests matching the regular
expression. For tests, the regular expression is split by unbracketed
slash (/) characters into a sequence of regular expressions, and each
part of a test's identifier must match the corresponding element in
func BenchmarkXxx(b *testing.B) { ... }
-A fuzz target is one named FuzzXxx and should have the signature,
+A fuzz test is one named FuzzXxx and should have the signature,
func FuzzXxx(f *testing.F) { ... }
The entire test file is presented as the example when it contains a single
example function, at least one other function, type, variable, or constant
-declaration, and no fuzz targets or test or benchmark functions.
+declaration, and no tests, benchmarks, or fuzz tests.
See the documentation of the testing package for more information.
`,
}
var noTestsToRun = []byte("\ntesting: warning: no tests to run\n")
-var noTargetsToFuzz = []byte("\ntesting: warning: no targets to fuzz\n")
-var tooManyTargetsToFuzz = []byte("\ntesting: warning: -fuzz matches more than one target, won't fuzz\n")
+var noFuzzTestsToFuzz = []byte("\ntesting: warning: no fuzz tests to fuzz\n")
+var tooManyFuzzTestsToFuzz = []byte("\ntesting: warning: -fuzz matches more than one fuzz test, won't fuzz\n")
type runCache struct {
disableCache bool // cache should be disabled for this run
if bytes.HasPrefix(out, noTestsToRun[1:]) || bytes.Contains(out, noTestsToRun) {
norun = " [no tests to run]"
}
- if bytes.HasPrefix(out, noTargetsToFuzz[1:]) || bytes.Contains(out, noTargetsToFuzz) {
- norun = " [no targets to fuzz]"
+ if bytes.HasPrefix(out, noFuzzTestsToFuzz[1:]) || bytes.Contains(out, noFuzzTestsToFuzz) {
+ norun = " [no fuzz tests to fuzz]"
}
- if bytes.HasPrefix(out, tooManyTargetsToFuzz[1:]) || bytes.Contains(out, tooManyTargetsToFuzz) {
- norun = " [will not fuzz, -fuzz matches more than one target]"
+ if bytes.HasPrefix(out, tooManyFuzzTestsToFuzz[1:]) || bytes.Contains(out, tooManyFuzzTestsToFuzz) {
+ norun = "[-fuzz matches more than one fuzz test, won't fuzz]"
}
if len(out) > 0 && !bytes.HasSuffix(out, []byte("\n")) {
// Ensure that the output ends with a newline before the "ok"
! stdout ^ok
! stdout 'fatal here'
stdout FAIL
-stdout 'f.Fuzz function'
+stdout 'fuzz target'
# Test that f.Error within f.Fuzz panics
! go test error_fuzz_fn_fuzz_test.go
! stdout ^ok
! stdout 'error here'
stdout FAIL
-stdout 'f.Fuzz function'
+stdout 'fuzz target'
# Test that f.Fail within f.Fuzz panics
! go test fail_fuzz_fn_fuzz_test.go
! stdout ^ok
stdout FAIL
-stdout 'f.Fuzz function'
+stdout 'fuzz target'
# Test that f.Skip within f.Fuzz panics
! go test skip_fuzz_fn_fuzz_test.go
! stdout ^ok
! stdout 'skip here'
stdout FAIL
-stdout 'f.Fuzz function'
+stdout 'fuzz target'
# Test that f.Skipped within f.Fuzz panics
! go test skipped_fuzz_fn_fuzz_test.go
! stdout ^ok
! stdout 'f.Skipped is'
stdout FAIL
-stdout 'f.Fuzz function'
+stdout 'fuzz target'
stdout 't.Skipped is false'
# Test that runtime.Goexit within the fuzz function is an error.
go test -fuzz ThisWillNotMatch -fuzztime 1x standalone_fuzz_test.go
! stdout '^ok.*no tests to run'
stdout '^ok'
-stdout 'no targets to fuzz'
+stdout 'no fuzz tests to fuzz'
[short] stop
# Matches no fuzz targets.
go test -run ThisWillNotMatch standalone_fuzz_test.go
stdout '^ok.*no tests to run'
-! stdout 'no targets to fuzz'
+! stdout 'no fuzz tests to fuzz'
-- standalone_fuzz_test.go --
package standalone_fuzz
# With fuzzing enabled, at most one target in the same package may match.
! go test -fuzz=. ./two
-stdout '^testing: will not fuzz, -fuzz matches more than one target: \[FuzzOne FuzzTwo\]$'
+stdout '^testing: will not fuzz, -fuzz matches more than one fuzz test: \[FuzzOne FuzzTwo\]$'
go test -fuzz=FuzzTwo -fuzztime=1x ./two
-- go.mod --
! exists want
! go test -fuzz=. -parallel=1 -fuzztime=110x -fuzzminimizetime=10x -v
stdout 'fuzzing process terminated unexpectedly'
-stdout 'Crash written to testdata'
+stdout 'Failing input written to testdata'
# Run the fuzz target without fuzzing. The fuzz function is called with the
# crashing input in testdata. The test passes if that input is identical to
# and doesn't write anything to testdata/fuzz
! go test -fuzz=FuzzWithAdd -run=FuzzWithAdd -fuzztime=1x
! stdout ^ok
-! stdout 'Crash written to testdata[/\\]fuzz[/\\]FuzzWithAdd[/\\]'
+! stdout 'Failing input written to testdata[/\\]fuzz[/\\]FuzzWithAdd[/\\]'
stdout FAIL
# Test that fuzzing a target with a sucess in f.Add and a fuzztime of only
# and doesn't write anything to testdata/fuzz
! go test -fuzz=FuzzWithTestdata -run=FuzzWithTestdata -fuzztime=1x
! stdout ^ok
-! stdout 'Crash written to testdata[/\\]fuzz[/\\]FuzzWithTestdata[/\\]'
-stdout 'found a crash while testing seed corpus entry: FuzzWithTestdata/1'
+! stdout 'Failing input written to testdata[/\\]fuzz[/\\]FuzzWithTestdata[/\\]'
+stdout 'failure while testing seed corpus entry: FuzzWithTestdata/1'
stdout FAIL
# Test that fuzzing a target with no seed corpus or cache finds a crash, prints
# it, and write it to testdata
! go test -fuzz=FuzzWithNoCache -run=FuzzWithNoCache -fuzztime=1x
! stdout ^ok
-stdout 'Crash written to testdata[/\\]fuzz[/\\]FuzzWithNoCache[/\\]'
+stdout 'Failing input written to testdata[/\\]fuzz[/\\]FuzzWithNoCache[/\\]'
stdout FAIL
# Write a crashing input to the cache
# and writes this as a "new" crash to testdata/fuzz
! go test -fuzz=FuzzWithCache -run=FuzzWithCache -fuzztime=1x
! stdout ^ok
-stdout 'Crash written to testdata[/\\]fuzz[/\\]FuzzWithCache[/\\]'
+stdout 'Failing input written to testdata[/\\]fuzz[/\\]FuzzWithCache[/\\]'
stdout FAIL
# Write a crashing input to the cache
stdout 'gathering baseline coverage'
stdout 'got the minimum size!'
stdout 'contains a non-zero byte of length 10'
-stdout 'Crash written to testdata[/\\]fuzz[/\\]FuzzWithMinimizableCache[/\\]'
+stdout 'Failing input written to testdata[/\\]fuzz[/\\]FuzzWithMinimizableCache[/\\]'
stdout FAIL
# Make sure this crash didn't come from fuzzing
# (the log line that states fuzzing began shouldn't have printed)
# the crash and doesn't write anything to testdata/fuzz -fuzztime=1x
! go test -fuzz=FuzzWithAdd -run=None
! stdout ^ok
-! stdout 'Crash written to testdata[/\\]fuzz[/\\]FuzzWithAdd[/\\]'
+! stdout 'Failing input written to testdata[/\\]fuzz[/\\]FuzzWithAdd[/\\]'
stdout FAIL
# Test that fuzzing a target (with -run=None set) with a sucess in f.Add and a
# testdata/fuzz prints the crash and doesn't write anything to testdata/fuzz
! go test -fuzz=FuzzWithTestdata -run=None -fuzztime=1x
! stdout ^ok
-! stdout 'Crash written to testdata[/\\]fuzz[/\\]FuzzWithTestdata[/\\]'
+! stdout 'Failing input written to testdata[/\\]fuzz[/\\]FuzzWithTestdata[/\\]'
stdout FAIL
# Write a crashing input to the cache
# prints the crash and writes this as a "new" crash to testdata/fuzz
! go test -fuzz=FuzzRunNoneWithCache -run=None -fuzztime=1x
! stdout ^ok
-stdout 'Crash written to testdata[/\\]fuzz[/\\]FuzzRunNoneWithCache[/\\]'
+stdout 'Failing input written to testdata[/\\]fuzz[/\\]FuzzRunNoneWithCache[/\\]'
stdout FAIL
# Clear the fuzz cache and make sure it's gone
go test -c
! exec ./x.test$GOEXE -test.fuzz=FuzzWithAdd -test.run=FuzzWithAdd -test.fuzztime=1x -test.fuzzcachedir=$WORK/cache
! stdout ^ok
-! stdout 'Crash written to testdata[/\\]fuzz[/\\]FuzzWithAdd[/\\]'
+! stdout 'Failing input written to testdata[/\\]fuzz[/\\]FuzzWithAdd[/\\]'
stdout FAIL
stderr warning
go test -c
! exec ./x.test$GOEXE -test.fuzz=FuzzWithTestdata -test.run=FuzzWithTestdata -test.fuzztime=1x -test.fuzzcachedir=$WORK/cache
! stdout ^ok
-! stdout 'Crash written to testdata[/\\]fuzz[/\\]FuzzWithTestdata[/\\]'
+! stdout 'Failing input written to testdata[/\\]fuzz[/\\]FuzzWithTestdata[/\\]'
stdout FAIL
stderr warning
// identifiers from other packages (or predeclared identifiers, such as
// "int") and the test file does not include a dot import.
// - The entire test file is the example: the file contains exactly one
-// example function, zero test, fuzz target, or benchmark function, and at
+// example function, zero test, fuzz test, or benchmark function, and at
// least one top-level function, type, variable, or constant declaration
// other than the example function.
func Examples(testFiles ...*ast.File) []*Example {
var list []*Example
for _, file := range testFiles {
- hasTests := false // file contains tests, fuzz targets, or benchmarks
+ hasTests := false // file contains tests, fuzz test, or benchmarks
numDecl := 0 // number of non-import declarations in the file
var flist []*Example
for _, decl := range file.Decls {
return "", false, false // no suitable comment found
}
-// isTest tells whether name looks like a test, example, fuzz target, or
+// isTest tells whether name looks like a test, example, fuzz test, or
// benchmark. It is a Test (say) if there is a character after Test that is not
// a lower-case letter. (We don't want Testiness.)
func isTest(name, prefix string) bool {
if result.crasherMsg != "" {
if c.warmupRun() && result.entry.IsSeed {
target := filepath.Base(c.opts.CorpusDir)
- fmt.Fprintf(c.opts.Log, "found a crash while testing seed corpus entry: %s/%s\n", target, testName(result.entry.Parent))
+ fmt.Fprintf(c.opts.Log, "failure while testing seed corpus entry: %s/%s\n", target, testName(result.entry.Parent))
stop(errors.New(result.crasherMsg))
break
}
// Send it back to a worker for minimization. Disable inputC so
// other workers don't continue fuzzing.
c.crashMinimizing = &result
- fmt.Fprintf(c.opts.Log, "fuzz: minimizing %d-byte crash file\n", len(result.entry.Data))
+ fmt.Fprintf(c.opts.Log, "fuzz: minimizing %d-byte failing input file\n", len(result.entry.Data))
c.queueForMinimization(result, nil)
} else if !crashWritten {
// Found a crasher that's either minimized or not minimizable.
)
func initFuzzFlags() {
- matchFuzz = flag.String("test.fuzz", "", "run the fuzz target matching `regexp`")
+ matchFuzz = flag.String("test.fuzz", "", "run the fuzz test matching `regexp`")
flag.Var(&fuzzDuration, "test.fuzztime", "time to spend fuzzing; default is to run indefinitely")
- flag.Var(&minimizeDuration, "test.fuzzminimizetime", "time to spend minimizing a value after finding a crash")
+ flag.Var(&minimizeDuration, "test.fuzzminimizetime", "time to spend minimizing a value after finding a failing input")
fuzzCacheDir = flag.String("test.fuzzcachedir", "", "directory where interesting fuzzing inputs are stored")
isFuzzWorker = flag.Bool("test.fuzzworker", false, "coordinate with the parent process to fuzz random values")
}
fuzzCacheDir *string
isFuzzWorker *bool
- // corpusDir is the parent directory of the target's seed corpus within
+ // corpusDir is the parent directory of the fuzz test's seed corpus within
// the package.
corpusDir = "testdata/fuzz"
)
-// fuzzWorkerExitCode is used as an exit code by fuzz worker processes after an internal error.
-// This distinguishes internal errors from uncontrolled panics and other crashes.
-// Keep in sync with internal/fuzz.workerExitCode.
+// fuzzWorkerExitCode is used as an exit code by fuzz worker processes after an
+// internal error. This distinguishes internal errors from uncontrolled panics
+// and other failiures. Keep in sync with internal/fuzz.workerExitCode.
const fuzzWorkerExitCode = 70
-// InternalFuzzTarget is an internal type but exported because it is cross-package;
-// it is part of the implementation of the "go test" command.
+// InternalFuzzTarget is an internal type but exported because it is
+// cross-package; it is part of the implementation of the "go test" command.
type InternalFuzzTarget struct {
Name string
Fn func(f *F)
}
-// F is a type passed to fuzz targets.
+// F is a type passed to fuzz tests.
//
-// A fuzz target may add seed corpus entries using F.Add or by storing files in
-// the testdata/fuzz/<FuzzTargetName> directory. The fuzz target must then
-// call F.Fuzz once to provide a fuzz function. See the testing package
-// documentation for an example, and see the F.Fuzz and F.Add method
-// documentation for details.
+// Fuzz tests run generated inputs against a provided fuzz target, which can
+// find and report potential bugs in the code being tested.
//
-// *F methods can only be called before (*F).Fuzz. Once inside the function
-// passed to (*F).Fuzz, only (*T) methods can be used. The only *F methods that
-// are allowed in the (*F).Fuzz function are (*F).Failed and (*F).Name.
+// A fuzz test runs the seed corpus by default, which includes entries provided
+// by (*F).Add and entries in the testdata/fuzz/<FuzzTestName> directory. After
+// any necessary setup and calls to (*F).Add, the fuzz test must then call
+// (*F).Fuzz to provide the fuzz target. See the testing package documentation
+// for an example, and see the F.Fuzz and F.Add method documentation for
+// details.
+//
+// *F methods can only be called before (*F).Fuzz. Once the the test is
+// executing the fuzz target, only (*T) methods can be used. The only *F methods
+// that are allowed in the (*F).Fuzz function are (*F).Failed and (*F).Name.
type F struct {
common
fuzzContext *fuzzContext
// Helper may be called simultaneously from multiple goroutines.
func (f *F) Helper() {
if f.inFuzzFn {
- panic("testing: f.Helper was called inside the f.Fuzz function, use t.Helper instead")
+ panic("testing: f.Helper was called inside the fuzz target, use t.Helper instead")
}
// common.Helper is inlined here.
// (*F).Fail may be called by (*T).Fail, which we should allow. However, we
// shouldn't allow direct (*F).Fail calls from inside the (*F).Fuzz function.
if f.inFuzzFn {
- panic("testing: f.Fail was called inside the f.Fuzz function, use t.Fail instead")
+ panic("testing: f.Fail was called inside the fuzz target, use t.Fail instead")
}
f.common.Helper()
f.common.Fail()
// (*F).Skipped may be called by tRunner, which we should allow. However, we
// shouldn't allow direct (*F).Skipped calls from inside the (*F).Fuzz function.
if f.inFuzzFn {
- panic("testing: f.Skipped was called inside the f.Fuzz function, use t.Skipped instead")
+ panic("testing: f.Skipped was called inside the fuzz target, use t.Skipped instead")
}
f.common.Helper()
return f.common.Skipped()
}
-// Add will add the arguments to the seed corpus for the fuzz target. This will
-// be a no-op if called after or within the Fuzz function. The args must match
-// those in the Fuzz function.
+// Add will add the arguments to the seed corpus for the fuzz test. This will be
+// a no-op if called after or within the fuzz target, and args must match the
+// arguments for the fuzz target.
func (f *F) Add(args ...interface{}) {
var values []interface{}
for i := range args {
panic("testing: F.Fuzz must receive a function")
}
if fnType.NumIn() < 2 || fnType.In(0) != reflect.TypeOf((*T)(nil)) {
- panic("testing: F.Fuzz function must receive at least two arguments, where the first argument is a *T")
+ panic("testing: fuzz target must receive at least two arguments, where the first argument is a *T")
}
// Save the types of the function to compare against the corpus.
fmt.Fprintf(f.w, "%v\n", err)
if crashErr, ok := err.(fuzzCrashError); ok {
crashPath := crashErr.CrashPath()
- fmt.Fprintf(f.w, "Crash written to %s\n", crashPath)
+ fmt.Fprintf(f.w, "Failing input written to %s\n", crashPath)
testName := filepath.Base(crashPath)
fmt.Fprintf(f.w, "To re-run:\ngo test -run=%s/%s\n", f.name, testName)
}
}); err != nil {
// Internal errors are marked with f.Fail; user code may call this too, before F.Fuzz.
// The worker will exit with fuzzWorkerExitCode, indicating this is a failure
- // (and 'go test' should exit non-zero) but a crasher should not be recorded.
+ // (and 'go test' should exit non-zero) but a failing input should not be recorded.
f.Errorf("communicating with fuzzing coordinator: %v", err)
}
type fuzzResult struct {
N int // The number of iterations.
T time.Duration // The total time taken.
- Error error // Error is the error from the crash
+ Error error // Error is the error from the failing input
}
func (r fuzzResult) String() string {
return s
}
-// fuzzCrashError is satisfied by a crash detected within the fuzz function.
+// fuzzCrashError is satisfied by a failing input detected while fuzzing.
// These errors are written to the seed corpus and can be re-run with 'go test'.
// Errors within the fuzzing framework (like I/O errors between coordinator
// and worker processes) don't satisfy this interface.
// CrashPath returns the path of the subtest that corresponds to the saved
// crash input file in the seed corpus. The test can be re-run with go test
- // -run=$target/$name $target is the fuzz target name, and $name is the
+ // -run=$test/$name $test is the fuzz test name, and $name is the
// filepath.Base of the string returned here.
CrashPath() string
}
-// fuzzContext holds fields common to all fuzz targets.
+// fuzzContext holds fields common to all fuzz tests.
type fuzzContext struct {
deps testDeps
mode fuzzMode
fuzzWorker
)
-// runFuzzTargets runs the fuzz targets matching the pattern for -run. This will
-// only run the f.Fuzz function for each seed corpus without using the fuzzing
-// engine to generate or mutate inputs.
-func runFuzzTargets(deps testDeps, fuzzTargets []InternalFuzzTarget, deadline time.Time) (ran, ok bool) {
+// runFuzzTests runs the fuzz tests matching the pattern for -run. This will
+// only run the (*F).Fuzz function for each seed corpus without using the
+// fuzzing engine to generate or mutate inputs.
+func runFuzzTests(deps testDeps, fuzzTests []InternalFuzzTarget, deadline time.Time) (ran, ok bool) {
ok = true
- if len(fuzzTargets) == 0 || *isFuzzWorker {
+ if len(fuzzTests) == 0 || *isFuzzWorker {
return ran, ok
}
m := newMatcher(deps.MatchString, *match, "-test.run")
if Verbose() {
root.chatty = newChattyPrinter(root.w)
}
- for _, ft := range fuzzTargets {
+ for _, ft := range fuzzTests {
if shouldFailFast() {
break
}
}
if mFuzz != nil {
if _, fuzzMatched, _ := mFuzz.fullName(nil, ft.Name); fuzzMatched {
- // If this target will be fuzzed, then don't run the seed corpus
+ // If this will be fuzzed, then don't run the seed corpus
// right now. That will happen later.
continue
}
return root.ran, !root.Failed()
}
-// runFuzzing runs the fuzz target matching the pattern for -fuzz. Only one such
-// fuzz target must match. This will run the fuzzing engine to generate and
-// mutate new inputs against the f.Fuzz function.
+// runFuzzing runs the fuzz test matching the pattern for -fuzz. Only one such
+// fuzz test must match. This will run the fuzzing engine to generate and
+// mutate new inputs against the fuzz target.
//
// If fuzzing is disabled (-test.fuzz is not set), runFuzzing
// returns immediately.
-func runFuzzing(deps testDeps, fuzzTargets []InternalFuzzTarget) (ok bool) {
- // TODO(katiehockman,jayconrod): Should we do something special to make sure
- // we don't print f.Log statements again with runFuzzing, since we already
- // would have printed them when we ran runFuzzTargets (ie. seed corpus run)?
- if len(fuzzTargets) == 0 || *matchFuzz == "" {
+func runFuzzing(deps testDeps, fuzzTests []InternalFuzzTarget) (ok bool) {
+ if len(fuzzTests) == 0 || *matchFuzz == "" {
return true
}
m := newMatcher(deps.MatchString, *matchFuzz, "-test.fuzz")
if Verbose() && !*isFuzzWorker {
root.chatty = newChattyPrinter(root.w)
}
- var target *InternalFuzzTarget
- var targetName string
+ var fuzzTest *InternalFuzzTarget
+ var testName string
var matched []string
- for i := range fuzzTargets {
- name, ok, _ := tctx.match.fullName(nil, fuzzTargets[i].Name)
+ for i := range fuzzTests {
+ name, ok, _ := tctx.match.fullName(nil, fuzzTests[i].Name)
if !ok {
continue
}
matched = append(matched, name)
- target = &fuzzTargets[i]
- targetName = name
+ fuzzTest = &fuzzTests[i]
+ testName = name
}
if len(matched) == 0 {
- fmt.Fprintln(os.Stderr, "testing: warning: no targets to fuzz")
+ fmt.Fprintln(os.Stderr, "testing: warning: no fuzz tests to fuzz")
return true
}
if len(matched) > 1 {
- fmt.Fprintf(os.Stderr, "testing: will not fuzz, -fuzz matches more than one target: %v\n", matched)
+ fmt.Fprintf(os.Stderr, "testing: will not fuzz, -fuzz matches more than one fuzz test: %v\n", matched)
return false
}
common: common{
signal: make(chan bool),
barrier: nil, // T.Parallel has no effect when fuzzing.
- name: targetName,
+ name: testName,
parent: &root,
level: root.level + 1,
chatty: root.chatty,
// TODO(#48132): adjust this to work with test2json.
f.chatty.Updatef(f.name, "=== FUZZ %s\n", f.name)
}
- go fRunner(f, target.Fn)
+ go fRunner(f, fuzzTest.Fn)
<-f.signal
return !f.failed
}
-// fRunner wraps a call to a fuzz target and ensures that cleanup functions are
+// fRunner wraps a call to a fuzz test and ensures that cleanup functions are
// called and status flags are set. fRunner should be called in its own
// goroutine. To wait for its completion, receive from f.signal.
//
// fRunner is analogous to tRunner, which wraps subtests started with T.Run.
-// Tests and fuzz targets work a little differently, so for now, these functions
-// aren't consolidated. In particular, because there are no F.Run and F.Parallel
-// methods, i.e., no fuzz sub-targets or parallel fuzz targets, a few
+// Unit tests and fuzz tests work a little differently, so for now, these
+// functions aren't consolidated. In particular, because there are no F.Run and
+// F.Parallel methods, i.e., no fuzz sub-tests or parallel fuzz tests, a few
// simplifications are made. We also require that F.Fuzz, F.Skip, or F.Fail is
// called.
func fRunner(f *F, fn func(*F)) {
- // When this goroutine is done, either because runtime.Goexit was called,
- // a panic started, or fn returned normally, record the duration and send
- // t.signal, indicating the fuzz target is done.
+ // When this goroutine is done, either because runtime.Goexit was called, a
+ // panic started, or fn returned normally, record the duration and send
+ // t.signal, indicating the fuzz test is done.
defer func() {
- // Detect whether the fuzz target panicked or called runtime.Goexit without
- // calling F.Fuzz, F.Fail, or F.Skip. If it did, panic (possibly replacing a
- // nil panic value). Nothing should recover after fRunner unwinds, so this
- // should crash the process and print stack. Unfortunately, recovering here
- // adds stack frames, but the location of the original panic should still be
+ // Detect whether the fuzz test panicked or called runtime.Goexit
+ // without calling F.Fuzz, F.Fail, or F.Skip. If it did, panic (possibly
+ // replacing a nil panic value). Nothing should recover after fRunner
+ // unwinds, so this should crash the process and print stack.
+ // Unfortunately, recovering here adds stack frames, but the location of
+ // the original panic should still be
// clear.
if f.Failed() {
atomic.AddUint32(&numFailed, 1)
if len(f.sub) > 0 {
// Unblock inputs that called T.Parallel while running the seed corpus.
- // This only affects fuzz targets run as normal tests.
+ // This only affects fuzz tests run as normal tests.
// While fuzzing, T.Parallel has no effect, so f.sub is empty, and this
// branch is not taken. f.barrier is nil in that case.
close(f.barrier)
// a function is called with randomly generated inputs to find bugs not
// anticipated by unit tests.
//
-// A fuzz target is a function that declares a set of "seed" inputs by calling
-// F.Add, then provides a fuzz function by calling F.Fuzz. A fuzz target has
-// the form:
-//
+// Functions of the form
// func FuzzXxx(*testing.F)
+// are considered fuzz tests.
//
// For example:
//
// })
// }
//
-// Seed inputs may be registered by calling F.Add or by storing files in the
-// directory testdata/fuzz/<Name> (where <Name> is the name of the fuzz target)
-// within the package containing the fuzz target. Seed inputs are optional, but
-// the fuzzing engine may find bugs more efficiently when provided with a set
-// of small seed inputs with good code coverage.
-//
-// The fuzz function provided to F.Fuzz must accept a *testing.T parameter,
-// followed by one or more parameters for random inputs. The types of arguments
-// passed to F.Add must be identical to the types of these parameters. The fuzz
-// function may signal that it's found a problem the same way tests do: by
-// calling T.Fail (or any method that calls it like T.Error or T.Fatal) or by
-// panicking.
+// A fuzz test maintains a seed corpus, or a set of inputs which are run by
+// default, and can seed input generation. Seed inputs may be registered by
+// calling (*F).Add or by storing files in the directory testdata/fuzz/<Name>
+// (where <Name> is the name of the fuzz test) within the package containing
+// the fuzz test. Seed inputs are optional, but the fuzzing engine may find
+// bugs more efficiently when provided with a set of small seed inputs with good
+// code coverage. These seed inputs can also serve as regression tests for bugs
+// identified through fuzzing.
+//
+// The function passed to (*F).Fuzz within the fuzz test is considered the fuzz
+// target. A fuzz target must accept a *T parameter, followed by one or more
+// parameters for random inputs. The types of arguments passed to (*F).Add must
+// be identical to the types of these parameters. The fuzz target may signal
+// that it's found a problem the same way tests do: by calling T.Fail (or any
+// method that calls it like T.Error or T.Fatal) or by panicking.
//
// When fuzzing is enabled (by setting the -fuzz flag to a regular expression
-// that matches a specific fuzz target), the fuzz function is called with
-// arguments generated by repeatedly making random changes to the seed inputs.
-// On supported platforms, 'go test' compiles the test executable with fuzzing
+// that matches a specific fuzz test), the fuzz target is called with arguments
+// generated by repeatedly making random changes to the seed inputs. On
+// supported platforms, 'go test' compiles the test executable with fuzzing
// coverage instrumentation. The fuzzing engine uses that instrumentation to
-// find and cache inputs that expand coverage, increasing the liklihood of
-// finding bugs. If the fuzz function finds a problem, the fuzzing engine writes
-// the inputs that caused the problem to a file in the directory
+// find and cache inputs that expand coverage, increasing the likelihood of
+// finding bugs. If the fuzz target fails for a given input, the fuzzing engine
+// writes the inputs that caused the failure to a file in the directory
// testdata/fuzz/<Name> within the package directory. This file later serves as
// a seed input. If the file can't be written at that location (for example,
// because the directory is read-only), the fuzzing engine writes the file to
// the fuzz cache directory within the build cache instead.
//
-// When fuzzing is disabled, the fuzz function is called with the seed inputs
+// When fuzzing is disabled, the fuzz target is called with the seed inputs
// registered with F.Add and seed inputs from testdata/fuzz/<Name>. In this
-// mode, the fuzz target acts much like a regular test, with subtests started
+// mode, the fuzz test acts much like a regular test, with subtests started
// with F.Fuzz instead of T.Run.
//
// TODO(#48255): write and link to documentation that will be helpful to users
// }
//
// The Skip method of *T can be used in a fuzz target if the input is invalid,
-// but should not be considered a crash. For example:
+// but should not be considered a failing input. For example:
//
// func FuzzJSONMarshalling(f *testing.F) {
// f.Fuzz(func(t *testing.T, b []byte) {
cleanupName string // Name of the cleanup function.
cleanupPc []uintptr // The stack trace at the point where Cleanup was called.
finished bool // Test function has completed.
- inFuzzFn bool // Whether the fuzz function, if this is one, is running.
+ inFuzzFn bool // Whether the fuzz target, if this is one, is running.
chatty *chattyPrinter // A copy of chattyPrinter, if the chatty flag is set.
bench bool // Whether the current test is a benchmark.
func (c *common) checkFuzzFn(name string) {
if c.inFuzzFn {
- panic(fmt.Sprintf("testing: f.%s was called inside the f.Fuzz function, use t.%s instead", name, name))
+ panic(fmt.Sprintf("testing: f.%s was called inside the fuzz target, use t.%s instead", name, name))
}
}
deadline := m.startAlarm()
haveExamples = len(m.examples) > 0
testRan, testOk := runTests(m.deps.MatchString, m.tests, deadline)
- fuzzTargetsRan, fuzzTargetsOk := runFuzzTargets(m.deps, m.fuzzTargets, deadline)
+ fuzzTargetsRan, fuzzTargetsOk := runFuzzTests(m.deps, m.fuzzTargets, deadline)
exampleRan, exampleOk := runExamples(m.deps.MatchString, m.examples)
m.stopAlarm()
if !testRan && !exampleRan && !fuzzTargetsRan && *matchBenchmarks == "" && *matchFuzz == "" {