tester

package
v0.10.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 23, 2025 License: MIT Imports: 9 Imported by: 3

README

Tester Package

If you’ve spent any time writing Go tests, you’ve probably encountered the joy of *testing.T. It’s the backbone of Go’s testing framework — powerful, flexible, and ubiquitous. But as your test suite grows, you might find yourself repeating the same chunks of test logic across multiple test cases. Enter test helpers: reusable functions that streamline your tests, improve readability, and reduce complexity. Libraries like assert are prime examples, turning verbose checks into concise assertions.

But here’s the catch: how do you test the test helpers themselves? After all, these are the tools you rely on to ensure your code works as expected. If they fail, your tests might silently lie to you. This is where the tester package comes to the rescue.

Package tester provides interface T which is a subset of testing.TB interface and Spy struct which helps with testing test helpers.

Test Manager Interface

The goal of T interface is to make testing of test helpers possible. We define test helper as code which uses *testing.T instances as test manager. By design T is a subset of the testing.TB interface to allow using implementers as well as *testing.T as a test helper argument.

Creating test helpers is part of making tests more readable. Instead of repeating big blocks of code in many test cases, we can create a helper and delegate part of testing procedures to it. A test helper usually receives some kind of test manager instance (usually *testing.T) as an argument, so it can log and provide the test outcome to the test runner.

Good example of test helpers is assertion functions in assert package, which improve test readability and in many cases reduce their complexity.

Usage

Anywhere where *testing.T is used you can replace it with tester.T interface as long as the test helper uses the following methods:

  • Spy.Cleanup(func())
  • Spy.Error(args ...any)
  • Spy.Errorf(format string, args ...any)
  • Spy.Fatal(args ...any)
  • Spy.Fatalf(format string, args ...any)
  • Spy.FailNow()
  • Spy.Failed() bool
  • Spy.Helper()
  • Spy.Log(args ...any)
  • Spy.Logf(format string, args ...any)
  • Spy.Name() string
  • Spy.Setenv(key, value string)
  • Spy.Skip(args ...any)
  • Spy.TempDir() string
  • Spy.Context() context.Context

So for example, a test helper:

// IsOdd asserts "have" is an odd number. Returns true if it is, otherwise marks
// the test as failed, writes an error message to the test log and returns false.
func IsOdd(t *testing.T, have int) bool {
	t.Helper()
	if have%2 != 0 {
		t.Errorf("expected %d to be odd", have)
		return false
	}
	return true
}

Can be refactored without any change to the body of the function as follows:

// IsOdd asserts "have" is an odd number. Returns true if it is, otherwise marks
// the test as failed, writes an error message to the test log and returns false.
func IsOdd(t tester.T, have int) bool {
	t.Helper()
	if have%2 != 0 {
		t.Errorf("expected %d to be odd", have)
		return false
	}
	return true
}

Once you replace *testing.T with implementer of tester.T (for example Spy instance) you can create tests for the helper.

Spy

The Spy type was designed to be a spy for tester.TB interface. The spy allows you to define expectations how the test manager instance is used by a test helper.

Testing Test Helpers

We can test the IsOdd test helper created above in the following way:

func Test_IsOdd(t *testing.T) {
    t.Run("error - is not odd number", func(t *testing.T) {
        // --- Given ---

		// Set up the spy with expectations
		tspy := tester.New(t)
		tspy.ExpectError()                              // Expect an error.
		tspy.ExpectLogEqual("expected %d to be odd", 2) // Expect log.
		tspy.Close()                                    // No more expectations.

		// --- When ---
		success := IsOdd(tspy, 2) // Run the helper.

		// --- Then ---
		if success { // Verify the outcome.
			t.Error("expected success to be false")
		}
		tspy.AssertExpectations() // Ensure all expectations were met.
	})

	t.Run("is odd number", func(t *testing.T) {
		// Given
		tspy := tester.New(t)
		tspy.Close()

		// When
		success := IsOdd(tspy, 3)

		// Then
		if !success {
			t.Error("expected success to be true")
		}

		// The `tspy.AssertExpectations()` is called automatically.
	})
}

Setting Spy Expectations

To set expectations for the Helper Under Test (HUT) Spy instance provides multiple Expect* methods.

tspy := tester.New(t)

tspy.ExpectCleanups(n)  // Expect HUT to call Cleanup exactly n times. 
tspy.ExpectError()      // Expect HUT to call one of the Error* methods at least once. 
tspy.ExpectFatal()      // Expect HUT to call one of the Fatal* methods at least once.
tspy.ExpectFail()       // Expect HUT to call one of the Error* or Fatal* at least once.  
tspy.ExpectHelpers(n)   // Expect HUT to call Helper method exactly n times. 
tspy.ExpectSetenv(k, v) // Expect HUT to call Setenv method with the key, value pair.
tspy.ExpectSkipped()    // Expect HUT to skip the test.
tspy.ExpectTempDir(n)   // Expect HUT to call TempDir n times.
tspy.ExpectFail()       // Expect HUT to call one of the Error* or Fatal* methods.
tspy.ExpectedNames(n)   // Expect HUT to call Name exactly n times.

// Log message expectations: 

tspy.ExpectLog(matcher, format, args...)  // Expect the logged message to match the formated string.
tspy.ExpectLogEqual(format, args...)      // Expect the logged message to equal to the formated string. 
tspy.ExpectLogContain(format, args...)    // Expect the logged message to contain the formated string.
tspy.ExpectLogNotContain(format, args...) // Expect the logged message not to contain the formated string.

Since each of the methods has great documentation, we encourage you to explore it for more details. Here we will just document some of the cases which might not be so obvious at the first glance.

Expectations For Helper

By default, when you instantiate Spy

tspy := tester.New(t)

it will expect at least one call to Helper method, but you can define exact number of times it should be called by adding optional argument

tspy := tester.New(t)

Now if the HUT does not make exactly 2 calls to Helper it will fail the test.

Executing Cleanup Functions

Execute Spy.Finish() to run all registered cleanups.

func Test_Spy_Cleanups(t *testing.T) {
	// --- Given ---
	tspy := tester.New(t, 0)
	tspy.ExpectCleanups(1)
	tspy.Close()

	// --- When ---
	var have int
	tspy.Cleanup(func() { have = 42 })

	// --- Then ---
	tspy.Finish()
	if have != 42 {
		t.Errorf("expected 42 got %d", have)
	}
}
Checking Spy State

At any point in time you may call Spy.Failed() to check if the HUT called any of the Error*, Fatal* or FailNow methods.

Get TempDir Paths

To get paths generated by Spy.TempDir use Spy.GetTempDir(idx) where idx is an index into the array of generated paths (zero indexed).

Examine Log Messages

Calling methods like Spy.Error* not only change the state of the test being executed but also log messages (usually to standard output). For example:

=== RUN   Test_relativeTo
=== RUN   Test_relativeTo/current_package
    helpers_test.go:27: expected values to be equal:
        	want: "case"
        	have: "cases"
=== RUN   Test_relativeTo/not_current_package
=== RUN   Test_relativeTo/nil_package
--- FAIL: Test_relativeTo (0.28s)
    --- FAIL: Test_relativeTo/current_package (0.00s)

    --- PASS: Test_relativeTo/not_current_package (0.00s)
    --- PASS: Test_relativeTo/nil_package (0.00s)

FAIL

The Spy provides a couple of ways to examine log messages. The ExpectLog(matcher MStrategy, format string, args ...any) where you provide matching strategy:

  • tester.Equal - the log must be exact match with given formatted string
  • tester.Contains - the log must contain given formatted string
  • tester.NotContains - the log must NOT contain given formatted string
  • tester.Regexp - the log must match regexp

Or using convenience methods:

  • ExpectLogEqual
  • ExpectLogContain
  • ExpectLogNotContain

which call ExpectLog with given matching strategy.

By default, if the HUT logged anything, and it was not examined the test will be failed. To change this behaviour use Spy.IgnoreLogs method.

Ignore Log Messages
func Test_Spy_IgnoreLogExamination(t *testing.T) {
	// --- Given ---
	tspy := tester.New(t, 0)
	tspy.ExpectError()
	// Without this line Spy will report an error 
	// that it did not expect the HUT to log.
	tspy.IgnoreLogs() 
	tspy.Close()

	// --- When ---
	tspy.Error("message")

	// --- Then ---
	tspy.AssertExpectations()
}

Examples

See tester.go and tester.go for more examples.

Documentation

Overview

Package tester provides structures to help with testing custom assertions and test helpers.

Index

Constants

View Source 
const FailNowMsg = "FailNow was called directly"

FailNowMsg represents a message the Spy.FailNow method uses in panic.

Variables

This section is empty.

Functions

This section is empty.

Types

type Spy 

type Spy struct {
	// contains filtered or unexported fields
}

Spy is a spy for tester.T interface.

Creating test helpers is an integral part of comprehensive testing, but those helpers in turn also need to be tested to make sure assertions made by them are implemented correctly. The Spy is a tool that makes testing such helpers very easy.

Pass an instance of Spy to the Helper Under Test (HUT) and assert the expected behavior using Spy.Expect* methods.

func New 

func New(tt *testing.T, expectHelpers ...int) *Spy

New returns new instance of Spy which implements T interface. The tt argument is used to proxy calls to testing.T.TempDir, testing.T.Setenv and testing.T.Context as well as to report errors when the Spy expectations are not met by Helper Under Test (HUT). The constructor function adds a cleanup function to tt which calls Spy.Finish and Spy.AssertExpectations methods to determine if the tt should be failed.

The call to New should be followed by zero or more calls to Spy.Expect* methods and finished with call to Spy.Close method: 

tspy := New(t)
tspy.ExpectError()
// tspy.ExpectXXX()
tspy.Close()

To assert expectations manually call the Spy.AssertExpectations, or it will be called automatically when test (tt) finishes.

Full example: 

t.Run("closes file at the end of test", func(t *testing.T) {
	// --- Given ---
	tspy := tester.New(t).ExpectCleanups(1).Close()

	// --- When ---
	fil := OpenFile(tspy, "testdata/file.txt")

	// --- Then ---
	tspy.AssertExpectations()
	assert.ErrorIs(t, os.ErrClosed, fil.Close())
})

If the optional argument expectHelpers is provided the Spy.ExpectHelpers will be called with it. See Spy.ExpectHelpers method documentation for details.

func (*Spy) AssertExpectations 

func (spy *Spy) AssertExpectations() bool

AssertExpectations asserts all expectations and returns true on success, false otherwise. Each failed expectation is logged using tt instance.

func (*Spy) Cleanup 

func (spy *Spy) Cleanup(f func())

Cleanup registers a function to be called when the test and all its subtests complete. The registered function is always called at the end of the test.

func (*Spy) Close 

func (spy *Spy) Close() *Spy

Close closes the instance. You cannot add any expectations to a closed instance.

func (*Spy) Context 

func (spy *Spy) Context() context.Context

func (*Spy) Error 

func (spy *Spy) Error(args ...any)

func (*Spy) Errorf 

func (spy *Spy) Errorf(format string, args ...any)

func (*Spy) ExamineLog added in v0.10.0 

func (spy *Spy) ExamineLog() string

ExamineLog returns so far logged messages.

func (*Spy) ExpectCleanups 

func (spy *Spy) ExpectCleanups(cnt int) *Spy

ExpectCleanups sets the number of expected calls to the Spy.Cleanup method.

func (*Spy) ExpectError 

func (spy *Spy) ExpectError() *Spy

ExpectError sets the expectation that HUT should call one of the Spy.Error or Spy.Errorf methods at least once.

func (*Spy) ExpectFail 

func (spy *Spy) ExpectFail() *Spy

ExpectFail sets expectation the HUT should call one of the Fatal* or Error* methods.

func (*Spy) ExpectFatal 

func (spy *Spy) ExpectFatal() *Spy

ExpectFatal sets expectation that HUT should call one of the Spy.Fatal or Spy.Fatalf methods at least once.

func (*Spy) ExpectHelpers 

func (spy *Spy) ExpectHelpers(cnt int) *Spy

ExpectHelpers sets expectation how many times HUT should call Spy.Helper method. The value -1 means the Spy.Helper method must be run at least once.

Method will panic if the cnt value is less than -1 or the method is called more than once.

func (*Spy) ExpectLog 

func (spy *Spy) ExpectLog(matcher Strategy, msg string, args ...any) *Spy

ExpectLog sets expectation the HUT should call one of the Spy.Log or Spy.Logf methods with a given message. The expected message is constructed using format and args arguments, which are the same as in fmt.Sprintf. The matcher strategy is used to match the message.

Method call will panic if Spy.IgnoreLogs was called before.

func (*Spy) ExpectLogContain 

func (spy *Spy) ExpectLogContain(format string, args ...any) *Spy

ExpectLogContain sets expectation the HUT should call one of the Log* methods. The expected message is constructed using format and args arguments which are the same as in fmt.Sprintf. The Contains strategy is used to match a log message.

Method call will panic if Spy.IgnoreLogs was called before.

func (*Spy) ExpectLogEqual 

func (spy *Spy) ExpectLogEqual(format string, args ...any) *Spy

ExpectLogEqual sets expectation the HUT should call one of the Log* methods. The expected message is constructed using format and args arguments which are the same as in fmt.Sprintf. The Equal strategy is used to match messages.

Method call will panic if Spy.IgnoreLogs was called before.

func (*Spy) ExpectLogNotContain 

func (spy *Spy) ExpectLogNotContain(format string, args ...any) *Spy

ExpectLogNotContain sets expectation the HUT should call one of the Log* methods. The expected message is constructed using format and args arguments which are the same as in fmt.Sprintf. The NotContains strategy is used to match log messages.

Method call will panic if Spy.IgnoreLogs was called before.

func (*Spy) ExpectSetenv 

func (spy *Spy) ExpectSetenv(key, value string) *Spy

ExpectSetenv sets expectation that given environment variable is set by the HUT.

func (*Spy) ExpectSkipped 

func (spy *Spy) ExpectSkipped() *Spy

ExpectSkipped sets expectation that HUT will skip the test.

func (*Spy) ExpectTempDir 

func (spy *Spy) ExpectTempDir(cnt int) *Spy

ExpectTempDir sets expectation the HUT should call Spy.TempDir cnt number of times. If cnt is -1 the Spy.TempDir method can be called any number of times.

func (*Spy) ExpectedNames 

func (spy *Spy) ExpectedNames(cnt int) *Spy

ExpectedNames sets expectation the HUT should call Spy.Name cnt number of times.

func (*Spy) FailNow 

func (spy *Spy) FailNow()

func (*Spy) Failed 

func (spy *Spy) Failed() bool

Failed reports whether the HUT called any of the Spy.Error, Spy.Errorf, Spy.Fatal, Spy.Fatalf or Spy.FailNow methods. It's worth noting this method returning false DOES NOT mean the Spy expectations were met. The HUT may have never called the methods listed previously, but the spy itself didn't meet expectations.

func (*Spy) Fatal 

func (spy *Spy) Fatal(args ...any)

func (*Spy) Fatalf 

func (spy *Spy) Fatalf(format string, args ...any)

func (*Spy) Finish 

func (spy *Spy) Finish() *Spy

Finish marks the end of the test. It can be called by hand, or it's called automatically by a cleanup function as described in New. After Spy.Finish is called, most of the Spy methods will panic when called - check specific method documentation for details.

func (*Spy) GetTempDir 

func (spy *Spy) GetTempDir(idx int) string

GetTempDir returns the Nth (zero-indexed) temporary directory path returned by Spy.TempDir method. It will fail the test if the Spy.TempDir method was never called or the index of the directory is invalid.

func (*Spy) Helper 

func (spy *Spy) Helper()

func (*Spy) IgnoreLogs 

func (spy *Spy) IgnoreLogs() *Spy

IgnoreLogs instruct Spy to ignore checking logged messages. Method will panic if any of the Spy.ExpectLog* methods were already called.

func (*Spy) Log 

func (spy *Spy) Log(args ...any)

func (*Spy) Logf 

func (spy *Spy) Logf(format string, args ...any)

func (*Spy) Name 

func (spy *Spy) Name() string

func (*Spy) Setenv 

func (spy *Spy) Setenv(key, value string)

func (*Spy) Skip 

func (spy *Spy) Skip(args ...any)

func (*Spy) TempDir 

func (spy *Spy) TempDir() string

type Strategy 

type Strategy string

Strategy is the strategy of matching logs produced by Helper Under Test (HUT). 

const (
	// Equal is a strategy where messages logged by HUT are matched exactly.
	Equal Strategy = "equal"

	// Contains is a strategy where messages logged by HUT contain a string.
	Contains Strategy = "contains"

	// NotContains is a strategy where messages logged by HUT don't contain a
	// string.
	NotContains Strategy = "not-contains"

	// Regexp is a strategy where messages logged by the HUT match regular
	// expression.
	Regexp Strategy = "regexp"
)

Log matching strategies.

type T 

type T interface {
	// Cleanup registers a function to be called when the test and all its
	// subtests complete. Cleanup functions will be called in last added,
	// first called order.
	Cleanup(func())

	// Error is equivalent to Log followed by Fail.
	Error(args ...any)

	// Errorf is equivalent to Logf followed by Fail.
	Errorf(format string, args ...any)

	// Fatal is equivalent to Log followed by FailNow.
	Fatal(args ...any)

	// Fatalf is equivalent to Logf followed by FailNow.
	Fatalf(format string, args ...any)

	// FailNow marks the function as having failed and stops its execution
	// by calling [runtime.Goexit] (which then runs all deferred calls in the
	// current goroutine). Execution will continue at the next test or
	// benchmark. FailNow must be called from the goroutine running the test or
	// benchmark function, not from other goroutines created during the test.
	// Calling FailNow does not stop.
	FailNow()

	// Failed reports whether the function has failed.
	Failed() bool

	// Helper marks the calling function as a test helper function. When
	// printing file and line information, that function will be skipped.
	// Helper may be called simultaneously from multiple goroutines.
	Helper()

	// Log formats its arguments using default formatting, analogous to Println,
	// and records the text in the error log. For tests, the text will be
	// printed only if the test fails or the -test.v flag is set. For
	// benchmarks, the text is always printed to avoid having performance
	// depend on the value of the -test.v flag.
	Log(args ...any)

	// Logf formats its arguments according to the format, analogous to Printf,
	// and records the text in the error log. A final newline is added if not
	// provided. For tests, the text will be printed only if the test fails or
	// the -test.v flag is set. For benchmarks, the text is always printed to
	// avoid having performance depend on the value of the -test.v flag.
	Logf(format string, args ...any)

	// Name returns the name of the running (sub-) test.
	//
	// The name will include the name of the test along with the names of any
	// nested subtests. If two sibling subtests have the same name, Name will
	// append a suffix to guarantee the returned name is unique.
	Name() string

	// Setenv calls os.Setenv(key, value) and uses Cleanup to restore the
	// environment variable to its original value after the test.
	//
	// This cannot be used in parallel tests.
	Setenv(key, value string)

	// Skip is equivalent to Log followed by SkipNow.
	Skip(args ...any)

	// TempDir returns a temporary directory for the test to use. The directory
	// is automatically removed by Cleanup when the test and all its subtests
	// complete. Each subsequent call to TempDir returns a unique directory;
	// if the directory creation fails, TempDir terminates the test by calling
	// Fatal.
	TempDir() string

	// Context returns a context that is canceled just before Cleanup
	// registered functions are called. Cleanup functions can wait for any
	// resources that shut down on Context.Done before the test or benchmark
	// completes.
	Context() context.Context
}

T is a subset of testing.TB interface it has been defined to allow mocking.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.