check

package module
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 8, 2023 License: BSD-2-Clause Imports: 24 Imported by: 0

README

PkgGoDev GitHub Actions CI Status GitHub Actions CodeQL Status

The Go language provides an internal testing library, named testing, which is relatively slim due to the fact that the standard library correctness by itself is verified using it. The check package, on the other hand, expects the standard library from Go to be working correctly, and builds on it to offer a richer testing framework for libraries and applications to use.

This is fork of package go-check/check with some additional features.

gocheck includes features such as:

  • Helpful error reporting to aid on figuring problems out (see below)
  • Richer test helpers: assertions which interrupt the test immediately, deep multi-type comparisons, string matching, etc
  • Suite-based grouping of tests
  • Fixtures: per suite and/or per test set up and tear down
  • Benchmarks integrated in the suite logic (with fixtures, etc)
  • Management of temporary directories
  • Panic-catching logic, with proper error reporting
  • Proper counting of successes, failures, panics, missed tests, skips, etc
  • Explicit test skipping
  • Support for expected failures
  • Verbosity flag which disables output caching (helpful to debug hanging tests, for instance)
  • Multi-line string reporting for more comprehensible failures
  • Inclusion of comments surrounding checks on failure reports
  • Fully tested (it manages to test itself reliably)

Compatibility with go test

gocheck works as an extension to the testing package and to the "go test" runner. That allows keeping all current tests and using gocheck-based tests right away for new tests without conflicts. The gocheck API was purposefully made similar to the testing package for a smooth migration.

Installing and updating

Install gocheck's check package with the following command:

go get -v github.com/essentialkaos/check

To ensure you're using the latest version, run the following instead:

go get -u -v github.com/essentialkaos/check

API documentation

The API documentation for gocheck's check package is available online at:

https://github.com/essentialkaos/check?docs

Basic example

package hello_test

import (
    "testing"
    "io"

    . "github.com/essentialkaos/check"
)

// Hook up gocheck into the "go test" runner.
func Test(t *testing.T) { TestingT(t) }

type MySuite struct{}

var _ = Suite(&MySuite{})

func (s *MySuite) TestHelloWorld(c *C) {
    c.Assert(42, Equals, "42")
    c.Assert(io.ErrClosedPipe, ErrorMatches, "io: .*on closed pipe")
    c.Check(42, Equals, 42)
}

See Assertions and checks below for more information on these tests.

Using fixtures

Fixtures are available by using one or more of the following methods in a test suite:

  • func (s *SuiteType) SetUpSuite(c *C) - Run once when the suite starts running;
  • func (s *SuiteType) SetUpTest(c *C) - Run before each test or benchmark starts running;
  • func (s *SuiteType) TearDownTest(c *C) - Run after each test or benchmark runs;
  • func (s *SuiteType) TearDownSuite(c *C) - Run once after all tests or benchmarks have finished running.

Here is an example preparing some data in a temporary directory before each test runs:

type Suite struct{
    dir string
}

func (s *MySuite) SetUpTest(c *C) {
    s.dir = c.MkDir()
    // Use s.dir to prepare some data.
}

func (s *MySuite) TestWithDir(c *C) {
    // Use the data in s.dir in the test.
}

Adding benchmarks

Benchmarks may be added by prefixing a method in the suite with Benchmark. The method will be called with the usual *C argument, but unlike a normal test it is supposed to put the benchmarked logic within a loop iterating c.N times.

For example:

func (s *MySuite) BenchmarkLogic(c *C) {
    for i := 0; i < c.N; i++ {
        // Logic to benchmark
    }
}

These methods are only run when in benchmark mode, using the -check.b flag, and will present a result similar to the following when run:

PASS: myfile.go:67: MySuite.BenchmarkLogic       100000    14026 ns/op
PASS: myfile.go:73: MySuite.BenchmarkOtherLogic  100000    21133 ns/op

All the fixture methods are run as usual for a test method.

To obtain the timing for normal tests, use the -check.v flag instead.

Skipping tests

Tests may be skipped with the Skip method within SetUpSuite, SetUpTest, or the test method itself. This allows selectively ignoring tests based on custom factors such as the architecture being run, flags provided to the test, or the availbility of resources (network, etc).

As an example, the following test suite will skip all the tests within the suite unless the -live option is provided to go test:

var live = flag.Bool("live", false, "Include live tests")

type LiveSuite struct{}

func (s *LiveSuite) SetUpSuite(c *C) {
    if !*live {
        c.Skip("-live not provided")
    }
}

Running tests and output sample

Use the go test tool as usual to run the tests:

$ go test

----------------------------------------------------------------------
FAIL: hello_test.go:16: S.TestHelloWorld

hello_test.go:17:
    c.Check(42, Equals, "42")
... obtained int = 42
... expected string = "42"

hello_test.go:18:
    c.Check(io.ErrClosedPipe, ErrorMatches, "BOOM")
... error string = "io: read/write on closed pipe"
... regex string = "BOOM"


OOPS: 0 passed, 1 FAILED
--- FAIL: hello_test.Test
FAIL

Assertions and checks

gocheck uses two methods of *C to verify expectations on values obtained in test cases: Assert and Check. Both of these methods accept the same arguments, and the only difference between them is that when Assert fails, the test is interrupted immediately, while Check will fail the test, return false, and allow it to continue for further checks.

Assert and Check have the following types:

func (c *C) Assert(obtained interface{}, chk Checker, ...args interface{})
func (c *C) Check(obtained interface{}, chk Checker, ...args interface{}) bool

They may be used as follows:

func (s *S) TestSimpleChecks(c *C) {
    c.Assert(value, Equals, 42)
    c.Assert(s, Matches, "hel.*there")
    c.Assert(err, IsNil)
    c.Assert(foo, Equals, bar, Commentf("#CPUs == %d", runtime.NumCPU())
}

The last statement will display the provided message next to the usual debugging information, but only if the check fails.

Custom verifications may be defined by implementing the Checker interface.

There are several standard checkers available:

DeepEquals — Сhecker verifies that the obtained value is deep-equal to the expected value. The check will work correctly even when facing slices, interfaces, and values of different types (which always fail the test).

c.Assert(array, DeepEquals, []string{"hi", "there"})

Equals — Checker verifies that the obtained value is equal to the expected value, according to usual Go semantics for ==.

c.Assert(value, Equals, 42)

ErrorMatches — Checker verifies that the error value is non nil and matches the regular expression provided.

c.Assert(err, ErrorMatches, "perm.*denied")

ErrorMatchesOS — Checker verifies that the error value is non nil and matches with OS-specific regular expression.

c.Assert(err, ErrorMatchesOS, map[string]string{"darwin":"perm.*denied", "linux":"non-readable:*"})

FitsTypeOf — Checker verifies that the obtained value is assignable to a variable with the same type as the provided sample value.

c.Assert(value, FitsTypeOf, int64(0))
c.Assert(value, FitsTypeOf, os.Error(nil))

HasLen — Checker verifies that the obtained value has the provided length.

c.Assert(list, HasLen, 5)

Implements — Checker verifies that the obtained value implements the interface specified via a pointer to an interface variable.

var e os.Error
c.Assert(err, Implements, &e)

IsNil — Checker tests whether the obtained value is nil.

c.Assert(err, IsNil)

Matches — Checker verifies that the string provided as the obtained value (or the string resulting from obtained.String()) matches the regular expression provided.

c.Assert(err, Matches, "perm.*denied")

NotNil — Checker verifies that the obtained value is not nil. This is an alias for Not(IsNil), made available since it's a fairly common check.

c.Assert(iface, NotNil)

PanicMatches — Checker verifies that calling the provided zero-argument function will cause a panic with an error value matching the regular expression provided.

c.Assert(func() { f(1, 2) }, PanicMatches, `open.*: no such file or directory`)

Panics — Checker verifies that calling the provided zero-argument function will cause a panic which is deep-equal to the provided value.

c.Assert(func() { f(1, 2) }, Panics, &SomeErrorType{"BOOM"}).

NotPanics — Checker verifies that calling the provided zero-argument function will not cause any panic.

c.Assert(func() { f(1, 2) }, NotPanics).

Selecting which tests to run

gocheck can filter tests out based on the test name, the suite name, or both. To run tests selectively, provide the command line option -check.f when running go test. Note that this option is specific to gocheck, and won't affect go test itself.

Some examples:

$ go test -check.f MyTestSuite
$ go test -check.f "Test.*Works"
$ go test -check.f "MyTestSuite.Test.*Works"

Verbose modes

gocheck offers two levels of verbosity through the -check.v and -check.vv flags. In the first mode, passing tests will also be reported. The second mode will disable log caching entirely and will stream starting and ending suite calls and everything logged in between straight to the output. This is useful to debug hanging tests, for instance.

Supported flags

Flag Description
check.f Regular expression selecting which tests and/or suites to run
check.v Verbose mode
check.vv Super verbose mode (disables output caching)
check.b Run benchmarks
check.btime Approximate run time for each benchmark (default: 1 second)
check.bmem Report memory benchmarks
check.list List the names of all tests that will be run
check.work Display and do not remove the test working directory
check.threads Number of parallel tests (default: 1)

License

gocheck is made available under the Simplified BSD License.

Documentation

Overview

Package check is a rich testing extension for Go's testing package.

For details about the project, see: 

https://kaos.sh/check

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func List 

func List(suite interface{}, runConf *RunConf) []string

List returns the names of the test functions in the given suite that will be run with the provided run configuration.

func ListAll 

func ListAll(runConf *RunConf) []string

ListAll returns the names of all the test functions registered with the Suite function that will be run with the provided run configuration.

func Suite 

func Suite(suite interface{}) interface{}

Suite registers the given value as a test suite to be run. Any methods starting with the Test prefix in the given value will be considered as a test method.

func TestingT 

func TestingT(testingT *testing.T)

TestingT runs all test suites registered with the Suite function, printing results to stdout, and reporting any failures back to the "testing" package.

Types

type C 

type C struct {
	// contains filtered or unexported fields
}

func (*C) Assert 

func (c *C) Assert(obtained interface{}, checker Checker, args ...interface{})

Assert ensures that the first value matches the expected value according to the provided checker. If they do not match, an error is logged, the test is marked as failed, and the test execution stops.

Some checkers may not need the expected argument (e.g. IsNil).

If the last value in args implements CommentInterface, it is used to log additional information instead of being passed to the checker (see Commentf for an example).

func (*C) Check 

func (c *C) Check(obtained interface{}, checker Checker, args ...interface{}) bool

Check verifies if the first value matches the expected value according to the provided checker. If they do not match, an error is logged, the test is marked as failed, and the test execution continues.

Some checkers may not need the expected argument (e.g. IsNil).

If the last value in args implements CommentInterface, it is used to log additional information instead of being passed to the checker (see Commentf for an example).

func (*C) Error 

func (c *C) Error(args ...interface{})

Error logs an error into the test error output and marks the test as failed. The provided arguments are assembled together into a string with fmt.Sprint.

func (*C) Errorf 

func (c *C) Errorf(format string, args ...interface{})

Errorf logs an error into the test error output and marks the test as failed. The provided arguments are assembled together into a string with fmt.Sprintf.

func (*C) ExpectFailure 

func (c *C) ExpectFailure(reason string)

ExpectFailure informs that the running test is knowingly broken for the provided reason. If the test does not fail, an error will be reported to raise attention to this fact. This method is useful to temporarily disable tests which cover well known problems until a better time to fix the problem is found, without forgetting about the fact that a failure still exists.

func (*C) Fail 

func (c *C) Fail()

Fail marks the currently running test as failed.

Something ought to have been previously logged so the developer can tell what went wrong. The higher level helper functions will fail the test and do the logging properly.

func (*C) FailNow 

func (c *C) FailNow()

FailNow marks the currently running test as failed and stops running it. Something ought to have been previously logged so the developer can tell what went wrong. The higher level helper functions will fail the test and do the logging properly.

func (*C) Failed 

func (c *C) Failed() bool

Failed returns whether the currently running test has already failed.

func (*C) Fatal 

func (c *C) Fatal(args ...interface{})

Fatal logs an error into the test error output, marks the test as failed, and stops the test execution. The provided arguments are assembled together into a string with fmt.Sprint.

func (*C) Fatalf 

func (c *C) Fatalf(format string, args ...interface{})

Fatlaf logs an error into the test error output, marks the test as failed, and stops the test execution. The provided arguments are assembled together into a string with fmt.Sprintf.

func (*C) GetTestLog 

func (c *C) GetTestLog() string

GetTestLog returns the current test error output.

func (*C) Log 

func (c *C) Log(args ...interface{})

Log logs some information into the test error output. The provided arguments are assembled together into a string with fmt.Sprint.

func (*C) Logf 

func (c *C) Logf(format string, args ...interface{})

Log logs some information into the test error output. The provided arguments are assembled together into a string with fmt.Sprintf.

func (*C) MkDir 

func (c *C) MkDir() string

Create a new temporary directory which is automatically removed after the suite finishes running.

func (*C) Output 

func (c *C) Output(calldepth int, s string) error

Output enables *C to be used as a logger in functions that require only the minimum interface of *log.Logger.

func (*C) ResetTimer 

func (c *C) ResetTimer()

ResetTimer sets the elapsed benchmark time to zero. It does not affect whether the timer is running.

func (*C) SetBytes 

func (c *C) SetBytes(n int64)

SetBytes informs the number of bytes that the benchmark processes on each iteration. If this is called in a benchmark it will also report MB/s.

func (*C) Skip 

func (c *C) Skip(reason string)

Skip skips the running test for the provided reason. If run from within SetUpTest, the individual test being set up will be skipped, and if run from within SetUpSuite, the whole suite is skipped.

func (*C) StartTimer 

func (c *C) StartTimer()

StartTimer starts timing a test. This function is called automatically before a benchmark starts, but it can also used to resume timing after a call to StopTimer.

func (*C) StopTimer 

func (c *C) StopTimer()

StopTimer stops timing a test. This can be used to pause the timer while performing complex initialization that you don't want to measure.

func (*C) Succeed 

func (c *C) Succeed()

Succeed marks the currently running test as succeeded, undoing any previous failures.

func (*C) SucceedNow 

func (c *C) SucceedNow()

SucceedNow marks the currently running test as succeeded, undoing any previous failures, and stops running the test.

func (*C) TestName 

func (c *C) TestName() string

TestName returns the current test name in the form "SuiteName.TestName"

type Checker 

type Checker interface {
	Info() *CheckerInfo
	Check(params []interface{}, names []string) (result bool, error string)
}

The Checker interface must be provided by checkers used with the Assert and Check verification methods. 

var DeepEquals Checker = &deepEqualsChecker{
	&CheckerInfo{Name: "DeepEquals", Params: []string{"obtained", "expected"}},
}

The DeepEquals checker verifies that the obtained value is deep-equal to the expected value. The check will work correctly even when facing slices, interfaces, and values of different types (which always fail the test).

For example: 

c.Assert(value, DeepEquals, 42)
c.Assert(array, DeepEquals, []string{"hi", "there"})

var Equals Checker = &equalsChecker{
	&CheckerInfo{Name: "Equals", Params: []string{"obtained", "expected"}},
}

The Equals checker verifies that the obtained value is equal to the expected value, according to usual Go semantics for ==.

For example: 

c.Assert(value, Equals, 42)

var ErrorMatches Checker = errorMatchesChecker{
	&CheckerInfo{Name: "ErrorMatches", Params: []string{"value", "regex"}},
}

The ErrorMatches checker verifies that the error value is non nil and matches the regular expression provided.

For example: 

c.Assert(err, ErrorMatches, "perm.*denied")

var ErrorMatchesOS Checker = errorMatchesOSChecker{
	&CheckerInfo{Name: "ErrorMatchesOS", Params: []string{"value", "regex-map"}},
}

The ErrorMatchesOS checker verifies that the error value is non nil and matches OS-specific regular expression.

For example: 

c.Assert(err, ErrorMatchesOS, map[string]error{"perm.*denied")

var FitsTypeOf Checker = &fitsTypeChecker{
	&CheckerInfo{Name: "FitsTypeOf", Params: []string{"obtained", "sample"}},
}

The FitsTypeOf checker verifies that the obtained value is assignable to a variable with the same type as the provided sample value.

For example: 

c.Assert(value, FitsTypeOf, int64(0))
c.Assert(value, FitsTypeOf, os.Error(nil))

var HasLen Checker = &hasLenChecker{
	&CheckerInfo{Name: "HasLen", Params: []string{"obtained", "n"}},
}

The HasLen checker verifies that the obtained value has the provided length. In many cases this is superior to using Equals in conjunction with the len function because in case the check fails the value itself will be printed, instead of its length, providing more details for figuring the problem.

For example: 

c.Assert(list, HasLen, 5)

var Implements Checker = &implementsChecker{
	&CheckerInfo{Name: "Implements", Params: []string{"obtained", "ifaceptr"}},
}

The Implements checker verifies that the obtained value implements the interface specified via a pointer to an interface variable.

For example: 

var e os.Error
c.Assert(err, Implements, &e)

var IsNil Checker = &isNilChecker{
	&CheckerInfo{Name: "IsNil", Params: []string{"value"}},
}

The IsNil checker tests whether the obtained value is nil.

For example: 

c.Assert(err, IsNil)

var Matches Checker = &matchesChecker{
	&CheckerInfo{Name: "Matches", Params: []string{"value", "regex"}},
}

The Matches checker verifies that the string provided as the obtained value (or the string resulting from obtained.String()) matches the regular expression provided.

For example: 

c.Assert(err, Matches, "perm.*denied")

var NotNil Checker = &notNilChecker{
	&CheckerInfo{Name: "NotNil", Params: []string{"value"}},
}

The NotNil checker verifies that the obtained value is not nil.

For example: 

c.Assert(iface, NotNil)

This is an alias for Not(IsNil), made available since it's a fairly common check. 

var NotPanics Checker = &notPanicsChecker{
	&CheckerInfo{Name: "NotPanics", Params: []string{"function"}},
}

The NotPanics checker verifies that calling the provided zero-argument function will not cause any panic.

For example: 

c.Assert(func() { f(1, 2) }, NotPanics).

var PanicMatches Checker = &panicMatchesChecker{
	&CheckerInfo{Name: "PanicMatches", Params: []string{"function", "expected"}},
}

The PanicMatches checker verifies that calling the provided zero-argument function will cause a panic with an error value matching the regular expression provided.

For example: 

c.Assert(func() { f(1, 2) }, PanicMatches, `open.*: no such file or directory`).

var Panics Checker = &panicsChecker{
	&CheckerInfo{Name: "Panics", Params: []string{"function", "expected"}},
}

The Panics checker verifies that calling the provided zero-argument function will cause a panic which is deep-equal to the provided value.

For example: 

c.Assert(func() { f(1, 2) }, Panics, &SomeErrorType{"BOOM"}).

func Not 

func Not(checker Checker) Checker

The Not checker inverts the logic of the provided checker. The resulting checker will succeed where the original one failed, and vice-versa.

For example: 

c.Assert(a, Not(Equals), b)

type CheckerInfo 

type CheckerInfo struct {
	Name   string
	Params []string
}

See the Checker interface.

func (*CheckerInfo) Info 

func (info *CheckerInfo) Info() *CheckerInfo

type CommentInterface 

type CommentInterface interface {
	CheckCommentString() string
}

CommentInterface must be implemented by types that attach extra information to failed checks. See the Commentf function for details.

func Commentf 

func Commentf(format string, args ...interface{}) CommentInterface

Commentf returns an infomational value to use with Assert or Check calls. If the checker test fails, the provided arguments will be passed to fmt.Sprintf, and will be presented next to the logged failure.

For example: 

c.Assert(v, Equals, 42, Commentf("Iteration #%d failed.", i))

Note that if the comment is constant, a better option is to simply use a normal comment right above or next to the line, as it will also get printed with any errors: 

c.Assert(l, Equals, 8192) // Ensure buffer size is correct (bug #123)

type Result 

type Result struct {
	Succeeded        int
	Failed           int
	Skipped          int
	Panicked         int
	FixturePanicked  int
	ExpectedFailures int
	Missed           int    // Not even tried to run, related to a panic in the fixture.
	RunError         error  // Houston, we've got a problem.
	WorkDir          string // If KeepWorkDir is true
}

func Run 

func Run(suite interface{}, runConf *RunConf) *Result

Run runs the provided test suite using the provided run configuration.

func RunAll 

func RunAll(runConf *RunConf) *Result

RunAll runs all test suites registered with the Suite function, using the provided run configuration.

func (*Result) Add 

func (r *Result) Add(other *Result)

func (*Result) Passed 

func (r *Result) Passed() bool

func (*Result) String 

func (r *Result) String() string

type RunConf 

type RunConf struct {
	Output        io.Writer
	Stream        bool
	Verbose       bool
	Filter        string
	Benchmark     bool
	BenchmarkTime time.Duration // Defaults to 1 second
	BenchmarkMem  bool
	KeepWorkDir   bool
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.