README
¶
expect
THIS PACKAGE IS forked from github.com/quii/pepper, as an experiment to play with
the naming of the pkg names to aid in readability/dev ux.
Type-safe, composable, extensible test matching for Go
Inspired by Hamcrest
When writing tests it is sometimes difficult to get the balance right between over specifying the test (and making it brittle to changes), and not specifying enough (making the test less valuable since it continues to pass even when the thing being tested is broken). Having a tool that allows you to pick out precisely the aspect under test and describe the values it should have, to a controlled level of precision, helps greatly in writing tests that are “just right”. Such tests fail when the behaviour of the aspect under test deviates from the expected behaviour, yet continue to pass when minor, unrelated changes to the behaviour are made.
From The Hamcrest Introduction
Out of the box, expect can work just like other test libraries in Go like is, where you
can make basic assertions on data.
Simple examples
expect.It(t, "Pepper").To(be.Eq("Pepper"))
expect has lots of built-in matchers, which you pass in to To, for common testing operations such as
examining comparable, string, io and *http.Response.
What is a Matcher[T] ? It's a function that takes a T, and returns a MatchResult
type Matcher[T any] func (T) MatchResult
Here is the definition of MatchResult
type MatchResult struct {
Description string
Matches bool
But string
SubjectName string
}
Here is how HaveAllCaps is defined
func HaveAllCaps(in string) matching.MatchResult {
return matching.MatchResult{
Description: "be in all caps",
Matches: strings.ToUpper(in) == in,
But: "it was not in all caps",
}
}
expect.It(t, "HELLO").To(HaveAllCaps)
Quite nice, but not all that different from libraries you already use. expect starts to come into its own when you start
taking advantage of composing matchers.
Composing matchers
expect.It(t, score).To(be.Greater(5).And(be.Less(10)))
The method And on Matcher[T], lets you compose matchers. And returns the composed Matcher[T], so you can
continue to chain more matchers however you like.
expect.It(t, score).To(be.Greater(5).And(be.Not(be.Greater(opponentScore))))
Not negates a matcher. By using matchers, and composing them with And, Not, Or, you can write very expressive
tests, cheaply.
Defining your own matchers
You can define your own matchers for your own types. Over time, the investment in writing matchers for your tests pays dividends, the cost of writing your tests decrease, as you reuse, mix and match the standard matchers and composition tools with your own.
Some will argue writing these matchers adds more code as if that's inherently a bad thing, but I would argue that the tests read far better, and don't suffer the problems you can run in to if you lazily assert on complex types.
In my experience of using matchers, over time as you find yourself testing more and more permutations of behaviour, the effort behind the matchers pays off in terms of making tests easier to write, read and maintain.
Here is an example of testing a todo-list
type Todo struct {
Name string `json:"name"`
Completed bool `json:"completed"`
LastUpdated time.Time `json:"last_updated"`
}
func WithCompletedTODO(todo Todo) MatchResult {
return MatchResult{
Description: "have a completed todo",
Matches: todo.Completed,
But: "it wasn't complete",
}
}
func WithTodoNameOf(todoName string) Matcher[Todo] {
return func (todo Todo) MatchResult {
return MatchResult{
Description: fmt.Sprintf("have a todo name of %q", todoName),
Matches: todo.Name == todoName,
But: fmt.Sprintf("it was %q", todo.Name),
}
}
}
func TestTodos(t *testing.T) {
t.Run("with completed todo", func (t *testing.T) {
res := httptest.NewRecorder()
res.Body.WriteString(`{"name": "Finish the side project", "completed": true}`)
expect.It(t, res.Result()).To(be.HTTPRespBody(be.ParsedJSON[Todo](WithCompletedTODO)))
})
t.Run("with a todo name", func (t *testing.T) {
res := httptest.NewRecorder()
res.Body.WriteString(`{"name": "Finish the side project", "completed": false}`)
expect.It(t, res.Result()).To(be.HTTPRespBody(be.ParsedJSON[Todo](WithTodoNameOf("Finish the side project"))))
})
t.Run("compose the matchers", func (t *testing.T) {
res := httptest.NewRecorder()
res.Body.WriteString(`{"name": "Egg", "completed": false}`)
res.Header().Add("content-type", "application/json")
expect.It(t, res.Result()).To(
be.HTTPStatus(http.StatusOK),
be.ContentTypeJSONHeader,
be.HTTPRespBody(be.ParsedJSON[Todo](WithTodoNameOf("Egg").And(be.Not(WithCompletedTODO)))),
)
})
}
Note how we can compose built-in matchers like HTTPStatus, ContentTypeJSONHeader and Not, with the custom-built
matchers to easily write very expressive tests that fail with very clear error messages. expect makes it really easy
to check JSON responses of your HTTP handlers.
Due to the compositional nature of Matcher[T], we can re-use our Matcher[Todo] for tests at different abstraction
levels; these matchers are not coupled to HTTP, we composed the matchers for this context. For instance, if you have
a TodoRepository, or a TodoService, you can use these same matchers in the tests for them too.
Test failure readability
One of the most frustrating areas working with automated tests is how often test failure quality is poor. I'm sure every developer has into this scenario:
test_foo.go:123-true was not equal to false
Computer, I already know that true is not equal to false. What was not false? What was true? What was the context?
expect makes it easy for you to write tests that give you a clear message when they fail.
t.Run("failure message", func (t *testing.T) {
res := httptest.NewRecorder()
res.WriteHeader(http.StatusNotFound)
expect.It(t, res.Result()).To(be.HttpStatus(http.StatusOK))
})
Here is the failing output
=== RUN TestHTTPTestMatchers/Status_code_matchers/OK/failure_message
matchers_test.go:292: expected the response to have status of 200, but it was 404
Embracing this approach with well-written matchers means you get readable test failures for free.
Summary
expect brings the following to the table
- ✅ Type-safe tests. No
interface{} - ✅ Composition to reduce boilerplate
- ✅ Clear test output as a first-class citizen
- ✅ A "standard library" of matchers to let you quickly write expressive tests for common scenarios out of the box
- ✅ Extensibility. You can write rich, re-useable matchers for your domain to help you write high quality, low maintenance tests
- ❌ Not a framework.
expectdoes not dictate how you set up your tests or how you design your code. All it does is help you write better assertions with less effort.
Examples
You can find high-level examples in the GoDoc.
The matchers, which you can find in the directories section also have examples.
You'll notice in the examples the following line:
t := &SpyTB{}
This is a test spy that is used to verify the output of the matches made. The examples call LastError() to see what
test output would happen, so you can see what the failures look like.
Finally, I have worked through my course Learn Go with Tests, using
expect to write assertions, so you can find more
examples at the Github repository
Trade-offs and optimisations
Type-safety
Now Go has generics, we have a more expressive language which lets us make matchers that are type-safe, rather than relying on reflection. The problem with reflection is it can lead to lazy test writing, especially when you're dealing with complex types. Developers can lazily assert on complex types, which makes the tests harder to follow and more brittle. See the curse of asserting on irrelevant detail for more on this.
The trade-off we're making here is you will have to make your own matchers for your own types at times. This is a good thing, as it forces you to think about what you're actually testing, and it makes your tests more readable and less brittle.
Due to the compositional nature of the library though, you should be able to leverage existing matchers for re-use. For example, you'll never have to make a matcher that parses JSON, you should instead use ParsedJSON in combination with a matcher of your type.
Due to Go having some constraints on where you can use generics, such as function types not being allowed to have type
parameters, the API isn't as friendly as it would be, if you used interface{}/any. However, this is a trade-off I am
OK with, in the name of type-safety.
Composition and re-use
Matchers should be designed with composition in mind. For instance, let's take a look at the body matcher for an HTTP response:
func HTTPResponseBody(bodyMatchers expect.Matcher[io.Reader]) expect.Matcher[*http.Response]
This allows the user to re-use io.Reader matchers that are already defined, compose them with And/Or/Not, and of
course users can define their own Matcher[io.Reader] too.
Test failure readability
Often the most expensive part of a test suite is trying to understand what has failed when a test goes red. This is why TDD emphasises the first step of writing a failing test and inspecting the output. It's a chance for you to see what it's like if the test fails 6 months later, and you've lost all context.
expect strives to make it easy for you to write tests that explain exactly what has gone wrong, but it does require a
little more upfront effort when writing your own matchers.
type MatchResult struct {
Description string
Matches bool
But string
SubjectName string
}
The bare minimum you need to write a matcher is to complete the Matches field, but to get good test failure
messages, you'll need to invest a little time filling the other fields and checking the failure reads nicely.
expect will not bend over backwards to write perfect English. It's important that the reason for test failure is
clear, but perfect grammar is not needed for this; and the complexity cost involved to make matchers "write" different
sentences depending on how they are used, is not worth it.
Benefits of matchers
A lot of the time people zero-in on the "fluency" of matchers. Whilst it's true that the ease of use does make using matchers attractive, I think there's a larger, perhaps less obvious benefit.
The curse of asserting on irrelevant detail
A trap many fall in to when they write tests is they end up writing tests with greedy assertions where you end up lazily writing tests where you check one complex object equals another.
Often when we write a test, we only really care about the state of one field in a struct, yet when we are greedy, we end up coupling our tests to other data needlessly. This makes the test:
- Brittle, if domain logic changes elsewhere that happens to change values you weren't interested in, your test will fail
- Difficult to read, it's less obvious which effect you were hoping to exercise
Matchers allow you to write domain specific matching code, focused on the specific effects you're looking for. When used well, with a domain-centric, well-designed codebase, you tend to build a useful library of matchers that you can ** re-use and compose** to write clear, consistently written, less brittle tests.
How to write your own matchers
Reminder: a matcher is a function that takes the thing you're trying to match against, returning a result
type Matcher[T any] func (T) MatchResult
Here is the definition of MatchResult
type MatchResult struct {
Description string
Matches bool
But string
SubjectName string
}
Here is how HaveAllCaps is defined
func HaveAllCaps(in string) matching.MatchResult {
return matching.MatchResult{
Description: "be in all caps",
Matches: strings.ToUpper(in) == in,
But: "it was not in all caps",
}
}
Higher-order matchers
This is fine for simple matchers where you want to assert on a static property of T. Often though, you'll want to
write matchers where you want to check a particular _ value of a property_ .
For this, no magic is required, just create a higher-order function that returns a Matcher[T].
A simple example is with Eq
func Eq[T comparable](in T) matching.Matcher[T] {
return func (got T) matching.MatchResult {
return matching.MatchResult{
Description: fmt.Sprintf("be equal to %v", in),
Matches: got == in,
But: fmt.Sprintf("it was %v", got),
}
}
}
Leveraging composition
When designing your higher-order matchers, think about how the value you are matching against could be matched with other matchers you may not have thought of.
For example, Size, I could've written like this:
func Size[T any](size int) expect.Matcher[[]T]
However, this needlessly couples the matcher to the specific matching I was currently catering for (logically Eq)
. Instead, we can design our matcher to be combined with other matchers that work on int.
func Size[T any](matcher expect.Matcher[int]) expect.Matcher[[]T] {
return func (items []T) expect.MatchResult {
return matcher(len(items))
}
}
This way, users can use this matcher in different ways, like checking if a slice has a size Less(5)
or Greater(3).
With this simple change, users can also leverage the other composition tools like And:
expect.It(t, catsInAHotel).To(HaveSize(be.Greater(3).And(be.Less(10))))
Tip: When designing your matcher, consider changing the argument(s) from T to Matcher[T].
Test support
expect makes testing matchers easy because you inject in the testing framework into It, so we can spy on it.
I have found writing testable examples to be a satisfying way of both documenting and testing matchers.
func ExampleContainItem() {
t := &SpyTB{}
anArray := []string{"HELLO", "WORLD"}
expect.It(t, anArray).To(ContainItem(HaveAllCaps))
fmt.Println(t.LastError())
// Output:
}
func ExampleContainItem_fail() {
t := &SpyTB{}
anArray := []string{"hello", "world"}
expect.It(t, anArray).To(ContainItem(HaveAllCaps))
fmt.Println(t.LastError())
// Output: expected [hello world] to contain an item in all caps, but it did not
}
However, if you wish to check multiple scenarios, polluting the go doc with lots of examples may not be appropriate, in which case, write some unit tests.
Check out some of the unit tests for some of the comparison matchers
func TestComparisonMatchers(t *testing.T) {
t.Run("Less than", func (t *testing.T) {
t.Run("passing", func (t *testing.T) {
expect.It(t, 5).To(be.Less(6))
})
t.Run("failing", func (t *testing.T) {
spytb.VerifyFailingMatcher(t, 6, be.Less(6), "expected 6 to be less than 6")
spytb.VerifyFailingMatcher(t, 6, be.Less(3), "expected 6 to be less than 3")
})
})
t.Run("Greater than", func (t *testing.T) {
t.Run("passing", func (t *testing.T) {
expect.It(t, 5).To(be.Greater(4))
})
t.Run("failing", func (t *testing.T) {
spytb.VerifyFailingMatcher(t, 6, be.Greater(6), "expected 6 to be greater than 6")
spytb.VerifyFailingMatcher(t, 2, be.Greater(10), "expected 2 to be greater than 10")
})
})
}
Contributing your own matchers
If you have a matcher you think would be useful to others, please consider contributing it to this library.
Please only submit matchers that work against types in the standard library. This keeps the library focused and backward compatible. It would be fantastic if over time this library matured into a rich suite of matchers so any dev can pick up Go and start writing excellent tests against the standard library, which already gets you so far in terms of getting work done.
Your PR will need the following
- At least two testable examples, one showing the matcher passing (with an empty output) and one showing the matcher failing with the expected failing output. This will help users understand how to use your matcher.
- Automated tests in general
- Go doc comments for the matcher
As discussed above, try to keep them "open" in terms of their design, so they can be composed with other matchers.
Documentation
¶
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Error ¶
Error is a helper function that will call t.Fatalf if the error is nil.
Example ¶
package main
import (
"errors"
"fmt"
"github.com/jsteenb2/expect"
)
func main() {
t := &expect.SpyTB{}
err := errors.New("oh no")
expect.Error(t, err)
fmt.Println(t.Result())
}
Output: Test passed
func ErrorOfType ¶
ErrorOfType is a helper function that will call t.Fatalf if the error is not of the expected type.
Example ¶
package main
import (
"errors"
"fmt"
"github.com/jsteenb2/expect"
)
func main() {
t := &expect.SpyTB{}
unauthorised := errors.New("unauthorised")
wrappedErr := fmt.Errorf("oh no: %w", unauthorised)
expect.ErrorOfType(t, wrappedErr, unauthorised)
fmt.Println(t.Result())
}
Output: Test passed
Example (Failing) ¶
package main
import (
"errors"
"fmt"
"github.com/jsteenb2/expect"
)
func main() {
t := &expect.SpyTB{}
unauthorised := errors.New("unauthorised")
wrappedErr := fmt.Errorf("oh no: %w", unauthorised)
expect.ErrorOfType(t, wrappedErr, errors.New("not found"))
fmt.Println(t.Result())
}
Output: Test failed: [expected error of type *errors.errorString, but got "oh no: unauthorised"]
func NoError ¶
NoError is a helper function that will call t.Fatalf if the error is not nil.
Example ¶
package main
import (
"errors"
"fmt"
"github.com/jsteenb2/expect"
)
func main() {
t := &expect.SpyTB{}
err := errors.New("oh no")
expect.NoError(t, err)
fmt.Println(t.Result())
}
Output: Test failed: [unexpected error: oh no]
Types ¶
type Inspector ¶
type Inspector[T any] struct { Subject T // contains filtered or unexported fields }
func It ¶
It is the entry point for the matcher DSL. Pass in the testing.TB and the subject you want to test.
func (Inspector[T]) To ¶
To is the method that actually runs the matchers. It will call Errorf on the testing.TB if any of the matchers fail.
Example ¶
package main
import (
"fmt"
"github.com/jsteenb2/expect"
"github.com/jsteenb2/expect/be"
)
func main() {
t := &expect.SpyTB{}
expect.It(t, "Pepper").To(be.Eq("Stanley"))
fmt.Println(t.Result())
}
Output: Test failed: [expected "Pepper" to be equal to "Stanley", but it was "Pepper"]
type MatchResult ¶
MatchResult describes the result of a match against a "subject".
func (MatchResult) Combine ¶
func (m MatchResult) Combine(other MatchResult) MatchResult
Combine merges two MatchResults into one.
func (MatchResult) Error ¶
func (m MatchResult) Error() string
func (MatchResult) Zero ¶
func (m MatchResult) Zero() bool
Zero returns true if the MatchResult is the zero value.
type Matcher ¶
type Matcher[T any] func(T) MatchResult
func (Matcher[T]) And ¶
And combines matchers with a boolean AND.
Example ¶
package main
import (
"fmt"
"github.com/jsteenb2/expect"
"github.com/jsteenb2/expect/be"
)
func main() {
t := &expect.SpyTB{}
player := Player{
Name: "Chris",
Points: 11,
}
expect.It(t, player).To(HaveScore(
be.Greater(5).And(be.Less(10)),
))
fmt.Println(t.Result())
}
type Player struct {
Name string
Points int
}
func (s Player) String() string {
return fmt.Sprintf("Player %s", s.Name)
}
func HaveScore(matcher expect.Matcher[int]) expect.Matcher[Player] {
return func(s Player) expect.MatchResult {
result := matcher(s.Points)
return expect.MatchResult{
Description: "score " + result.Description,
Matches: result.Matches,
But: fmt.Sprintf("it was %d", s.Points),
}
}
}
Output: Test failed: [expected Player Chris to score be greater than 5 and be less than 10, but it was 11]
func (Matcher[T]) Or ¶
Or combines matchers with a boolean OR.
Example ¶
package main
import (
"fmt"
"github.com/jsteenb2/expect"
)
func main() {
t := &expect.SpyTB{}
tshirt := TShirt{Colour: "yellow"}
expect.It(t, tshirt).To(HaveColour("blue").Or(HaveColour("red")))
fmt.Println(t.Result())
}
type TShirt struct {
Colour string
}
func (t TShirt) String() string {
return "the t-shirt"
}
func HaveColour(colour string) expect.Matcher[TShirt] {
return func(t TShirt) expect.MatchResult {
return expect.MatchResult{
Description: fmt.Sprintf("have colour %q", colour),
Matches: t.Colour == colour,
But: fmt.Sprintf("it was %q", t.Colour),
}
}
}
Output: Test failed: [expected the t-shirt to have colour "blue" or have colour "red", but it was "yellow"]
Source Files
Directories