gandalf

README

Gandalf

One Tool to rule them all, One Tool to CI them, One Tool to test them all and in the darkness +1 them

At the highest level Gandalf is a tool for contract testing a web API.

Docs

https://godoc.org/github.com/JumboInteractiveLimited/Gandalf

Documentation

Overview

One Tool to rule them all, One Tool to CI them, One Tool to test them all and in the darkness +1 them.

Gandalf is designed to provide a language and stack agnostic HTTP API contract testing suite and prototyping toolchain. This is achieved by; running an HTTP API (aka provider), connecting to it as a real client (aka consumer) of the provider, asserting that it matches various rules (aka contracts). Optionally, once a contract is written you can then generate an approximation of the API (this happens just before the contract is tested) in the form of a mock. This allows for rapid prototyping and/or parallel development of the real consumer and provider implementations.

Gandalf has no allegiance to any specific paradigms, technologies, or concepts and should bend to fit real world use cases as opposed to vice versa. This means if Gandalf does something one way today it does not mean that tomorrow it could not support a different way provided someone has a use for it.

While Gandalf does use golang and the go test framework, it is not specific to go as at its core it just makes HTTP requests and checks the responses. Your web server or clients can be written in any language/framework. The official documentation also uses JSON and RESTful API's as examples but Gandalf supports any and all paradigms or styles of API.

Most go programs are compiled down to a binary and executed, Gandalf is designed to be used as a library to write your own tests and decorate the test binary instead. For example, Gandalf does have several command line switches however they are provided to the `go test` command instead of some non existent `Gandalf` command. This allows Gandalf to get all kind of testing and benchmarking support for free while being a well known stable base to build upon.

Contract testing can be a bit nebulous and also has various option prefixes such as Consumer Driven, Gandalf cares not for any prefixes (who writes contracts and where is up to you) nor does it care if you are testing the interface or your API or the business logic or some combination of both, no one will save you from blowing your own foot off if you choose to.

Index

• Variables
• func BenchmarkInOrder(b *testing.B, contracts []*Contract)
• func GetRequestBody(r *http.Request) string
• func GetResponseBody(r *http.Response) string
• func Main(m *testing.M)
• func MainWithHandler(m *testing.M, handler http.Handler)
• func SaneResponse() *http.Response
• type Checker
• type Contract
  • func (c *Contract) Assert(t Testable)
  • func (c *Contract) Benchmark(b *testing.B)
• type DynamicRequester
  • func (r *DynamicRequester) Call(run int) (*http.Response, error)
  • func (r *DynamicRequester) GetRequest() *http.Request
• type Exporter
• type Requester
• type SimpleChecker
  • func (c *SimpleChecker) Assert(res *http.Response) error
  • func (c *SimpleChecker) GetResponse() *http.Response
• type SimpleRequester
  • func NewSimpleRequester(method, url, body string, headers http.Header, timeout time.Duration) *SimpleRequester
  • func (r *SimpleRequester) Call(run int) (*http.Response, error)
  • func (r *SimpleRequester) GetRequest() *http.Request
• type State
  • func GetState() *State
  • func (s *State) Clear()
  • func (s *State) ClearKey(key string)
  • func (s *State) ClearRegex(expr string) error
  • func (s *State) GetResponse(key string) *http.Response
• type Testable
• type ToMMock
  • func (m *ToMMock) Save(c *Contract) error
• type ToMultiple
  • func ExportToMultiple(es ...Exporter) *ToMultiple
  • func (m *ToMultiple) Save(c *Contract) error
• type ToState
  • func (e *ToState) Save(c *Contract) error

Examples

• Contract
• DynamicRequester
• SimpleChecker
• SimpleRequester
• ToMMock
• ToState

Variables

var MockDelay int

MockDelay sets the sleep/timeout period after exporting a mock definition. set this to the number of milliseconds or use the `-gandalf.mock-delay` cli switch.

var MockSavePath string

Gandalf can be configured with custom flags given to the `go test` command or be setting the respective global variables.

MockSavePath tells exporters where to write generated mock should they have that functionality, eg. for mmock ingestion. use the `-gandalf.mock-dest` cli switch to specify where

var MockSkip bool

MockSkip when set to true will not write mock definitions to disk. You can also override this wth the `-gandalf.mmock-skip` cli switch.

var OverrideChaos bool

OverrideChaos enables MMock definitions support chaos testing with random 5xx responses by setting the ChaoticEvil switch in ToMMock exporters. You can also override this in all definitions with the `-gandalf.mmock-chaos` cli switch.

var OverrideColour bool

OverrideColour will force coloured cli output regardless of being a TTY or not. This can be set using the `-gandalf.colour` switch.

var OverrideHTTPS bool

OverrideHTTPS if true will make all external requests use HTTPS. This may be required when targeting a production environment. This can be done using the `-gandalf.provider-https` cli switch.

var OverrideHost string

OverrideHost rewrites the target provider api to be targeted when making real outbound via requesters that are correctly written to use this such as SimpleRequester. can be overridden globally with the `-gandalf.provider-host` cli switch.

var OverrideHostSuffix string

OverrideHostSuffix rewrites the target provider hostname. This can be useful if your contracts reference different hosts for various services, then setting OverrideHostSuffix to your dev instances domain to retarget at runtime. This can be done using the `-gandalf.provider-suffix` cli switch.

var OverrideWebroot string

OverrideWebroot gets prepended to all requests URI's. This can be useful when targeting an environment that uses webroot routing to the service to be tested. This can be done using the `-gandalf.provider-webroot` cli switch.

Functions

func BenchmarkInOrder

func BenchmarkInOrder(b *testing.B, contracts []*Contract)

BenchmarkInOrder takes a list of contracts and benchmarks the time it takes to call each of their requests in sequence before starting the next run. This may be useful if a list of contracts defined, for example, a common customer journey to be benchmarked.

func GetRequestBody

func GetRequestBody(r *http.Request) string

GetRequestBody reads the body from the request to be returned but also creates a new reader to put the body back into the response, allowing multiple reads.

func GetResponseBody

func GetResponseBody(r *http.Response) string

GetResponseBody reads the body from the response to be returned but also creates a new reader to put the body back into the response, allowing multiple reads.

func Main

func Main(m *testing.M)

Main should be run by TestMain in order for Gandalf to analyze the whole test run.

func TestMain(m *testing.M) {
  gandalf.Main(m)
}

func MainWithHandler

func MainWithHandler(m *testing.M, handler http.Handler)

MainWithHandler wraps around Main and will start listening and serving the given http.Handler (or any third party mux/router that conforms to http.Handler) on a random port to to run contracts against over the loopback network interface. This allows for code coverage reports of your server implementation when written in Go. If handler param is nil then the default Go mux will be used.

func SaneResponse

func SaneResponse() *http.Response

SaneResponse returns a new HTTP response that should be sane; it has a 200 status code, body of "A", HTTP/1.1 protocol, etc.

Types

type Checker

type Checker interface {
	// If the given response satisfies the Checker's criteria no error will be returned,
	// otherwise an error describing what check failed on the given response.
	Assert(*http.Response) error
	// Get a new response that would satisfy this Checker's criteria.
	GetResponse() *http.Response
}

A Checker is an object that can assert that a given HTTP response meets some kind of criteria. A Checker should also be able to provide an HTTP response that would meet its checks to act as as a basis for; examples, mocks, or validation.

type Contract

type Contract struct {
	// Unique identifier for this contract.
	Name    string
	Check   Checker
	Request Requester
	Export  Exporter
	// Run stores the number of times that Assert has been run and executed all parts of the contract.
	// This allows for some information such as the request to differ per call if desired.
	Run int
	// If an Optional Contract fails its checks it will not fail the whole test run.
	Optional bool
	// Set to true after this contract is tested the first time, pass or fail.
	Tested bool // internal state to mark if a contract has already been tested
	// contains filtered or unexported fields
}

Contract is at the core of Gandalf, it represents the contract between a consumer and provider in two main parts, the Request, and the Check. The Request object is responsible for geting information into Gandalf from the provider for testing. Then the response is given to the Check object to that the response meets whatever criteria the Checker supports.

Example

So you are building a web API, it will change the world, you decide your server needs to store some data given by the user and you land on a set of CRUD (Create, Read, Update, and Delete) style restful endpoints. What we will do is create a sequence of Contract's that describe the CRUD functionality before you start writing your web server code to implement it, maybe someone else wants to get started on the web client and you want to provide them a fake version of your for dev.

_ = []*Contract{
	{Name: "Read_Missing", // Start by trying to read data before anything is created.
		Request: NewSimpleRequester("GET", "http://provider/data/thing", "", nil, time.Second),
		Check: &SimpleChecker{ // Check that the body exactly matches what we expected.
			HTTPStatus:  404,  // Should 404 be cause thing has not been created.
			ExampleBody: "{}", // Body must match this since no body check is provided.
		},
		Export: &ToMMock{ // For rapid/parallel development, we output to mmock definitions.
			Scenario:      "data",                  // This is part of the data scenario.
			TriggerStates: []string{"not_started"}, // When the data scenario is in this state (the default) this definition will be used.
		},
	},

	{Name: "Create", // Create some data.
		Request: NewSimpleRequester( // POST to /data a thing of type 1.
			"POST", "http://provider/data",
			`{"name":"thing","type":1}`, // Note the type.
			nil, time.Second),
		Check: &SimpleChecker{
			HTTPStatus: 201, // 201 means we have indeed created some data.
			Headers: http.Header{
				"Content-Type": []string{"application/json"},
				"Location":     []string{"/data/thing"}, // Expect the data object to live at this endpoint.
			},
			ExampleBody: "{}",
		},
		Export: &ToMMock{
			Scenario:      "data",
			TriggerStates: []string{"not_started"},
			NewState:      "created", // if this definition is triggered change the data scenario to created.
		},
	},

	{Name: "Read_Created", // Read the data back after creating it, very similar to Read_missing.
		Request: NewSimpleRequester("GET", "http://provider/data/thing", "", nil, time.Second),
		Check: &SimpleChecker{
			HTTPStatus: 200,
			Headers: http.Header{
				"Content-Type": []string{"application/json"},
			},
			ExampleBody: `{"name":"thing","type":1}`,
			BodyCheck: p.JSONChecks(p.PathChecks{ // Here we want to extract JSON values and check them.
				"$.name+": c.Equality(`"thing"`), // Extract the value of the name field and verify it is a JSON string storing thing.
				"$.type+": c.Equality("1"),       // Extract the value of the type field type and check that it is a JSON integer of value 1.
			}),
		},
		Export: &ToMMock{
			Scenario:      "data",
			TriggerStates: []string{"created"},
		},
	},

	{Name: "Update", // Update the data.
		Request: NewSimpleRequester(
			"PUT", "http://provider/data/thing",
			`{"type":2}`, // Update just the type field.
			nil, time.Second),
		Check: &SimpleChecker{
			HTTPStatus: 201,
			Headers: http.Header{
				"Content-Type": []string{"application/json"},
			},
			ExampleBody: "{}",
		},
		Export: &ToMMock{
			Scenario:      "data",
			TriggerStates: []string{"created"},
			NewState:      "updated", // Change to this new state so that the next GET can be different to mock state.
		},
	},

	{Name: "Read_Updated", // Read the data again, very similar to previous Read_* contracts but with different values.
		Request: NewSimpleRequester("GET", "http://provider/data/thing", "", nil, time.Second),
		Check: &SimpleChecker{
			HTTPStatus: 200,
			Headers: http.Header{
				"Content-Type": []string{"application/json"},
			},
			ExampleBody: `{"name":"thing","type":2}`,
			BodyCheck: p.JSONChecks(p.PathChecks{
				"$.name+": c.Equality(`"thing"`),
				"$.type+": c.Equality("2"), // Here the value is 2, only possible after updating it from 1.
			}),
		},
		Export: &ToMMock{
			Scenario:      "data",
			TriggerStates: []string{"updated"}, // This definition will only be used when the data scenario is in the updated state.
		},
	},

	{Name: "Delete", // Now lets delete the data.
		Request: NewSimpleRequester("DELETE", "http://provider/data/thing", "", nil, time.Second),
		Check: &SimpleChecker{
			HTTPStatus: 200,
			Headers: http.Header{
				"Content-Type": []string{"application/json"},
			},
			ExampleBody: "{}",
		},
		Export: &ToMMock{
			Scenario:      "data",
			TriggerStates: []string{"updated"},
			NewState:      "not_started", // Closes the scenario loop by going back to the starting state.
		},
	},

	{Name: "Read_Deleted", // Pretty much the first contract, Read_Missing but at the end to confirm the deletion.
		Request: NewSimpleRequester("GET", "http://provider/data/thing", "", nil, time.Second),
		Check: &SimpleChecker{
			HTTPStatus: 404, // now the data is deleted it should be missing, thus 404.
			Headers: http.Header{
				"Content-Type": []string{"application/json"},
			},
			ExampleBody: "{}",
		},
		Export: &ToMMock{
			Scenario:      "data",
			TriggerStates: []string{"not_started"},
		},
	},
}

func (*Contract) Assert

func (c *Contract) Assert(t Testable)

Assert runs a request and checks response on the contract causing a pass or fail. This executes the Exporter before and after calling the Requester, allowing for pre and post exporters.

func (*Contract) Benchmark

func (c *Contract) Benchmark(b *testing.B)

Benchmark just the Requester's ability to provider responses in sequence. This uses the benchmark run counter instead of the Contract.Run field.

type DynamicRequester

type DynamicRequester struct {
	Builder func(run int) Requester
	// contains filtered or unexported fields
}

DynamicRequester allows for generating requests using a a function that creates a requester each time it is called. This is useful for requests that should change at runtime based on, for example, State values. Caches a single requester based on the run for debouncing.

Example

_ = Contract{
	Name: "SimpleContract",
	Request: &DynamicRequester{
		Builder: func(run int) Requester {
			// Each run will get a different post id
			return NewSimpleRequester("GET", fmt.Sprintf("http://provider/post/%d", run), "", nil, time.Second*5)
		},
	},
}

func (*DynamicRequester) Call

func (r *DynamicRequester) Call(run int) (*http.Response, error)

Call executes the builder (or retrieve from cache if the run is the same as the last Call execution) then Call the requester passing on the run.

func (*DynamicRequester) GetRequest

func (r *DynamicRequester) GetRequest() *http.Request

GetRequest passes down to the current Requester's GetRequest method. This uses the last run given to Call (or 0) as the run to give to the builder.

type Exporter

type Exporter interface {
	Save(contract *Contract) error
}

An Exporter represents a method of transforming and exporting the given contract.

type Requester

type Requester interface {
	Call(run int) (*http.Response, error)
	GetRequest() *http.Request
}

Requester knows how to reliably call a provider service to get a HTTP response that may later be used in a Checker. A Requester should also be able to provide an HTTP request to act as a basis for; examples, mocks, or self testing.

type SimpleChecker

type SimpleChecker struct {
	// HTTP Status code expected, ignored if left as default (0).
	HTTPStatus int
	// Assert that these headers have at least the values given. ignored if left as default.
	Headers http.Header
	// Uses a check.Func to assert the body is as expected.
	BodyCheck check.Func
	// Provide an example response body that should meet BodyCheck.
	ExampleBody string
}

SimpleChecker implements a Checker that asserts the expected HTTP status code, headers, and uses pathing.check.Func for checking the contents of the body.

Example

_ = Contract{
	Name: "SimpleCheckerContract",
	Check: &SimpleChecker{
		HTTPStatus: 200,
		Headers: http.Header{
			"Content-Type": []string{"application/json; charset=utf-8"},
		},
		ExampleBody: "{}",
		BodyCheck:   check.Equality("{}"),
	},
}

func (*SimpleChecker) Assert

func (c *SimpleChecker) Assert(res *http.Response) error

Assert the given HTTP response meets all checks. Executes methods in the following order:

1. SimpleChecker.assertStatus
2. SimpleChecker.assertHeaders
3. SimpleChecker.assertBody

func (*SimpleChecker) GetResponse

func (c *SimpleChecker) GetResponse() *http.Response

GetResponse returns a new HTTP response that should meet all checks.

type SimpleRequester

type SimpleRequester struct {
	Request *http.Request
	Timeout time.Duration
	// contains filtered or unexported fields
}

SimpleRequester implements a Requester that executes the stored Request each time.

Example

_ = Contract{
	Name:    "SimpleContract",
	Request: NewSimpleRequester("GET", "https://api.github.com", "", nil, time.Second*5),
}

func NewSimpleRequester

func NewSimpleRequester(method, url, body string, headers http.Header, timeout time.Duration) *SimpleRequester

NewSimpleRequester is a wrapper to easily create a SimpleRequester given a limited set of common inputs.

func (*SimpleRequester) Call

func (r *SimpleRequester) Call(run int) (*http.Response, error)

Call the Request. The last response is stored to be given on multiple calls for the same run.

func (*SimpleRequester) GetRequest

func (r *SimpleRequester) GetRequest() *http.Request

GetRequest SimpleRequester.Request.

type State

type State struct {
	KV map[string]interface{}
}

State is an in memory repository that can be used to perform stateful requests and response checks. This uses a thread safe singleton pattern and should not be instantiated anywhere other than GetState.

func GetState

func GetState() *State

GetState return the thread safe global State instance.

func (*State) Clear

func (s *State) Clear()

Clear wipes all state clean.

func (*State) ClearKey

func (s *State) ClearKey(key string)

ClearKey wipes a single key.

func (*State) ClearRegex

func (s *State) ClearRegex(expr string) error

ClearRegex wipes all keys that match expr.

func (*State) GetResponse

func (s *State) GetResponse(key string) *http.Response

GetResponse returns data stored at "{{Key}}.response".

type Testable

type Testable interface {
	Helper()
	Fatalf(format string, args ...interface{})
	Skipf(format string, args ...interface{})
}

Testable is the common interface between tests and benchmarks required to handle them interchangeably.

type ToMMock

type ToMMock struct {
	// The state(s) that the Scenario must be in to trigger this mock.
	TriggerStates []string
	// The Scenario to which state is stored.
	Scenario string
	// The state to transition the scenario to when this mock is triggered.
	NewState string
	// When set this is used for the request path definition instead of the path from the Contract's Requestor.
	Path string
	// Enables chaos testing by causing the mock, when triggered, may return a 5xx instead.
	ChaoticEvil bool
	// If true MMock will require the request headers to match exactly to trigger this mock.
	// This should be left false (the default ) for dynamic headers such as tokens/id's.
	MatchHeaders bool
	// If true MMock will require the request body to match exactly to trigger this mock.
	// This should be left false (the default ) for dynamic requests such as tokens/id's.
	MatchBody bool
	// contains filtered or unexported fields
}

ToMMock exports Contract as MMock definitions to build a fake api endpoint with optional state via MMock scenarios. MMock (https://github.com/jmartin82/mmock) is an http mocking server.

Example

_ = &Contract{
	Name: "MMockContract",
	Export: &ToMMock{
		Scenario:      "happy_path",
		TriggerStates: []string{"not_started"},
		NewState:      "started",
		ChaoticEvil:   true,
	},
}

func (*ToMMock) Save

func (m *ToMMock) Save(c *Contract) error

Save a valid MMock definition to a json file with the contract name as the filename. This incurs disk IO so is restricted to only saving once per instance.

type ToMultiple

type ToMultiple struct {
	Exporters []Exporter
}

ToMultiple allows using multiple Exporter structs in one contract.

func ExportToMultiple

func ExportToMultiple(es ...Exporter) *ToMultiple

ExportToMultiple is a convenience function for creating a ToMultiple.

func (*ToMultiple) Save

func (m *ToMultiple) Save(c *Contract) error

Save loops through Exporters and gives the Contract to each Save method, stopping on the first error.

type ToState

type ToState struct {
	Key string
	// contains filtered or unexported fields
}

ToState is an exporter that will store the response for later usage.

Example

_ = []*Contract{
	{Name: "ExampleStateContractRetrieve",
		Request: NewSimpleRequester("GET", "http://provider/token", "", nil, time.Second),
		Check: &SimpleChecker{
			HTTPStatus: 200,
		},
		Export: &ToState{
			Key: "ExampleStateContract",
		}},
	{Name: "ExampleStateContractUse",
		Request: &DynamicRequester{
			Builder: func(_ int) Requester {
				// get a token from the body of the last contract's response.
				body := GetResponseBody(GetState().GetResponse("ExampleStateContract"))
				found, err := pathing.GJSON(body, "result.token")
				if err != nil || len(found) == 0 {
					panic("Could not get the token from previous ExampleStateContract response")
				}
				token := found[0]
				// use the token in the header to get protected information
				return NewSimpleRequester("GET", "http://provider/info", "",
					http.Header{"Authorization": {"Bearer " + token}}, time.Second)
			},
		},
		Check: &SimpleChecker{
			HTTPStatus: 200,
		}},
}

func (*ToState) Save

func (e *ToState) Save(c *Contract) error

Save the response of the current Requester run to a key in State.KV of the format (go tmpl style) "{{ ToState.Key }}.response" each time. It is expected that the Requester implements debouncing/caching so that Requester.Call can be rexecuted in the same run.