hypert

README

Hypert - HTTP API Testing Made Easy

Hypert is an open-source Go library that simplifies testing of HTTP API clients. It provides a convenient way to record and replay HTTP interactions, making it easy to create reliable and maintainable tests for your API clients.

Features

• Record and replay HTTP interactions
• Request sanitization to remove sensitive information
• Request validation to ensure the integrity of recorded requests
• Seamless integration with Go's http.Client
• Extensible and configurable options

Getting Started

1. Install Hypert:

go get github.com/areknoster/hypert

1. Use hypert.TestClient to create an http.Client instance for testing:

func TestMyAPI(t *testing.T) {
	httpClient := hypert.TestClient(t, true) // true to record real requests
	// Use the client to make API requests. 
	// The requests and responses would be stored in ./testdata/TestMyAPI
	myAPI := NewMyAPI(httpClient, os.GetEnv("API_SECRET")) 
	// Make an API request with your adapter.
	// use static arguments, so that validation against recorded requests can happen
	stuff, err := myAPI.GetStuff(time.Date(2022, 1, 1, 0, 0, 0, 0, time.UTC)) 
	if err != nil {
		t.Fatalf("failed to get stuff: %v", err)
	}
    // Assertions on the actual API response
	if stuff.ID != "ID-FROM-RESP" {
		t.Errorf("stuff ")
	}
}

After you're done with building and testing your integration, change the mode to replay

func TestMyAPI(t *testing.T) {
    httpClient := hypert.TestClient(t, false) // false to replay stored requests
    // Now client would validate requests against what's stored in ./testdata/TestMyAPI/*.req.http 
    // and load the response from  ./testdata/TestMyAPI/*.resp.http
    myAPI := NewMyAPI(httpClient, os.GetEnv("API_SECRET"))
    // HTTP requests are validated against what was prevously recorded. 
    // This behaviour can be customized using WithRequestValidator option
    stuff, err := myAPI.GetStuff(time.Date(2022, 1, 1, 0, 0, 0, 0, time.UTC)) 
    if err != nil {
        t.Fatalf("failed to get stuff: %v", err)
    }
    // Same assertions that were true on actual API responses should be true for replayed API responses.
	if stuff.ID != "ID-FROM-RESP" {
        t.Errorf("stuff ")
    }
}

Now your tests:

• are deterministic
• are fast
• bring the same confidence as integration tests

Stability

I plan to maintain backward compatibility as much as possible, but breaking changes may occur before the first stable release, v1.0.0 if major issues are discovered.

Examples

Check out the examples directory for sample usage of Hypert in different scenarios.

Contributing

Contributions are welcome! If you find a bug or have a feature request, please open an issue on the GitHub repository. If you'd like to contribute code, please fork the repository and submit a pull request.

License

Hypert is released under the MIT License.

---

Documentation

Overview

Package hypert enables rapid testing of real HTTP APIs integrations. It is designed to be used in the tests of components, that rely on go standard library http.Client to integrate with external services.

Quickstart: inject TestClient to a component that makes HTTP requests and see what happens.

See examples directory for real-world samples of its use.

Index

• func DefaultTestdataDir(t *testing.T) string
• func TestClient(t T, recordModeOn bool, opts ...Option) *http.Client
• type NamingScheme
• type Option
  • func WithNamingScheme(n NamingScheme) Option
  • func WithParentHTTPClient(c *http.Client) Option
  • func WithRequestSanitizer(sanitizer RequestSanitizer) Option
  • func WithRequestValidator(v RequestValidator) Option
• type RequestData
  • func (r RequestData) String() string
• type RequestSanitizer
  • func ComposedRequestSanitizer(s ...RequestSanitizer) RequestSanitizer
  • func DefaultHeadersSanitizer() RequestSanitizer
  • func DefaultQueryParamsSanitizer() RequestSanitizer
  • func DefaultRequestSanitizer() RequestSanitizer
  • func HeadersSanitizer(headers ...string) RequestSanitizer
  • func SanitizerQueryParams(params ...string) RequestSanitizer
• type RequestSanitizerFunc
  • func (f RequestSanitizerFunc) SanitizeRequest(req *http.Request) *http.Request
• type RequestValidator
  • func ComposedRequestValidator(validators ...RequestValidator) RequestValidator
  • func DefaultRequestValidator() RequestValidator
  • func HeadersValidator() RequestValidator
  • func MethodValidator() RequestValidator
  • func PathValidator() RequestValidator
  • func QueryParamsValidator() RequestValidator
  • func SchemeValidator() RequestValidator
• type RequestValidatorFunc
  • func (f RequestValidatorFunc) Validate(t T, recorded RequestData, got RequestData)
• type SequentialNamingScheme
  • func NewSequentialNamingScheme(dir string) (*SequentialNamingScheme, error)
  • func (s *SequentialNamingScheme) FileNames(_ RequestData) (reqFile, respFile string)
• type T

Functions

func DefaultTestdataDir

func DefaultTestdataDir(t *testing.T) string

DefaultTestdataDir returns fully qualified directory name following <your package directory>/testdata/<name of the test> convention.

Note, that it relies on runtime.Caller function with given skip stack number. Because of that, usually you'd want to call this function directly in a file that belongs to a directory that the test data directory should be placed in.

func TestClient

func TestClient(t T, recordModeOn bool, opts ...Option) *http.Client

TestClient returns a new http.Client that will either dump requests to the given dir or read them from it. It is the main entrypoint for using the hypert's capabilities. It provides sane defaults, that can be overwritten using Option arguments. Option's can be initialized using With* functions.

The returned *http.Client should be injected to given component before the tests are run.

In most scenarios, you'd set recordModeOn to true during the development, when you have set up the authentication to the HTTP API you're using. This will result in the requests and response pairs being stored in <package name>/testdata/<test name>/<sequential number>.(req|resp).http Before the requests are stored, they are sanitized using DefaultRequestSanitizer. It can be adjusted using WithRequestSanitizer option. Ensure that sanitization works as expected, otherwise sensitive details might be committed

recordModeOn should be false when given test is not actively worked on, so in most cases the committed value should be false. This mode will result in the requests and response pairs previously stored being replayed, mimicking interactions with actual HTTP APIs, but skipping making actual calls.

Types

type NamingScheme

type NamingScheme interface {
	FileNames(RequestData) (reqFile, respFile string)
}

NamingScheme defines an interface that is used by hypert's test client to store or retrieve files with HTTP requests.

FileNames returns a pair of filenames that request and response should be stored in, when Record Mode is active, and retrieved from when Replay Mode is active.

File names should be fully qualified (not relative).

Each invocation should yield a pair of file names that wasn't yielded before during lifetime of given hypert's http client.

This method should be safe for concurrent use. This requirement can be skipped, if you are the user of the package, and know, that all invocations would be sequential.

type Option

type Option func(*config)

Option can be used to customize TestClient behaviour. See With* functions to find customization options

func WithNamingScheme

func WithNamingScheme(n NamingScheme) Option

WithNamingScheme allows user to set the naming scheme for the recorded requests. By default, the naming scheme is set to SequentialNamingScheme.

func WithParentHTTPClient

func WithParentHTTPClient(c *http.Client) Option

WithParentHTTPClient allows user to set the custom parent http client. TestClient's will use passed client's transport in record mode to make actual HTTP calls.

func WithRequestSanitizer

func WithRequestSanitizer(sanitizer RequestSanitizer) Option

WithRequestSanitizer configures RequestSanitizer. You may consider using RequestSanitizerFunc, ComposedRequestSanitizer, NoopRequestSanitizer, SanitizerQueryParams, HeadersSanitizer helper functions to compose sanitization rules or implement your own, custom sanitizer.

func WithRequestValidator

func WithRequestValidator(v RequestValidator) Option

WithRequestValidator allows user to set the request validator.

type RequestData

type RequestData struct {
	Headers   http.Header
	URL       *url.URL
	Method    string
	BodyBytes []byte
}

RequestData is some data related to the request, that can be used to create filename in the NamingScheme's FileNames method implementations or during request validation. The fields are cloned from request's fields and their modification will not affect actual request's values.

func (RequestData) String

func (r RequestData) String() string

type RequestSanitizer

type RequestSanitizer interface {
	SanitizeRequest(req *http.Request) *http.Request
}

RequestSanitizer ensures, that no sensitive data is written to the request records. The sanitized version would be stored, whilst the original one would be sent in the record mode. It is allowed to mutate the request in place, because it is copied before invoking the RoundTrip method.

func ComposedRequestSanitizer

func ComposedRequestSanitizer(s ...RequestSanitizer) RequestSanitizer

ComposedRequestSanitizer is a sanitizer that sequentially runs passed sanitizers.

func DefaultHeadersSanitizer

func DefaultHeadersSanitizer() RequestSanitizer

DefaultHeadersSanitizer is HeadersSanitizer with the most common headers that should be sanitized in most cases.

func DefaultQueryParamsSanitizer

func DefaultQueryParamsSanitizer() RequestSanitizer

DefaultQueryParamsSanitizer is SanitizerQueryParams with with the most common query params that should be sanitized in most cases.

func DefaultRequestSanitizer

func DefaultRequestSanitizer() RequestSanitizer

DefaultRequestSanitizer returns a RequestSanitizer that sanitizes headers and query parameters.

func HeadersSanitizer

func HeadersSanitizer(headers ...string) RequestSanitizer

HeadersSanitizer sets listed headers to "SANITIZED". Lookup DefaultHeadersSanitizer for a default value.

func SanitizerQueryParams

func SanitizerQueryParams(params ...string) RequestSanitizer

SanitizerQueryParams sets listed query params in stored request URL to SANITIZED value. Lookup DefaultQueryParamsSanitizer for a default value.

type RequestSanitizerFunc

type RequestSanitizerFunc func(req *http.Request) *http.Request

RequestSanitizerFunc is a helper type for a function that implements RequestSanitizer interface.

func (RequestSanitizerFunc) SanitizeRequest

func (f RequestSanitizerFunc) SanitizeRequest(req *http.Request) *http.Request

type RequestValidator

type RequestValidator interface {
	Validate(t T, recorded RequestData, got RequestData)
}

RequestValidator does assertions, that allows to make assertions on request that was caught by TestClient in the replay mode.

func ComposedRequestValidator

func ComposedRequestValidator(validators ...RequestValidator) RequestValidator

func DefaultRequestValidator

func DefaultRequestValidator() RequestValidator

func HeadersValidator

func HeadersValidator() RequestValidator

HeadersValidator validates headers of the request. It is not sensitive to the order of headers. User-Agent is removed from the comparison, because it is added deeper in the http client call.

func MethodValidator

func MethodValidator() RequestValidator

MethodValidator validates the method of the request.

func PathValidator

func PathValidator() RequestValidator

func QueryParamsValidator

func QueryParamsValidator() RequestValidator

QueryParamsValidator validates query parameters of the request. It is not sensitive to the order of query parameters.

func SchemeValidator

func SchemeValidator() RequestValidator

SchemeValidator validates the scheme of the request.

type RequestValidatorFunc

type RequestValidatorFunc func(t T, recorded RequestData, got RequestData)

func (RequestValidatorFunc) Validate

func (f RequestValidatorFunc) Validate(t T, recorded RequestData, got RequestData)

type SequentialNamingScheme

type SequentialNamingScheme struct {
	// contains filtered or unexported fields
}

SequentialNamingScheme should be initialized using NewSequentialNamingScheme function. It names the files following (<dir>/0.req.http, <dir>/1.resp.http), (<dir>/1.req.http, <dir>/1.resp.http) convention.

func NewSequentialNamingScheme

func NewSequentialNamingScheme(dir string) (*SequentialNamingScheme, error)

NewSequentialNamingScheme initializes SequentialNamingScheme naming scheme, that implements NamingScheme interface.

'dir' parameter indicates, in which directory should the sequential requests and responses be placed. The directory is created with 0760 permissions if doesn't exists. You can use DefaultTestdataDir function for a sane default directory name.

func (*SequentialNamingScheme) FileNames

func (s *SequentialNamingScheme) FileNames(_ RequestData) (reqFile, respFile string)

type T

type T interface {
	Helper()
	Name() string
	Log(args ...any)
	Logf(format string, args ...any)
	Error(args ...any)
	Errorf(format string, args ...any)
	Fatal(args ...any)
	Fatalf(format string, args ...any)
}

T is a subset of testing.T interface that is used by hypert's functions. custom T's implementation can be used to e.g. make logs silent, stop failing on errors and others.