testingfiles

README

Using reference files for large output

A want reference file is compared to data from a got source. Comparison is provided for File, Buffer or ReadCloser where a file is the least efficient.

When comparison fails, a file is created with got_ prefix from the byte where the first difference appeared. No further check on the file is done.

When running tests for the first time, they might fail as no want file is usually available. The produced got file can be renamed into a want file to have a second successful run.

Offline test

func TestOffline(t *testing.T) {
	if err := tearDownOffline(handler(...), t.Name()); err != nil {
    		t.Error(err)
    }
}

func tearDownOffline(b *bytes.Buffer, s string) (err error) {
	if b == nil {
		return errors.New("bytes.Buffer is nil")
	}
	testingfiles.OutputDir("output")
	
	if err := testingfiles.BufferCompare(b, s); err != nil {
        return err
    }
    return nil
}

Online test

func TestOnline(t *testing.T) {
	resp, err := http.Get(getAppUrl("").String())
	if err != nil {
		t.Fatal(err)
	}
	tearDown(t, resp)
}

func tearDown(t *testing.T, resp *http.Response) {
	if resp == nil {
		t.Fatal("response is nil")
	}
	if resp.StatusCode != 200 {
		t.Fatalf("request failed with error %d for %s", resp.StatusCode, s)
	}
	testingfiles.OutputDir("output")
	
	if err := testingfiles.ReadCloserCompare(resp.Body, t.Bame()); err != nil {
        t.Error(err)
    }
}

Working directory

Reference files are expected to reside in a working directory which defaults to ./output. Using a subdirectory avoids having the data files mixed with source code. The directory is not created but its existence is checked. If the working directory is unavailable, tests will panic. In CI scripts, the working directory is created before running the tests.

Testing of the module

Testing can be online or offline.

Online is used to read a reference page. Offline requires to provide the reference file.

Testing usage is demonstrated in modules of largeoutput repository.

Benchmarking between string and bytes.Buffer is inconclusive inline with documented behavior.

go version go1.13beta1 windows/amd64
pkg: github.com/iwdgo/testingfiles
BenchmarkGetPageStringToFile-4                 1        1029407400 ns/op
BenchmarkGetPageBufferToFile-4                 1        1108362700 ns/op
BenchmarkGetPageBufferCompare-4                2         512716000 ns/op
BenchmarkGetPageReadCloserCompare-4            3         545020000 ns/op

Documentation

Overview

Package testingfiles provides primitives to use files as reference for testing

Index

• func BufferCompare(got *bytes.Buffer, want string) error
• func BufferToFile(fname string, content *bytes.Buffer)
• func FileCompare(got, want string) error
• func OutputDir(s string)
• func ReadCloserCompare(got io.ReadCloser, want string) error
• func ReadCloserToFile(fname string, content io.ReadCloser) error
• func StringToFile(fname string, content []byte)

Functions

func BufferCompare

func BufferCompare(got *bytes.Buffer, want string) error

BufferCompare compares the buffer to a file. If an error occurs, got file is created and the error is returned. First char in the got file is the erroneous char. If identical, nil is returned. First byte index is 0

func BufferToFile

func BufferToFile(fname string, content *bytes.Buffer)

BufferToFile produces a file named fname with the content

func FileCompare

func FileCompare(got, want string) error

FileCompare checks large outputs of a test when a file storage is more convenient or required. Names of the files to compare are passed as arguments and searched in the working directory.

func OutputDir

func OutputDir(s string)

OutputDir changes the default dir to the folder where reference files (want files) are stored. Only the base of the directory is expected. If found, change default directory to it. When not found, check if ../test contains the folder.

func ReadCloserCompare

func ReadCloserCompare(got io.ReadCloser, want string) error

ReadCloserCompare compares a ReadCloser to a file. If an error occurs, got file is created and the error is returned. Last read byte is absent from the got file but available in the error message. If identical, nil is returned. Logic and method are identical to *buffer.Bytes but duplicating the code avoids ReadAll. First byte index is 0 TODO Benchmark ReadAll against specific byte by byte code

func ReadCloserToFile

func ReadCloserToFile(fname string, content io.ReadCloser) error

ReadCloserToFile creates a file named fname with the content

func StringToFile

func StringToFile(fname string, content []byte)

StringToFile produces a file named fname with the content