tests

package
v0.0.0-...-0ae59ff Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 20, 2020 License: MIT Imports: 27 Imported by: 0

README

Server Integration Tests

This directory contains integration tests for the database.

To run them using an in-process local server:

go test ./tests

They can also be run against a remote server running in a separate process or machine

URL=http://127.0.0.1:8086 go test -parallel 1 ./tests

When running tests against a remote server, -parallel 1 is currently needed as many of the tests use the same DB and RP names which causes tests to fail when run concurrently.

When adding tests, try to add tests that will always work for remote server usage.

Structure

Currently, the file server_test.go has integration tests for single node scenarios. At some point we'll need to add cluster tests, and may add them in a different file, or rename server_test.go to server_single_node_test.go or something like that.

What is in a test?

Each test is broken apart effectively into the following areas:

  • Write sample data
  • Use cases for table driven test, that include a command (typically a query) and an expected result.

When each test runs it does the following:

  • init: determines if there are any writes and if so, writes them to the in-memory database
  • queries: iterate through each query, executing the command, and comparing the results to the expected result.

Idempotent - Allows for parallel tests

Each test should be idempotent, meaning that its data will not be affected by other tests, or use cases within the table tests themselves. This allows for parallel testing, keeping the test suite total execution time very low.

Basic sample test
// Ensure the server can have a database with multiple measurements.
func TestServer_Query_Multiple_Measurements(t *testing.T) {
    t.Parallel()
    s := OpenServer(NewConfig(), "")
    defer s.Close()

    if err := s.CreateDatabaseAndRetentionPolicy("db0", newRetentionPolicyInfo("rp0", 1, 1*time.Hour)); err != nil {
        t.Fatal(err)
    }

    // Make sure we do writes for measurements that will span across shards
    writes := []string{
        fmt.Sprintf("cpu,host=server01 value=100,core=4 %d", mustParseTime(time.RFC3339Nano, "2000-01-01T00:00:00Z").UnixNano()),
        fmt.Sprintf("cpu1,host=server02 value=50,core=2 %d", mustParseTime(time.RFC3339Nano, "2015-01-01T00:00:00Z").UnixNano()),
    }
    test := NewTest("db0", "rp0")
    test.write = strings.Join(writes, "\n")

    test.addQueries([]*Query{
        &Query{
            name:    "measurement in one shard but not another shouldn't panic server",
            command: `SELECT host,value  FROM db0.rp0.cpu`,
            exp:     `{"results":[{"series":[{"name":"cpu","tags":{"host":"server01"},"columns":["time","value"],"values":[["2000-01-01T00:00:00Z",100]]}]}]}`,
        },
    }...)

    if err := test.init(s); err != nil {
        t.Fatalf("test init failed: %s", err)
    }

    for _, query := range test.queries {
        if query.skip {
            t.Logf("SKIP:: %s", query.name)
            continue
        }
        if err := query.Execute(s); err != nil {
            t.Error(query.Error(err))
        } else if !query.success() {
            t.Error(query.failureMessage())
        }
    }
}

Let's break this down:

In this test, we first tell it to run in parallel with the t.Parallel() call.

We then open a new server with:

s := OpenServer(NewConfig(), "")
defer s.Close()

If needed, we create a database and default retention policy. This is usually needed when inserting and querying data. This is not needed if you are testing commands like CREATE DATABASE, SHOW DIAGNOSTICS, etc.

if err := s.CreateDatabaseAndRetentionPolicy("db0", newRetentionPolicyInfo("rp0", 1, 1*time.Hour)); err != nil {
    t.Fatal(err)
}

Next, set up the write data you need:

writes := []string{
    fmt.Sprintf("cpu,host=server01 value=100,core=4 %d", mustParseTime(time.RFC3339Nano, "2000-01-01T00:00:00Z").UnixNano()),
    fmt.Sprintf("cpu1,host=server02 value=50,core=2 %d", mustParseTime(time.RFC3339Nano, "2015-01-01T00:00:00Z").UnixNano()),
}

Create a new test with the database and retention policy:

test := NewTest("db0", "rp0")

Send in the writes:

test.write = strings.Join(writes, "\n")

Add some queries (the second one is mocked out to show how to add more than one):

test.addQueries([]*Query{
    &Query{
        name:    "measurement in one shard but not another shouldn't panic server",
        command: `SELECT host,value  FROM db0.rp0.cpu`,
        exp:     `{"results":[{"series":[{"name":"cpu","tags":{"host":"server01"},"columns":["time","value"],"values":[["2000-01-01T00:00:00Z",100]]}]}]}`,
    },
    &Query{
        name:    "another test here...",
        command: `Some query command`,
        exp:     `the expected results`,
    },
}...)

The rest of the code is boilerplate execution code. It is purposefully not refactored out to a helper to make sure the test failure reports the proper lines for debugging purposes.

Running the tests

To run the tests:

go test ./cmd/influxd/run -parallel 500 -timeout 10s
Running a specific test
go test ./cmd/influxd/run -parallel 500 -timeout 10s -run TestServer_Query_Fill
Verbose feedback

By default, all logs are silenced when testing. If you pass in the -v flag, the test suite becomes verbose, and enables all logging in the system

go test ./cmd/influxd/run -parallel 500 -timeout 10s -run TestServer_Query_Fill -v

Documentation

Overview

This package is a set of convenience helpers and structs to make integration testing easier

Index

Constants

View Source 
const (
	EMPTY = "empty"
	RUN   = "run"
	STOP  = "stop"
)

Variables

View Source 
var LosAngeles = mustParseLocation("America/Los_Angeles")

Functions

func Addr 

func Addr(host string, port int) string

func CleanCluster 

func CleanCluster()

func DumpConfig 

func DumpConfig(c interface{}, path string) error

func HTTPPost 

func HTTPPost(url string, content []byte) (results string, err error)

func MustReadAll 

func MustReadAll(r io.Reader) []byte

MustReadAll reads r. Panic on error.

func NewRetentionPolicySpec 

func NewRetentionPolicySpec(name string, rf int, duration time.Duration) *meta.RetentionPolicySpec

form a correct retention policy given name, replication factor and duration

func RemoteEnabled 

func RemoteEnabled() bool

func SplitAddr 

func SplitAddr(addr string) (host string, port int)

Types

type ClusterManager 

type ClusterManager struct {
	DataServerNumber int //data server number
	MetaServerNumber int //meta server number
	Config           *Config

	Status string
	// contains filtered or unexported fields
}

func NewClusterManager 

func NewClusterManager(c *Config, size int) *ClusterManager

func OpenCluster 

func OpenCluster(c *Config, size int) *ClusterManager

func (*ClusterManager) Clean 

func (me *ClusterManager) Clean() error

func (*ClusterManager) Close 

func (me *ClusterManager) Close() error

func (*ClusterManager) DataServerById 

func (me *ClusterManager) DataServerById(id uint64) *DataServer

func (*ClusterManager) Open 

func (me *ClusterManager) Open() error

func (*ClusterManager) SelectOneDataServer 

func (me *ClusterManager) SelectOneDataServer() *DataServer

func (*ClusterManager) WaitDataServersRun 

func (me *ClusterManager) WaitDataServersRun()

type Config 

type Config struct {
	*run.Config
	// contains filtered or unexported fields
}

Config is a test wrapper around a run.Config. It also contains a root temp directory, making cleanup easier.

func NewConfig 

func NewConfig() *Config

NewConfig returns the default config with temporary paths.

func NewDefaultConfig 

func NewDefaultConfig() *Config

type DataServer 

type DataServer struct {
	Config  *Config
	BinPath string
	Status  string
	ID      uint64
	// contains filtered or unexported fields
}

func NewDataServer 

func NewDataServer(cm *ClusterManager, c *Config, NodeID uint64, binPath string) *DataServer

func OpenServer 

func OpenServer(c *Config) *DataServer

OpenServer opens a test server.

func (*DataServer) Clean 

func (me *DataServer) Clean()

func (*DataServer) Close 

func (me *DataServer) Close()

func (*DataServer) Closed 

func (me *DataServer) Closed() bool

func (*DataServer) CreateDatabase 

func (me *DataServer) CreateDatabase(db string) (*meta.DatabaseInfo, error)

func (*DataServer) CreateDatabaseAndRetentionPolicy 

func (me *DataServer) CreateDatabaseAndRetentionPolicy(db string, rp *meta.RetentionPolicySpec, makeDefault bool) error

func (*DataServer) CreateSubscription 

func (me *DataServer) CreateSubscription(database, rp, name, mode string, destinations []string) error

func (*DataServer) DropDatabase 

func (me *DataServer) DropDatabase(db string) error

func (*DataServer) MustWrite 

func (me *DataServer) MustWrite(db, rp, body string, params url.Values) string

func (*DataServer) NodeID 

func (me *DataServer) NodeID() uint64

func (*DataServer) Open 

func (me *DataServer) Open() error

func (*DataServer) Query 

func (me *DataServer) Query(query string) (string, error)

func (*DataServer) QueryWithParams 

func (me *DataServer) QueryWithParams(query string, values url.Values) (string, error)

func (*DataServer) Reset 

func (me *DataServer) Reset() error

func (*DataServer) Restart 

func (me *DataServer) Restart()

func (*DataServer) SetLogOutput 

func (me *DataServer) SetLogOutput(w io.Writer)

func (*DataServer) URL 

func (me *DataServer) URL() string

func (*DataServer) WaitRun 

func (me *DataServer) WaitRun()

func (*DataServer) Write 

func (me *DataServer) Write(db, rp, body string, params url.Values) (string, error)

func (*DataServer) WritePoints 

func (me *DataServer) WritePoints(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, user meta.User, points []models.Point) error

type MetaServer 

type MetaServer struct {
	Config   raftmeta.Config
	BinPath  string
	RootPath string
	ID       uint64

	Status string
	// contains filtered or unexported fields
}

func NewMetaServer 

func NewMetaServer(c raftmeta.Config, id uint64, rootPath, binPath string) *MetaServer

func (*MetaServer) Clean 

func (me *MetaServer) Clean() error

func (*MetaServer) Close 

func (me *MetaServer) Close()

func (*MetaServer) Open 

func (me *MetaServer) Open() error

func (*MetaServer) Restart 

func (me *MetaServer) Restart()

func (*MetaServer) WaitRun 

func (me *MetaServer) WaitRun()

type Query 

type Query struct {
	// contains filtered or unexported fields
}

func (*Query) Error 

func (q *Query) Error(err error) string

func (*Query) Execute 

func (q *Query) Execute(s Server) (err error)

Execute runs the command and returns an err if it fails

type RemoteServer 

type RemoteServer struct {
	// contains filtered or unexported fields
}

RemoteServer is a Server that is accessed remotely via the HTTP API

func (*RemoteServer) Close 

func (s *RemoteServer) Close()

func (*RemoteServer) Closed 

func (s *RemoteServer) Closed() bool

func (*RemoteServer) CreateDatabase 

func (s *RemoteServer) CreateDatabase(db string) (*meta.DatabaseInfo, error)

func (*RemoteServer) CreateDatabaseAndRetentionPolicy 

func (s *RemoteServer) CreateDatabaseAndRetentionPolicy(db string, rp *meta.RetentionPolicySpec, makeDefault bool) error

func (*RemoteServer) CreateSubscription 

func (s *RemoteServer) CreateSubscription(database, rp, name, mode string, destinations []string) error

func (*RemoteServer) DropDatabase 

func (s *RemoteServer) DropDatabase(db string) error

func (RemoteServer) HTTPGet 

func (s RemoteServer) HTTPGet(url string) (results string, err error)

HTTPGet makes an HTTP GET request to the server and returns the response.

func (RemoteServer) HTTPPost 

func (s RemoteServer) HTTPPost(url string, content []byte) (results string, err error)

HTTPPost makes an HTTP POST request to the server and returns the response.

func (RemoteServer) MustQuery 

func (s RemoteServer) MustQuery(query string) string

MustQuery executes a query against the server and returns the results.

func (RemoteServer) MustQueryWithParams 

func (s RemoteServer) MustQueryWithParams(query string, values url.Values) string

MustQueryWithParams executes a query against the server and returns the results.

func (RemoteServer) MustWrite 

func (s RemoteServer) MustWrite(db, rp, body string, params url.Values) string

MustWrite executes a write to the server. Panic on error.

func (*RemoteServer) Open 

func (s *RemoteServer) Open() error

func (RemoteServer) Query 

func (s RemoteServer) Query(query string) (results string, err error)

Query executes a query against the server and returns the results.

func (RemoteServer) QueryWithParams 

func (s RemoteServer) QueryWithParams(query string, values url.Values) (results string, err error)

Query executes a query against the server and returns the results.

func (*RemoteServer) Reset 

func (s *RemoteServer) Reset() error

Reset attempts to remove all database state by dropping everything

func (*RemoteServer) SetLogOutput 

func (s *RemoteServer) SetLogOutput(w io.Writer)

func (*RemoteServer) URL 

func (s *RemoteServer) URL() string

func (RemoteServer) Write 

func (s RemoteServer) Write(db, rp, body string, params url.Values) (results string, err error)

Write executes a write against the server and returns the results.

func (*RemoteServer) WritePoints 

func (s *RemoteServer) WritePoints(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, user meta.User, points []models.Point) error

type Server 

type Server interface {
	URL() string
	Open() error
	SetLogOutput(w io.Writer)
	Close()
	Closed() bool

	CreateDatabase(db string) (*meta.DatabaseInfo, error)
	CreateDatabaseAndRetentionPolicy(db string, rp *meta.RetentionPolicySpec, makeDefault bool) error
	CreateSubscription(database, rp, name, mode string, destinations []string) error
	DropDatabase(db string) error
	Reset() error

	Query(query string) (results string, err error)
	QueryWithParams(query string, values url.Values) (results string, err error)

	Write(db, rp, body string, params url.Values) (results string, err error)
	MustWrite(db, rp, body string, params url.Values) string
	WritePoints(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, user meta.User, points []models.Point) error
}

Server represents a test wrapper for run.Server.

func OpenDefaultServer 

func OpenDefaultServer(c *Config) Server

OpenDefaultServer opens a test server with a default database & retention policy.

type Test 

type Test struct {
	// contains filtered or unexported fields
}

func NewTest 

func NewTest(db, rp string) Test

type Tests 

type Tests map[string]Test

type Write 

type Write struct {
	// contains filtered or unexported fields
}

type WriteError 

type WriteError struct {
	// contains filtered or unexported fields
}

func (WriteError) Body 

func (wr WriteError) Body() string

func (WriteError) Error 

func (wr WriteError) Error() string

func (WriteError) StatusCode 

func (wr WriteError) StatusCode() int

type Writes 

type Writes []*Write

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.