testbed

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 16, 2020 License: Apache-2.0 Imports: 47 Imported by: 0

Documentation

Overview

Package testbed allows to easily set up a test that requires running the agent and a load generator, measure and define resource consumption expectations for the agent, fail tests automatically when expectations are exceeded.

Each test case requires a agent configuration file and (optionally) load generator spec file. Test cases are defined as regular Go tests.

Agent and load generator must be pre-built and their paths must be specified in test bed config file. The config file location must be provided in TESTBED_CONFIG env variable.

Index

Constants

View Source
const DefaultJaegerPort = 14250
View Source
const DefaultOCPort = 56565
View Source
const DefaultOTLPPort = 55680
View Source
const DefaultZipkinAddressPort = 9411

Variables

View Source
var ErrSkipTests = errors.New("skip tests")

ErrSkipTests indicates that the tests must be skipped.

Functions

func DoTestMain

func DoTestMain(m *testing.M)

DoTestMain is intended to be run from TestMain somewhere in the test suit. This enables the testbed.

func GenerateSpanID

func GenerateSpanID(id uint64) []byte

func GenerateTraceID

func GenerateTraceID(id uint64) []byte

func GetAvailablePort

func GetAvailablePort(t *testing.T) int

func LoadConfig

func LoadConfig() error

LoadConfig loads test bed config.

func SaveResults

func SaveResults()

func Start

func Start() error

Types

type DataReceiver

type DataReceiver interface {
	Start(tc *MockTraceConsumer, mc *MockMetricConsumer) error
	Stop()

	// Generate a config string to place in exporter part of collector config
	// so that it can send data to this receiver.
	GenConfigYAMLStr() string

	// Return protocol name to use in collector config pipeline.
	ProtocolName() string
}

DataReceiver allows to receive traces or metrics. This is an interface that must be implemented by all protocols that want to be used in MockBackend. Note the terminology: testbed.DataReceiver is something that can listen and receive data from Collector and the corresponding entity in the Collector that sends this data is an exporter.

type DataReceiverBase

type DataReceiverBase struct {
	// Port on which to listen.
	Port int
}

DataReceiverBase implement basic functions needed by all receivers.

func (*DataReceiverBase) GetExporters

func (*DataReceiverBase) GetExtensions

Return map of extensions. Only enabled and created extensions will be returned.

func (*DataReceiverBase) GetFactory

func (mb *DataReceiverBase) GetFactory(kind component.Kind, componentType configmodels.Type) component.Factory

GetFactory of the specified kind. Returns the factory for a component type.

func (*DataReceiverBase) ReportFatalError

func (mb *DataReceiverBase) ReportFatalError(err error)

type DataSender

type DataSender interface {
	// Start sender and connect to the configured endpoint. Must be called before
	// sending data.
	Start() error

	// Send any accumulated data.
	Flush()

	// Return the port to which this sender will send data.
	GetCollectorPort() int

	// Generate a config string to place in receiver part of collector config
	// so that it can receive data from this sender.
	GenConfigYAMLStr() string

	// Return protocol name to use in collector config pipeline.
	ProtocolName() string
}

DataSender defines the interface that allows sending data. This is an interface that must be implemented by all protocols that want to be used in LoadGenerator. Note the terminology: testbed.DataSender is something that sends data to Collector and the corresponding entity that receives the data in the Collector is a receiver.

type DataSenderOverTraceExporter

type DataSenderOverTraceExporter struct {
	Port int
	// contains filtered or unexported fields
}

DataSenderOverTraceExporter partially implements TraceDataSender via a TraceExporter.

func NewDataSenderOverExporter

func NewDataSenderOverExporter(port int) *DataSenderOverTraceExporter

NewDataSenderOverExporter creates a new sender that will send to the specified port after Start is called.

func (*DataSenderOverTraceExporter) Flush

func (ds *DataSenderOverTraceExporter) Flush()

func (*DataSenderOverTraceExporter) GetCollectorPort

func (ds *DataSenderOverTraceExporter) GetCollectorPort() int

func (*DataSenderOverTraceExporter) SendSpans

func (ds *DataSenderOverTraceExporter) SendSpans(traces pdata.Traces) error

type DataSenderOverTraceExporterOld

type DataSenderOverTraceExporterOld struct {
	Port int
	// contains filtered or unexported fields
}

DataSenderOverTraceExporter partially implements TraceDataSender via a TraceExporter.

func NewDataSenderOverExporterOld

func NewDataSenderOverExporterOld(port int) *DataSenderOverTraceExporterOld

NewDataSenderOverExporter creates a new sender that will send to the specified port after Start is called.

func (*DataSenderOverTraceExporterOld) Flush

func (ds *DataSenderOverTraceExporterOld) Flush()

func (*DataSenderOverTraceExporterOld) GetCollectorPort

func (ds *DataSenderOverTraceExporterOld) GetCollectorPort() int

func (*DataSenderOverTraceExporterOld) SendSpans

type GlobalConfig

type GlobalConfig struct {
	Agent         string
	LoadGenerator string `mapstructure:"load-generator"`
}

GlobalConfig defines test bed configuration.

type JaegerDataReceiver

type JaegerDataReceiver struct {
	DataReceiverBase
	// contains filtered or unexported fields
}

JaegerDataReceiver implements Jaeger format receiver.

func NewJaegerDataReceiver

func NewJaegerDataReceiver(port int) *JaegerDataReceiver

func (*JaegerDataReceiver) GenConfigYAMLStr

func (jr *JaegerDataReceiver) GenConfigYAMLStr() string

func (*JaegerDataReceiver) ProtocolName

func (jr *JaegerDataReceiver) ProtocolName() string

func (*JaegerDataReceiver) Start

func (*JaegerDataReceiver) Stop

func (jr *JaegerDataReceiver) Stop()

type JaegerGRPCDataSender

type JaegerGRPCDataSender struct {
	DataSenderOverTraceExporter
}

JaegerGRPCDataSender implements TraceDataSender for Jaeger thrift_http protocol.

func NewJaegerGRPCDataSender

func NewJaegerGRPCDataSender(port int) *JaegerGRPCDataSender

NewJaegerGRPCDataSender creates a new Jaeger protocol sender that will send to the specified port after Start is called.

func (*JaegerGRPCDataSender) GenConfigYAMLStr

func (je *JaegerGRPCDataSender) GenConfigYAMLStr() string

func (*JaegerGRPCDataSender) ProtocolName

func (je *JaegerGRPCDataSender) ProtocolName() string

func (*JaegerGRPCDataSender) Start

func (je *JaegerGRPCDataSender) Start() error

type LoadGenerator

type LoadGenerator struct {
	// contains filtered or unexported fields
}

LoadGenerator is a simple load generator.

func NewLoadGenerator

func NewLoadGenerator(sender DataSender) (*LoadGenerator, error)

NewLoadGenerator creates a load generator that sends data using specified sender.

func (*LoadGenerator) DataItemsSent

func (lg *LoadGenerator) DataItemsSent() uint64

func (*LoadGenerator) GetStats

func (lg *LoadGenerator) GetStats() string

GetStats returns the stats as a printable string.

func (*LoadGenerator) IncDataItemsSent

func (lg *LoadGenerator) IncDataItemsSent()

IncDataItemsSent is used when a test bypasses the LoadGenerator and sends data directly via TestCases's Sender. This is necessary so that the total number of sent items in the end is correct, because the reports are printed from LoadGenerator's fields. This is not the best way, a better approach would be to refactor the reports to use their own counter and load generator and other sending sources to contribute to this counter. This could be done as a future improvement.

func (*LoadGenerator) Start

func (lg *LoadGenerator) Start(options LoadOptions)

Start the load.

func (*LoadGenerator) Stop

func (lg *LoadGenerator) Stop()

Stop the load.

type LoadOptions

type LoadOptions struct {
	// DataItemsPerSecond specifies how many spans or metric data points to generate each second.
	DataItemsPerSecond int

	// ItemsPerBatch specifies how many spans or metric data points per batch to generate.
	// Should be greater than zero. The number of batches generated per second will be
	// DataItemsPerSecond/ItemsPerBatch.
	ItemsPerBatch int

	// Attributes to add to each generated data item. Can be empty.
	Attributes map[string]string
}

LoadOptions defines the options to use for generating the load.

type MetricDataSender

type MetricDataSender interface {
	DataSender
	SendMetrics(metrics data.MetricData) error
}

MetricDataSender defines the interface that allows sending metric data. It adds ability to send a batch of Metrics to the DataSender interface.

type MetricDataSenderOld

type MetricDataSenderOld interface {
	DataSender
	SendMetrics(metrics consumerdata.MetricsData) error
}

MetricDataSender defines the interface that allows sending metric data. It adds ability to send a batch of Metrics to the DataSender interface.

type MockBackend

type MockBackend struct {
	ReceivedTraces     []pdata.Traces
	ReceivedMetrics    []pdata.Metrics
	ReceivedTracesOld  []consumerdata.TraceData
	ReceivedMetricsOld []consumerdata.MetricsData
	// contains filtered or unexported fields
}

MockBackend is a backend that allows receiving the data locally.

func NewMockBackend

func NewMockBackend(logFilePath string, receiver DataReceiver) *MockBackend

NewMockBackend creates a new mock backend that receives data using specified receiver.

func (*MockBackend) ClearReceivedItems

func (mb *MockBackend) ClearReceivedItems()

ClearReceivedItems clears the list of received traces and metrics. Note: counters return by DataItemsReceived() are not cleared, they are cumulative.

func (*MockBackend) ConsumeMetric

func (mb *MockBackend) ConsumeMetric(md pdata.Metrics)

func (*MockBackend) ConsumeMetricOld

func (mb *MockBackend) ConsumeMetricOld(md consumerdata.MetricsData)

ConsumeMetricOld consumes metric data in old representation

func (*MockBackend) ConsumeTrace

func (mb *MockBackend) ConsumeTrace(td pdata.Traces)

func (*MockBackend) ConsumeTraceOld

func (mb *MockBackend) ConsumeTraceOld(td consumerdata.TraceData)

ConsumeTraceOld consumes trace data in old representation

func (*MockBackend) DataItemsReceived

func (mb *MockBackend) DataItemsReceived() uint64

DataItemsReceived returns total number of received spans and metrics.

func (*MockBackend) EnableRecording

func (mb *MockBackend) EnableRecording()

EnableRecording enables recording of all data received by MockBackend.

func (*MockBackend) GetStats

func (mb *MockBackend) GetStats() string

func (*MockBackend) ReportFatalError

func (mb *MockBackend) ReportFatalError(err error)

func (*MockBackend) Start

func (mb *MockBackend) Start() error

Start a backend.

func (*MockBackend) Stop

func (mb *MockBackend) Stop()

Stop the backend

type MockMetricConsumer

type MockMetricConsumer struct {
	// contains filtered or unexported fields
}

func (*MockMetricConsumer) ConsumeMetrics

func (mc *MockMetricConsumer) ConsumeMetrics(_ context.Context, md pdata.Metrics) error

func (*MockMetricConsumer) ConsumeMetricsData

func (mc *MockMetricConsumer) ConsumeMetricsData(ctx context.Context, md consumerdata.MetricsData) error

type MockTraceConsumer

type MockTraceConsumer struct {
	// contains filtered or unexported fields
}

func (*MockTraceConsumer) ConsumeTraceData

func (tc *MockTraceConsumer) ConsumeTraceData(ctx context.Context, td consumerdata.TraceData) error

func (*MockTraceConsumer) ConsumeTraces

func (tc *MockTraceConsumer) ConsumeTraces(ctx context.Context, td pdata.Traces) error

type OCDataReceiver

type OCDataReceiver struct {
	DataReceiverBase
	// contains filtered or unexported fields
}

OCDataReceiver implements OpenCensus format receiver.

func NewOCDataReceiver

func NewOCDataReceiver(port int) *OCDataReceiver

NewOCDataReceiver creates a new OCDataReceiver that will listen on the specified port after Start is called.

func (*OCDataReceiver) GenConfigYAMLStr

func (or *OCDataReceiver) GenConfigYAMLStr() string

func (*OCDataReceiver) ProtocolName

func (or *OCDataReceiver) ProtocolName() string

func (*OCDataReceiver) Start

func (*OCDataReceiver) Stop

func (or *OCDataReceiver) Stop()

type OCMetricsDataSender

type OCMetricsDataSender struct {
	// contains filtered or unexported fields
}

OCMetricsDataSender implements MetricDataSender for OpenCensus metrics protocol.

func NewOCMetricDataSender

func NewOCMetricDataSender(port int) *OCMetricsDataSender

NewOCMetricDataSender creates a new OpenCensus metric protocol sender that will send to the specified port after Start is called.

func (*OCMetricsDataSender) Flush

func (ome *OCMetricsDataSender) Flush()

func (*OCMetricsDataSender) GenConfigYAMLStr

func (ome *OCMetricsDataSender) GenConfigYAMLStr() string

func (*OCMetricsDataSender) GetCollectorPort

func (ome *OCMetricsDataSender) GetCollectorPort() int

func (*OCMetricsDataSender) ProtocolName

func (ome *OCMetricsDataSender) ProtocolName() string

func (*OCMetricsDataSender) SendMetrics

func (ome *OCMetricsDataSender) SendMetrics(metrics consumerdata.MetricsData) error

func (*OCMetricsDataSender) Start

func (ome *OCMetricsDataSender) Start() error

type OCTraceDataSender

type OCTraceDataSender struct {
	DataSenderOverTraceExporterOld
}

OCTraceDataSender implements TraceDataSender for OpenCensus trace protocol.

func NewOCTraceDataSender

func NewOCTraceDataSender(port int) *OCTraceDataSender

NewOCTraceDataSender creates a new OCTraceDataSender that will send to the specified port after Start is called.

func (*OCTraceDataSender) GenConfigYAMLStr

func (ote *OCTraceDataSender) GenConfigYAMLStr() string

func (*OCTraceDataSender) ProtocolName

func (ote *OCTraceDataSender) ProtocolName() string

func (*OCTraceDataSender) Start

func (ote *OCTraceDataSender) Start() error

type OTLPDataReceiver

type OTLPDataReceiver struct {
	DataReceiverBase
	// contains filtered or unexported fields
}

OTLPDataReceiver implements OTLP format receiver.

func NewOTLPDataReceiver

func NewOTLPDataReceiver(port int) *OTLPDataReceiver

NewOTLPDataReceiver creates a new OTLPDataReceiver that will listen on the specified port after Start is called.

func (*OTLPDataReceiver) GenConfigYAMLStr

func (or *OTLPDataReceiver) GenConfigYAMLStr() string

func (*OTLPDataReceiver) ProtocolName

func (or *OTLPDataReceiver) ProtocolName() string

func (*OTLPDataReceiver) Start

func (*OTLPDataReceiver) Stop

func (or *OTLPDataReceiver) Stop()

type OTLPMetricsDataSender

type OTLPMetricsDataSender struct {
	// contains filtered or unexported fields
}

OTLPMetricsDataSender implements MetricDataSender for OpenCensus metrics protocol.

func NewOTLPMetricDataSender

func NewOTLPMetricDataSender(port int) *OTLPMetricsDataSender

NewOTLPMetricDataSender creates a new OpenCensus metric protocol sender that will send to the specified port after Start is called.

func (*OTLPMetricsDataSender) Flush

func (ome *OTLPMetricsDataSender) Flush()

func (*OTLPMetricsDataSender) GenConfigYAMLStr

func (ome *OTLPMetricsDataSender) GenConfigYAMLStr() string

func (*OTLPMetricsDataSender) GetCollectorPort

func (ome *OTLPMetricsDataSender) GetCollectorPort() int

func (*OTLPMetricsDataSender) ProtocolName

func (ome *OTLPMetricsDataSender) ProtocolName() string

func (*OTLPMetricsDataSender) SendMetrics

func (ome *OTLPMetricsDataSender) SendMetrics(metrics data.MetricData) error

func (*OTLPMetricsDataSender) Start

func (ome *OTLPMetricsDataSender) Start() error

type OTLPTraceDataSender

type OTLPTraceDataSender struct {
	DataSenderOverTraceExporter
}

OTLPTraceDataSender implements TraceDataSender for OpenCensus trace protocol.

func NewOTLPTraceDataSender

func NewOTLPTraceDataSender(port int) *OTLPTraceDataSender

NewOTLPTraceDataSender creates a new OTLPTraceDataSender that will send to the specified port after Start is called.

func (*OTLPTraceDataSender) GenConfigYAMLStr

func (ote *OTLPTraceDataSender) GenConfigYAMLStr() string

func (*OTLPTraceDataSender) ProtocolName

func (ote *OTLPTraceDataSender) ProtocolName() string

func (*OTLPTraceDataSender) Start

func (ote *OTLPTraceDataSender) Start() error

type ResourceConsumption

type ResourceConsumption struct {
	CPUPercentAvg float64
	CPUPercentMax float64
	RAMMiBAvg     uint32
	RAMMiBMax     uint32
}

type ResourceSpec

type ResourceSpec struct {
	// Percentage of one core the process is expected to consume at most.
	// Test is aborted and failed if consumption during
	// ResourceCheckPeriod exceeds this number. If 0 the CPU
	// consumption is not monitored and does not affect the test result.
	ExpectedMaxCPU uint32

	// Maximum RAM in MiB the process is expected to consume.
	// Test is aborted and failed if consumption exceeds this number.
	// If 0 memory consumption is not monitored and does not affect
	// the test result.
	ExpectedMaxRAM uint32

	// Period during which CPU and RAM of the process are measured.
	// Bigger numbers will result in more averaging of short spikes.
	ResourceCheckPeriod time.Duration
}

ResourceSpec is a resource consumption specification.

type Results

type Results struct {
	// contains filtered or unexported fields
}

func (*Results) Add

func (r *Results) Add(testName string, result *TestResult)

Add results for one test.

func (*Results) Init

func (r *Results) Init(resultsDir string)

func (*Results) Save

func (r *Results) Save()

Save the total results and close the file.

type TestCase

type TestCase struct {
	Sender   DataSender
	Receiver DataReceiver

	LoadGenerator *LoadGenerator
	MockBackend   *MockBackend

	// ErrorSignal indicates an error in the test case execution, e.g. process execution
	// failure or exceeding resource consumption, etc. The actual error message is already
	// logged, this is only an indicator on which you can wait to be informed.
	ErrorSignal chan struct{}

	// Duration is the requested duration of the tests. Configured via TESTBED_DURATION
	// env variable and defaults to 15 seconds if env variable is unspecified.
	Duration time.Duration
	// contains filtered or unexported fields
}

TestCase defines a running test case.

func NewTestCase

func NewTestCase(
	t *testing.T,
	sender DataSender,
	receiver DataReceiver,
	opts ...TestCaseOption,
) *TestCase

NewTestCase creates a new TestCase. It expects agent-config.yaml in the specified directory.

func (*TestCase) AgentMemoryInfo

func (tc *TestCase) AgentMemoryInfo() (uint32, uint32, error)

AgentMemoryInfo returns raw memory info struct about the agent as returned by github.com/shirou/gopsutil/process

func (*TestCase) EnableRecording

func (tc *TestCase) EnableRecording()

EnableRecording enables recording of all data received by MockBackend.

func (*TestCase) SetResourceLimits

func (tc *TestCase) SetResourceLimits(resourceSpec ResourceSpec)

SetResourceLimits sets expected limits for resource consmption. Error is signaled if consumption during ResourceCheckPeriod exceeds the limits. Limits are modified only for non-zero fields of resourceSpec, all zero-value fields fo resourceSpec are ignored and their previous values remain in effect.

func (*TestCase) Sleep

func (tc *TestCase) Sleep(d time.Duration)

Sleep for specified duration or until error is signaled.

func (*TestCase) StartAgent

func (tc *TestCase) StartAgent(args ...string)

StartAgent starts the agent and redirects its standard output and standard error to "agent.log" file located in the test directory.

func (*TestCase) StartBackend

func (tc *TestCase) StartBackend()

StartBackend starts the specified backend type.

func (*TestCase) StartLoad

func (tc *TestCase) StartLoad(options LoadOptions)

StartLoad starts the load generator and redirects its standard output and standard error to "load-generator.log" file located in the test directory.

func (*TestCase) Stop

func (tc *TestCase) Stop()

Stop stops the load generator, the agent and the backend.

func (*TestCase) StopAgent

func (tc *TestCase) StopAgent()

StopAgent stops agent process.

func (*TestCase) StopBackend

func (tc *TestCase) StopBackend()

StopBackend stops the backend.

func (*TestCase) StopLoad

func (tc *TestCase) StopLoad()

StopLoad stops load generator.

func (*TestCase) ValidateData

func (tc *TestCase) ValidateData()

ValidateData validates data by comparing the number of items sent by load generator and number of items received by mock backend.

func (*TestCase) WaitFor

func (tc *TestCase) WaitFor(cond func() bool, errMsg ...interface{}) bool

WaitFor is like WaitForN but with a fixed duration of 10 seconds

func (*TestCase) WaitForN

func (tc *TestCase) WaitForN(cond func() bool, duration time.Duration, errMsg ...interface{}) bool

WaitForN the specific condition for up to a specified duration. Records a test error if time is out and condition does not become true. If error is signaled while waiting the function will return false, but will not record additional test error (we assume that signaled error is already recorded in indicateError()).

type TestCaseOption

type TestCaseOption struct {
	// contains filtered or unexported fields
}

TestCaseOption defines a TestCase option.

func WithConfigFile

func WithConfigFile(file string) TestCaseOption

WithConfigFile allows a custom configuration file for TestCase.

func WithSkipResults

func WithSkipResults() TestCaseOption

WithSkipResults option disables writing out results file for a TestCase.

func (TestCaseOption) Apply

func (o TestCaseOption) Apply(t *TestCase)

Apply takes a TestCase and runs the option function on it.

type TestResult

type TestResult struct {
	// contains filtered or unexported fields
}

type TraceDataSender

type TraceDataSender interface {
	DataSender
	SendSpans(traces pdata.Traces) error
}

TraceDataSender defines the interface that allows sending trace data. It adds ability to send a batch of Spans to the DataSender interface.

type TraceDataSenderOld

type TraceDataSenderOld interface {
	DataSender
	SendSpans(traces consumerdata.TraceData) error
}

TraceDataSender defines the interface that allows sending trace data. It adds ability to send a batch of Spans to the DataSender interface.

type ZipkinDataReceiver

type ZipkinDataReceiver struct {
	DataReceiverBase
	// contains filtered or unexported fields
}

ZipkinDataReceiver implements Zipkin format receiver.

func NewZipkinDataReceiver

func NewZipkinDataReceiver(port int) *ZipkinDataReceiver

func (*ZipkinDataReceiver) GenConfigYAMLStr

func (zr *ZipkinDataReceiver) GenConfigYAMLStr() string

func (*ZipkinDataReceiver) ProtocolName

func (zr *ZipkinDataReceiver) ProtocolName() string

func (*ZipkinDataReceiver) Start

func (*ZipkinDataReceiver) Stop

func (zr *ZipkinDataReceiver) Stop()

type ZipkinDataSender

type ZipkinDataSender struct {
	DataSenderOverTraceExporterOld
}

ZipkinDataSender implements TraceDataSender for Zipkin http protocol.

func NewZipkinDataSender

func NewZipkinDataSender(port int) *ZipkinDataSender

NewZipkinDataSender creates a new Zipkin protocol sender that will send to the specified port after Start is called.

func (*ZipkinDataSender) GenConfigYAMLStr

func (zs *ZipkinDataSender) GenConfigYAMLStr() string

func (*ZipkinDataSender) ProtocolName

func (zs *ZipkinDataSender) ProtocolName() string

func (*ZipkinDataSender) Start

func (zs *ZipkinDataSender) Start() error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL