base

package
v0.14.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 16, 2023 License: Apache-2.0 Imports: 51 Imported by: 0

Documentation

Overview

Package base defines Iter8's experiment, task and metric data structures. It contains the core logic for running an experiment.

Index

Constants

View Source 
const (
	// CounterMetricType corresponds to Prometheus Counter metric type
	CounterMetricType MetricType = "Counter"
	// GaugeMetricType corresponds to Prometheus Gauge metric type
	GaugeMetricType MetricType = "Gauge"
	// HistogramMetricType corresponds to a Histogram metric type
	HistogramMetricType MetricType = "Histogram"
	// SampleMetricType corresponds to a Sample metric type
	SampleMetricType MetricType = "Sample"
	// SummaryMetricType corresponds to a Summary metric type
	SummaryMetricType MetricType = "Summary"

	// CountAggregator corresponds to aggregation of type count
	CountAggregator AggregationType = "count"
	// MeanAggregator corresponds to aggregation of type mean
	MeanAggregator AggregationType = "mean"
	// StdDevAggregator corresponds to aggregation of type stddev
	StdDevAggregator AggregationType = "stddev"
	// MinAggregator corresponds to aggregation of type min
	MinAggregator AggregationType = "min"
	// MaxAggregator corresponds to aggregation of type max
	MaxAggregator AggregationType = "max"
	// PercentileAggregator corresponds to aggregation of type max
	PercentileAggregator AggregationType = "percentile"
	// PercentileAggregatorPrefix corresponds to prefix for percentiles
	PercentileAggregatorPrefix = "p"
)
View Source 
const (
	// AssessTaskName is the name of the task this file implements
	AssessTaskName = "assess"
)
View Source 
const (
	// CollectABNMetricsTaskName is the name of the task to read A/B/n metric values
	CollectABNMetricsTaskName = "abnmetrics"
)
View Source 
const (
	// CollectGRPCTaskName is the name of this task which performs load generation and metrics collection for gRPC services.
	CollectGRPCTaskName = "grpc"
)
View Source 
const (
	// CollectHTTPTaskName is the name of this task which performs load generation and metrics collection.
	CollectHTTPTaskName = "http"
)
View Source 
const (
	// CustomMetricsTaskName is the name of this task which fetches metrics templates, constructs metric specs, and then fetches metrics for each version from metric provider databases
	CustomMetricsTaskName = "custommetrics"
)
View Source 
const (
	// NotifyTaskName is the task name
	NotifyTaskName = "notify"
)
View Source 
const (
	// ReadinessTaskName is the task name
	ReadinessTaskName = "ready"
)
View Source 
const (
	// RunTaskName is the name of the run task which performs running of a shell script
	RunTaskName = "run"
)

Variables

View Source 
var MajorMinor = "v0.14"

MajorMinor is the minor version of Iter8 set this manually whenever the major or minor version changes

View Source 
var Version = "v0.14.0"

Version is the semantic version of Iter8 (with the `v` prefix) Version is intended to be set using LDFLAGS at build time

Functions

func BoolPointer added in v0.9.3 

func BoolPointer(b bool) *bool

BoolPointer takes bool as input, creates a new variable with the input value, and returns a pointer to the variable

func CompletePath 

func CompletePath(prefix string, suffix string) string

CompletePath is a helper function for converting file paths, specified relative to the caller of this function, into absolute ones. CompletePath is useful in tests and enables deriving the absolute path of experiment YAML files.

func CreateExperimentYaml added in v0.13.11 

func CreateExperimentYaml(t *testing.T, template string, url string, output string)

CreateExperimentYaml creates an experiment.yaml file from a template and a URL

func CreateTemplate added in v0.12.6 

func CreateTemplate(tplString string) (*template.Template, error)

CreateTemplate creates a template from a string

func FuncMapWithToYAML added in v0.12.6 

func FuncMapWithToYAML() template.FuncMap

FuncMapWithToYAML return sprig text function map with a toYaml function

func GetTrackingHandler added in v0.13.11 

func GetTrackingHandler(breadcrumb *bool) func(w http.ResponseWriter, r *http.Request)

GetTrackingHandler creates a handler for fhttp.DynamicHTTPServer that sets a variable to true This can be used to verify that the handler was called.

func NormalizeMetricName added in v0.8.29 

func NormalizeMetricName(m string) (string, error)

NormalizeMetricName normalizes percentile values in metric names

func RunExperiment added in v0.9.3 

func RunExperiment(reuseResult bool, driver Driver) error

RunExperiment runs an experiment

func StringPointer added in v0.8.8 

func StringPointer(s string) *string

StringPointer takes string as input, creates a new variable with the input value, and returns a pointer to the variable

func ToYAML added in v0.12.6 

func ToYAML(v interface{}) string

ToYAML takes an interface, marshals it to yaml, and returns a string. It will always return a string, even on marshal error (empty string).

This is designed to be called from a template.

func Uniq added in v0.8.30 

func Uniq(list interface{}) []interface{}

Uniq deduplicates a list We have switched from uniq to Uniq, since we want to use it in other packages

Types

type ABNMetricsInputs added in v0.11.12 

type ABNMetricsInputs struct {
	Endpoint *string `json:"endpoint" yaml:"endpoint"`
	// Application is name of application to evaluate
	Application string `json:"application" yaml:"application"`
}

ABNMetricsInputs is the inputs for for the abnmetrics task

type AggregationType added in v0.8.29 

type AggregationType string

AggregationType identifies the type of the metric aggregator.

type Driver added in v0.9.3 

type Driver interface {
	// Read the experiment
	Read() (*Experiment, error)

	// Write the experiment
	Write(e *Experiment) error

	// GetRevision returns the experiment revision
	GetRevision() int
}

Driver enables interacting with experiment result stored externally

type Experiment 

type Experiment struct {
	// Spec is the sequence of tasks that constitute this experiment
	Spec ExperimentSpec `json:"spec" yaml:"spec"`

	// Result is the current results from this experiment.
	// The experiment may not have completed in which case results may be partial.
	Result *ExperimentResult `json:"result" yaml:"result"`
	// contains filtered or unexported fields
}

Experiment struct containing spec and result

func BuildExperiment added in v0.9.3 

func BuildExperiment(driver Driver) (*Experiment, error)

BuildExperiment builds an experiment

func (*Experiment) Completed added in v0.9.3 

func (exp *Experiment) Completed() bool

Completed returns true if the experiment is complete

func (*Experiment) NoFailure added in v0.9.3 

func (exp *Experiment) NoFailure() bool

NoFailure returns true if no task in the experiment has failed

func (*Experiment) SLOs 

func (exp *Experiment) SLOs() bool

SLOs returns true if all versions satisfy SLOs

type ExperimentResult 

type ExperimentResult struct {
	// Revision of this experiment
	Revision int `json:"revision,omitempty" yaml:"revision,omitempty"`

	// StartTime is the time when the experiment run started
	StartTime time.Time `json:"startTime" yaml:"startTime"`

	// NumLoops is the number of iterations this experiment has been running for
	NumLoops int `json:"numLoops" yaml:"numLoops"`

	// NumCompletedTasks is the number of completed tasks
	NumCompletedTasks int `json:"numCompletedTasks" yaml:"numCompletedTasks"`

	// Failure is true if any of its tasks failed
	Failure bool `json:"failure" yaml:"failure"`

	// Insights produced in this experiment
	Insights *Insights `json:"insights,omitempty" yaml:"insights,omitempty"`

	// Iter8Version is the version of Iter8 CLI that created this result object
	Iter8Version string `json:"iter8Version" yaml:"iter8Version"`
}

ExperimentResult defines the current results from the experiment

type ExperimentSpec added in v0.8.29 

type ExperimentSpec []Task

ExperimentSpec specifies the set of tasks in this experiment

func (*ExperimentSpec) UnmarshalJSON added in v0.8.29 

func (s *ExperimentSpec) UnmarshalJSON(data []byte) error

UnmarshalJSON will unmarshal an experiment spec from bytes This is a custom JSON unmarshaler

type HTTPParam added in v0.11.3 

type HTTPParam struct {
	// Name is the name of the HTTP parameter
	Name string `json:"name" yaml:"name"`

	// Value is the value of the HTTP parameter
	Value string `json:"value" yaml:"value"`
}

HTTPParam defines an HTTP parameter

type HistBucket added in v0.8.29 

type HistBucket struct {
	// Lower endpoint of a histogram bucket
	Lower float64 `json:"lower" yaml:"lower"`
	// Upper endpoint of a histogram bucket
	Upper float64 `json:"upper" yaml:"upper"`
	// Count is the frequency count of the bucket
	Count uint64 `json:"count" yaml:"count"`
}

HistBucket is a single bucket in a histogram

type Insights 

type Insights struct {
	// NumVersions is the number of app versions detected by Iter8
	NumVersions int `json:"numVersions" yaml:"numVersions"`

	// VersionNames is list of version identifiers if known
	VersionNames []VersionInfo `json:"versionNames" yaml:"versionNames"`

	// MetricsInfo identifies the metrics involved in this experiment
	MetricsInfo map[string]MetricMeta `json:"metricsInfo,omitempty" yaml:"metricsInfo,omitempty"`

	// NonHistMetricValues:
	// the outer slice must be the same length as the number of app versions
	// the map key must match name of a metric in MetricsInfo
	// the inner slice contains the list of all observed metric values for given version and given metric; float value [i]["foo/bar"][k] is the [k]th observation for version [i] for the metric bar under backend foo.
	// this struct is meant exclusively for metrics of type other than histogram
	NonHistMetricValues []map[string][]float64 `json:"nonHistMetricValues,omitempty" yaml:"nonHistMetricValues,omitempty"`

	// HistMetricValues:
	// the outer slice must be the same length as the number of app versions
	// the map key must match name of a histogram metric in MetricsInfo
	// the inner slice contains the list of all observed histogram buckets for a given version and given metric; value [i]["foo/bar"][k] is the [k]th observed bucket for version [i] for the hist metric `bar` under backend `foo`.
	HistMetricValues []map[string][]HistBucket `json:"histMetricValues,omitempty" yaml:"histMetricValues,omitempty"`

	// SummaryMetricValues:
	// the outer slice must be the same length as the number of tracks
	// the map key must match the name of the summary metric in MetricsInfo
	SummaryMetricValues []map[string]summarymetrics.SummaryMetric

	// SLOs involved in this experiment
	SLOs *SLOLimits `json:"SLOs,omitempty" yaml:"SLOs,omitempty"`

	// SLOsSatisfied indicator matrices that show if upper and lower SLO limits are satisfied
	SLOsSatisfied *SLOResults `json:"SLOsSatisfied,omitempty" yaml:"SLOsSatisfied,omitempty"`
}

Insights records the number of versions in this experiment, metric values and SLO indicators for each version, metrics metadata for all metrics, and SLO definitions for all SLOs

func (*Insights) GetMetricsInfo added in v0.8.30 

func (in *Insights) GetMetricsInfo(nm string) (*MetricMeta, error)

GetMetricsInfo gets metric meta for the given normalized metric name

func (*Insights) ScalarMetricValue added in v0.8.29 

func (in *Insights) ScalarMetricValue(i int, m string) *float64

ScalarMetricValue gets the value of the given scalar metric for the given version

func (*Insights) TrackVersionStr added in v0.11.12 

func (in *Insights) TrackVersionStr(i int) string

TrackVersionStr creates a string of version name/track for display purposes

type KubeDriver added in v0.10.4 

type KubeDriver struct {
	// EnvSettings provides generic Kubernetes options
	*cli.EnvSettings
	// contains filtered or unexported fields
}

KubeDriver embeds Kube configuration, and enables interaction with a Kubernetes cluster through Kube APIs

func NewFakeKubeDriver added in v0.10.4 

func NewFakeKubeDriver(s *cli.EnvSettings, objects ...runtime.Object) *KubeDriver

NewFakeKubeDriver creates and returns a new KubeDriver with fake clients

func NewKubeDriver added in v0.10.4 

func NewKubeDriver(s *cli.EnvSettings) *KubeDriver

NewKubeDriver creates and returns a new KubeDriver

type Metric added in v0.9.3 

type Metric struct {
	// Name is the name of the metric
	Name string `json:"name" yaml:"name"`

	// Description is the description of the metric
	Description *string `json:"description,omitempty" yaml:"description,omitempty"`

	// Type is the type of the metric, either gauge or counter
	Type string `json:"type" yaml:"type"`

	// Units is the unit of the metric, which can be omitted for unitless metrics
	Units *string `json:"units,omitempty" yaml:"units,omitempty"`

	// Params is the set of HTTP parameters that need to be sent
	Params *[]HTTPParam `json:"params,omitempty" yaml:"params,omitempty"`

	// Body is the HTTP request body that needs to be sent
	Body *string `json:"body,omitempty" yaml:"body,omitempty"`

	// JqExpression is the jq expression that can extract the value from the HTTP
	// response
	JqExpression string `json:"jqExpression" yaml:"jqExpression"`
}

Metric defines how to construct HTTP requests and parse HTTP responses when querying a database for a metric. Metric struct also includes metadata such as the name and description of the metric.

type MetricMeta 

type MetricMeta struct {
	// Description is a human readable description of the metric
	Description string `json:"description" yaml:"description"`
	// Units for this metric (if any)
	Units *string `json:"units,omitempty" yaml:"units,omitempty"`
	// Type of the metric. Example: counter
	Type MetricType `json:"type" yaml:"type"`
}

MetricMeta describes a metric

type MetricType 

type MetricType string

MetricType identifies the type of the metric.

type ProviderSpec added in v0.11.3 

type ProviderSpec struct {
	// URL is the database endpoint
	URL string `json:"url" yaml:"url"`

	// Method is the HTTP method that needs to be used
	Method string `json:"method" yaml:"method"`

	// Headers is the set of HTTP headers that need to be sent
	Headers map[string]string `json:"headers" yaml:"headers"`

	// Metrics is the set of metrics that can be obtained
	Metrics []Metric `json:"metrics" yaml:"metrics"`
}

ProviderSpec specifies how to get metrics from a provider

type Report added in v0.11.10 

type Report struct {
	// Timestamp is when the report was created
	// For example: 2022-08-09 15:10:36.569745 -0400 EDT m=+12.599643189
	TimeStamp string `json:"timeStamp" yaml:"timeStamp"`

	// Completed is whether or not the experiment has completed
	Completed bool `json:"completed" yaml:"completed"`

	// NoTaskFailures is whether or not the experiment had any tasks that failed
	NoTaskFailures bool `json:"noTaskFailures" yaml:"noTaskFailures"`

	// NumTasks is the number of tasks in the experiment
	NumTasks int `json:"numTasks" yaml:"numTasks"`

	// NumCompletedTasks is the number of completed tasks in the experiment
	NumCompletedTasks int `json:"numCompletedTasks" yaml:"numCompletedTasks"`

	// NumLoops is the current loop of the experiment
	NumLoops int `json:"numLoops" yaml:"numLoops"`

	// Experiment is the experiment struct
	Experiment *Experiment `json:"experiment" yaml:"experiment"`
}

Report is the data that is given to the payload template

type SLO 

type SLO struct {
	// Metric is the fully qualified metric name in the backendName/metricName format
	Metric string `json:"metric" yaml:"metric"`

	// Limit is the acceptable limit for this metric
	Limit float64 `json:"limit" yaml:"limit"`
}

SLO is a service level objective

type SLOLimits added in v0.11.0 

type SLOLimits struct {
	// Upper limits for metrics
	Upper []SLO `json:"upper,omitempty" yaml:"upper,omitempty"`

	// Lower limits for metrics
	Lower []SLO `json:"lower,omitempty" yaml:"lower,omitempty"`
}

SLOLimits specify upper or lower limits for metrics

type SLOResults added in v0.11.0 

type SLOResults struct {
	// Upper limits for metrics
	// Upper[i][j] specifies if upper SLO i is satisfied by version j
	Upper [][]bool `json:"upper,omitempty" yaml:"upper,omitempty"`

	// Lower limits for metrics
	// Lower[i][j] specifies if lower SLO i is satisfied by version j
	Lower [][]bool `json:"lower,omitempty" yaml:"lower,omitempty"`
}

SLOResults specify the results of SLO evaluations

type Task 

type Task interface {
	// contains filtered or unexported methods
}

Task is the building block of an experiment spec An experiment spec is a sequence of tasks

type TaskMeta added in v0.8.29 

type TaskMeta struct {
	// Task is the name of the task
	Task *string `json:"task,omitempty" yaml:"task,omitempty"`
	// Run is the script used in a run task
	// Specify either Task or Run but not both
	Run *string `json:"run,omitempty" yaml:"run,omitempty"`
	// If is the condition used to determine if this task needs to run
	// If the condition is not satisfied, then it is skipped in an experiment
	// Example: SLOs()
	If *string `json:"if,omitempty" yaml:"if,omitempty"`
}

TaskMeta provides common fields used across all tasks

type VersionInfo added in v0.11.12 

type VersionInfo struct {
	// Version name
	Version string `json:"version" yaml:"version"`

	// Track identifier assigned to version
	Track string `json:"track" yaml:"track"`
}

VersionInfo is basic information about a version

Source Files

View all Source files

Directories

Path Synopsis
Package internal provides gRPC code used for testing load-test-grpc
 Package internal provides gRPC code used for testing load-test-grpc
helloworld/helloworld
Package helloworld implements the helloworld grpc service.
 Package helloworld implements the helloworld grpc service.
Package log enables logging for Iter8.
 Package log enables logging for Iter8.
Package summarymetrics provides a summary metric implementation
 Package summarymetrics provides a summary metric implementation

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.