Documentation
¶
Index ¶
- func NewOpsCollector() (Collector, OpsCollector)
- func StreamOperationsFromCSV(r io.Reader, analyzeOnly bool, offset, limit int, ...) error
- type Benchmark
- type Categories
- type Category
- type CmpReqs
- type CmpRequests
- type CmpSegment
- type Collector
- type Common
- type Comparison
- type Delete
- type Fanout
- type Get
- type List
- type Mixed
- type MixedDistribution
- type Multipart
- type MultipartPut
- type Operation
- type Operations
- func (o Operations) ActiveTimeRange(allThreads bool) (start, end time.Time)
- func (o Operations) AvgDuration() time.Duration
- func (o Operations) AvgSize() int64
- func (o Operations) ByEndpoint() map[string]Operations
- func (o Operations) CSV(w io.Writer, comment string) error
- func (o Operations) ClientIDs(prefix string) []string
- func (o Operations) Clients() int
- func (o Operations) Clone() Operations
- func (o Operations) Duration() time.Duration
- func (o Operations) Endpoints() []string
- func (o Operations) Errors() []string
- func (o Operations) FilterByEndpoint(endpoint string) Operations
- func (o Operations) FilterByHasTTFB(hasTTFB bool) Operations
- func (o Operations) FilterByOp(opType string) Operations
- func (o Operations) FilterErrors() Operations
- func (o Operations) FilterFirst() Operations
- func (o Operations) FilterInsideRange(start, end time.Time) Operations
- func (o Operations) FilterLast() Operations
- func (o Operations) FilterSuccessful() Operations
- func (o Operations) FirstObjPerOp() int
- func (o Operations) FirstObjSize() int64
- func (o Operations) FirstOpType() string
- func (o Operations) HasError() bool
- func (o Operations) Hosts() int
- func (o Operations) IsMixed() bool
- func (o Operations) IsMultiTouch() bool
- func (o Operations) Median(m float64) Operation
- func (o Operations) MinMaxSize() (minSize, maxSize int64)
- func (o Operations) MultipleSizes() bool
- func (o Operations) NErrors() int
- func (o Operations) OffsetThreads(n uint16) uint16
- func (o Operations) OpThroughput() Throughput
- func (o Operations) OpTypes() []string
- func (o Operations) Segment(so SegmentOptions) Segments
- func (o Operations) SetClientID(id string)
- func (o Operations) SingleSizeSegment() SizeSegment
- func (o Operations) SortByClient()
- func (o Operations) SortByDuration()
- func (o Operations) SortByEndTime()
- func (o Operations) SortByEndpoint()
- func (o Operations) SortByOpType()
- func (o Operations) SortByStartTime()
- func (o Operations) SortByTTFB()
- func (o Operations) SortByThroughput()
- func (o Operations) SortByThroughputNonZero() Operations
- func (o Operations) SortSplitByClient(prefix string) map[string]Operations
- func (o Operations) SortSplitByEndpoint() map[string]Operations
- func (o Operations) SortSplitByOpType() map[string]Operations
- func (o Operations) SplitSizes(minShare float64) []SizeSegment
- func (o Operations) StdDev() time.Duration
- func (o Operations) TTFB(start, end time.Time) TTFB
- func (o Operations) Threads() int
- func (o Operations) TimeRange() (start, end time.Time)
- func (o Operations) Total(allThreads bool) Segment
- type OpsCollector
- type Put
- type Retention
- type S3Zip
- type Segment
- type SegmentOptions
- type Segments
- func (s Segments) CSV(w io.Writer, hostOvr string) error
- func (s Segments) Clone() Segments
- func (s Segments) Median(m float64) Segment
- func (s Segments) Print(w io.Writer) error
- func (s Segments) SortByObjsPerSec()
- func (s Segments) SortByOpsEnded()
- func (s Segments) SortByThroughput()
- func (s Segments) SortByTime()
- type SizeSegment
- type Snowball
- type Stat
- type TTFB
- type TTFBCmp
- type Throughput
- type Versioned
- type VersionedDistribution
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewOpsCollector ¶ added in v1.1.0
func NewOpsCollector() (Collector, OpsCollector)
NewOpsCollector returns a collector that will collect all operations in memory. After calling Close the returned function can be used to retrieve the operations.
Types ¶
type Benchmark ¶
type Benchmark interface { // Prepare for the benchmark run Prepare(ctx context.Context) error // Start will execute the main benchmark. // Operations should begin executing when the start channel is closed. Start(ctx context.Context, wait chan struct{}) error // Cleanup up after the benchmark run. Cleanup(ctx context.Context) // GetCommon returns the common parameters. GetCommon() *Common }
type Categories ¶ added in v1.1.0
type Categories uint64
Categories is a bitfield that represents potentially several categories.
func NewCategories ¶ added in v1.1.0
func NewCategories(c ...Category) Categories
func (Categories) Split ¶ added in v1.1.0
func (c Categories) Split() []Category
Split returns the categories
func (Categories) String ¶ added in v1.1.0
func (c Categories) String() string
type Category ¶ added in v1.1.0
type Category uint8
A Category allows requests to be separated into different categories.
type CmpReqs ¶ added in v0.5.3
type CmpReqs struct { CmpRequests Before, After CmpRequests }
func (*CmpReqs) Compare ¶ added in v0.5.3
func (c *CmpReqs) Compare(before, after Operations)
type CmpRequests ¶ added in v0.5.3
type CmpSegment ¶
type CmpSegment struct {
Before, After *Segment
ThroughputPerSec float64
ObjPerSec float64
OpsEndedPerSec float64
}
CmpSegment is s comparisons between two segments.
func (*CmpSegment) Compare ¶
func (c *CmpSegment) Compare(before, after Segment)
Compare sets c to a comparison between before and after.
func (CmpSegment) String ¶
func (c CmpSegment) String() string
String returns a string representation of the segment comparison.
type Collector ¶
type Collector interface { // AutoTerm will check if throughput is within 'threshold' (0 -> ) for wantSamples, // when the current operations are split into 'splitInto' segments. // The minimum duration for the calculation can be set as well. // Segment splitting may cause less than this duration to be used. AutoTerm(ctx context.Context, op string, threshold float64, wantSamples, splitInto int, minDur time.Duration) context.Context // Receiver returns the receiver of input Receiver() chan<- Operation // AddOutput allows to add additional inputs. AddOutput(...chan<- Operation) // Close the collector Close() }
func NewNullCollector ¶ added in v0.7.0
func NewNullCollector() Collector
NewNullCollector collects operations, but discards them.
type Common ¶
type Common struct { // Default Put options. PutOpts minio.PutObjectOptions PrepareProgress chan float64 // Custom is returned to server if set by clients. Custom map[string]string // ExtraFlags contains extra flags to add to remote clients. ExtraFlags map[string]string // A source is a source that can be used for a single goroutine. // It will have a shared prefix, if configured so. Source func() generator.Source ExtraOut []chan<- Operation // Error should log an error similar to fmt.Print(data...) Error func(data ...interface{}) Client func() (cl *minio.Client, done func()) Collector Collector Location string Bucket string // Auto termination is set when this is > 0. AutoTermDur time.Duration // ClientIdx is the client index. // Will be 0 if single client. ClientIdx int AutoTermScale float64 Concurrency int // Running in client mode. ClientMode bool Locking bool // Clear bucket before benchmark Clear bool // DiscardOutput output. DiscardOutput bool // indicates if we prefer a terse output useful in lengthy runs // Does destination support versioning? Versioned bool // ratelimiting RpsLimiter *rate.Limiter // Transport used. Transport http.RoundTripper // UpdateStatus UpdateStatus func(s string) }
Common contains common benchmark parameters.
type Comparison ¶
type Comparison struct { TTFB *TTFBCmp Op string Average CmpSegment Fastest CmpSegment Median CmpSegment Slowest CmpSegment Reqs CmpReqs }
Comparison is a comparison between two benchmarks.
func Compare ¶
func Compare(before, after Operations, analysis time.Duration, allThreads bool) (*Comparison, error)
type Delete ¶
type Delete struct { Common CreateObjects int BatchSize int ListExisting bool ListFlat bool ListPrefix string // contains filtered or unexported fields }
Delete benchmarks delete speed.
type Fanout ¶ added in v0.7.3
Fanout benchmarks upload speed.
type Get ¶
type Get struct { Common // Default Get options. GetOpts minio.GetObjectOptions ListPrefix string CreateObjects int Versions int RandomRanges bool RangeSize int64 ListExisting bool ListFlat bool // contains filtered or unexported fields }
Get benchmarks download speed.
type List ¶
type List struct { Common CreateObjects int Versions int NoPrefix bool Metadata bool MaxKeys int // contains filtered or unexported fields }
List benchmarks listing speed.
type Mixed ¶
type Mixed struct { Common Dist *MixedDistribution GetOpts minio.GetObjectOptions StatOpts minio.StatObjectOptions CreateObjects int }
Mixed benchmarks mixed operations all inclusive.
type MixedDistribution ¶
type MixedDistribution struct { // Operation -> distribution. Distribution map[string]float64 // contains filtered or unexported fields }
MixedDistribution keeps track of operation distribution and currently available objects.
func (*MixedDistribution) Generate ¶
func (m *MixedDistribution) Generate(allocObjs int) error
func (*MixedDistribution) Objects ¶ added in v0.2.0
func (m *MixedDistribution) Objects() generator.Objects
type Multipart ¶ added in v0.6.2
type Multipart struct { Common // Default Get options. GetOpts minio.GetObjectOptions ObjName string UploadID string CreateParts int PartStart int // contains filtered or unexported fields }
Multipart benchmarks multipart upload+download speed.
func (*Multipart) AfterPrepare ¶ added in v0.6.2
type MultipartPut ¶ added in v1.1.2
MultipartPut benchmarks multipart upload speed.
func (*MultipartPut) Cleanup ¶ added in v1.1.2
func (g *MultipartPut) Cleanup(ctx context.Context)
Cleanup up after the benchmark run.
type Operation ¶
type Operation struct { Start time.Time `json:"start"` End time.Time `json:"end"` FirstByte *time.Time `json:"first_byte"` OpType string `json:"type"` Err string `json:"err"` File string `json:"file,omitempty"` ClientID string `json:"client_id"` Endpoint string `json:"endpoint"` ObjPerOp int `json:"ops"` Size int64 `json:"size"` Thread uint16 `json:"thread"` Categories Categories `json:"cat"` }
func (Operation) Aggregate ¶
Aggregate the operation into segment if it belongs there. Done returns true if operation is starting after segment ended.
func (Operation) BytesPerSec ¶
func (o Operation) BytesPerSec() Throughput
type Operations ¶
type Operations []Operation
func EmptyOpsCollector ¶ added in v1.1.0
func EmptyOpsCollector() Operations
func OperationsFromCSV ¶
func OperationsFromCSV(r io.Reader, analyzeOnly bool, offset, limit int, log func(msg string, v ...interface{})) (Operations, error)
OperationsFromCSV will load operations from CSV.
func (Operations) ActiveTimeRange ¶
func (o Operations) ActiveTimeRange(allThreads bool) (start, end time.Time)
ActiveTimeRange returns the "active" time range. All threads must have completed at least one request and the last start time of any thread. If there is no active time range both values will be the same.
func (Operations) AvgDuration ¶
func (o Operations) AvgDuration() time.Duration
AvgDuration returns the average operation duration.
func (Operations) AvgSize ¶
func (o Operations) AvgSize() int64
AvgSize returns the average operation size.
func (Operations) ByEndpoint ¶
func (o Operations) ByEndpoint() map[string]Operations
ByEndpoint separates the operations by endpoint.
func (Operations) CSV ¶
func (o Operations) CSV(w io.Writer, comment string) error
CSV will write the operations to w as CSV. The comment, if any, is written at the end of the file, each line prefixed with '# '.
func (Operations) ClientIDs ¶ added in v1.0.0
func (o Operations) ClientIDs(prefix string) []string
func (Operations) Clients ¶ added in v0.3.25
func (o Operations) Clients() int
Clients returns the number of clients.
func (Operations) Clone ¶ added in v0.3.43
func (o Operations) Clone() Operations
Clone the operations.
func (Operations) Duration ¶ added in v0.3.18
func (o Operations) Duration() time.Duration
Duration returns the full duration from start of first operation to end of the last.
func (Operations) Endpoints ¶
func (o Operations) Endpoints() []string
Endpoints returns the endpoints as a sorted slice.
func (Operations) FilterByEndpoint ¶
func (o Operations) FilterByEndpoint(endpoint string) Operations
FilterByEndpoint returns operations run against a specific endpoint. Always returns a copy.
func (Operations) FilterByHasTTFB ¶
func (o Operations) FilterByHasTTFB(hasTTFB bool) Operations
FilterByHasTTFB returns operations that has or has not time to first byte.
func (Operations) FilterByOp ¶
func (o Operations) FilterByOp(opType string) Operations
FilterByOp returns operations of a specific type.
func (Operations) FilterErrors ¶
func (o Operations) FilterErrors() Operations
FilterErrors returns all operations with errors.
func (Operations) FilterFirst ¶ added in v0.3.41
func (o Operations) FilterFirst() Operations
FilterFirst returns the first operation on any file.
func (Operations) FilterInsideRange ¶
func (o Operations) FilterInsideRange(start, end time.Time) Operations
FilterInsideRange returns operations that are inside the specified time range. Operations starting before start or ending after end are discarded.
func (Operations) FilterLast ¶ added in v0.5.4
func (o Operations) FilterLast() Operations
FilterLast returns the last operation on any file.
func (Operations) FilterSuccessful ¶ added in v0.3.10
func (o Operations) FilterSuccessful() Operations
FilterSuccessful returns the successful requests.
func (Operations) FirstObjPerOp ¶
func (o Operations) FirstObjPerOp() int
FirstObjPerOp returns the number of objects per operation of the first entry, or 0 if there are no ops.
func (Operations) FirstObjSize ¶ added in v0.3.0
func (o Operations) FirstObjSize() int64
FirstObjSize returns the size of the first entry, 0 if there are no ops.
func (Operations) FirstOpType ¶
func (o Operations) FirstOpType() string
FirstOpType returns the type of the first entry empty string if there are no ops.
func (Operations) HasError ¶ added in v0.3.27
func (o Operations) HasError() bool
HasError returns whether one or more operations failed.
func (Operations) IsMixed ¶ added in v0.3.0
func (o Operations) IsMixed() bool
IsMixed returns true if different operation types are overlapping.
func (Operations) IsMultiTouch ¶ added in v0.3.41
func (o Operations) IsMultiTouch() bool
IsMultiTouch returns true if the same files are touched multiple times.
func (Operations) Median ¶
func (o Operations) Median(m float64) Operation
Median returns the m part median of the assumed sorted list of operations. m is clamped to the range 0 -> 1.
func (Operations) MinMaxSize ¶
func (o Operations) MinMaxSize() (minSize, maxSize int64)
MinMaxSize returns the minimum and maximum operation sizes.
func (Operations) MultipleSizes ¶
func (o Operations) MultipleSizes() bool
MultipleSizes returns whether there are multiple operation sizes.
func (Operations) NErrors ¶ added in v0.7.5
func (o Operations) NErrors() int
NErrors returns the number of errors found.
func (Operations) OffsetThreads ¶
func (o Operations) OffsetThreads(n uint16) uint16
OffsetThreads adds an offset to all thread ids and returns the next thread number.
func (Operations) OpThroughput ¶
func (o Operations) OpThroughput() Throughput
OpThroughput returns the average throughput in B/s.
func (Operations) OpTypes ¶
func (o Operations) OpTypes() []string
OpTypes returns a list of the operation types in the order they appear if not overlapping or in alphabetical order if mixed.
func (Operations) Segment ¶
func (o Operations) Segment(so SegmentOptions) Segments
Segment will segment the operations o. Operations should be of the same type. Operations will be sorted by start time.
func (Operations) SetClientID ¶
func (o Operations) SetClientID(id string)
SetClientID will set the client ID for all operations.
func (Operations) SingleSizeSegment ¶ added in v0.3.0
func (o Operations) SingleSizeSegment() SizeSegment
func (Operations) SortByClient ¶ added in v1.0.0
func (o Operations) SortByClient()
SortByClient will sort the operations by client. Earliest operations first.
func (Operations) SortByDuration ¶
func (o Operations) SortByDuration()
SortByDuration will sort the operations by duration taken to complete. Fastest operations first.
func (Operations) SortByEndTime ¶ added in v0.3.43
func (o Operations) SortByEndTime()
SortByEndTime will sort the operations by end time. Earliest operations first.
func (Operations) SortByEndpoint ¶ added in v0.6.9
func (o Operations) SortByEndpoint()
SortByEndpoint will sort the operations by end-point. Earliest operations first.
func (Operations) SortByOpType ¶ added in v0.7.5
func (o Operations) SortByOpType()
SortByOpType will sort the operations by operation type. Earliest operations first.
func (Operations) SortByStartTime ¶
func (o Operations) SortByStartTime()
SortByStartTime will sort the operations by start time. Earliest operations first.
func (Operations) SortByTTFB ¶
func (o Operations) SortByTTFB()
SortByTTFB sorts by time to first byte. Smallest first.
func (Operations) SortByThroughput ¶
func (o Operations) SortByThroughput()
SortByThroughput will sort the operations by throughput. Fastest operations first.
func (Operations) SortByThroughputNonZero ¶ added in v1.1.0
func (o Operations) SortByThroughputNonZero() Operations
SortByThroughputNonZero will sort the operations by throughput. Fastest operations first.
func (Operations) SortSplitByClient ¶ added in v1.0.0
func (o Operations) SortSplitByClient(prefix string) map[string]Operations
SortSplitByClient will sort operations by endpoint and split by host.
func (Operations) SortSplitByEndpoint ¶ added in v0.6.9
func (o Operations) SortSplitByEndpoint() map[string]Operations
SortSplitByEndpoint will sort operations by endpoint and split by host.
func (Operations) SortSplitByOpType ¶ added in v0.7.5
func (o Operations) SortSplitByOpType() map[string]Operations
SortSplitByOpType will sort operations by op + start time and split by op.
func (Operations) SplitSizes ¶
func (o Operations) SplitSizes(minShare float64) []SizeSegment
SplitSizes will return log10 separated data. Specify the share of requests that must be in a segment to return it.
func (Operations) StdDev ¶ added in v0.6.7
func (o Operations) StdDev() time.Duration
StdDev returns the standard deviation.
func (Operations) TTFB ¶
func (o Operations) TTFB(start, end time.Time) TTFB
TTFB returns time to first byte stats for all operations completely within the time segment.
func (Operations) Threads ¶
func (o Operations) Threads() int
Threads returns the number of threads found.
func (Operations) TimeRange ¶
func (o Operations) TimeRange() (start, end time.Time)
TimeRange returns the full time range from start of first operation to end of the last.
func (Operations) Total ¶
func (o Operations) Total(allThreads bool) Segment
Total will return the total of active operations. See ActiveTimeRange how this is determined. Specify whether one operation for all threads should be skipped or just a single.
type OpsCollector ¶ added in v1.1.0
type OpsCollector func() Operations
type Put ¶
Put benchmarks upload speed.
type Retention ¶ added in v0.5.2
type Retention struct { Common CreateObjects int Versions int // contains filtered or unexported fields }
Retention benchmarks download speed.
type S3Zip ¶ added in v0.6.4
type S3Zip struct { Common ZipObjName string CreateFiles int // contains filtered or unexported fields }
S3Zip benchmarks download from a zip file.
type Segment ¶
type Segment struct { EndsBefore time.Time `json:"ends_before"` Start time.Time `json:"start"` OpType string `json:"op"` Host string `json:"host"` OpsStarted int `json:"ops_started"` PartialOps int `json:"partial_ops"` FullOps int `json:"full_ops"` OpsEnded int `json:"ops_ended"` Objects float64 `json:"objects"` Errors int `json:"errors"` ReqAvg float64 `json:"req_avg_ms"` // Average duration of operations ending in segment. TotalBytes int64 `json:"total_bytes"` ObjsPerOp int `json:"objects_per_op"` }
A Segment represents totals of operations in a specific time segment starting at Start and ending before EndsBefore.
func (Segment) ShortString ¶
ShortString returns a string representation of the segment without ops ended/s.
func (Segment) SpeedPerSec ¶
SpeedPerSec returns mb/s for the segment and the ops ended per second.
type SegmentOptions ¶
type SegmentOptions struct { From time.Time PerSegDuration time.Duration AllThreads bool MultiOp bool }
SegmentOptions describe options used to segment operations.
type Segments ¶
type Segments []Segment
Segments is a slice of segment elements.
func (Segments) SortByObjsPerSec ¶
func (s Segments) SortByObjsPerSec()
SortByObjsPerSec sorts the segments by the number of distributed objects processed. Lowest first.
func (Segments) SortByOpsEnded ¶
func (s Segments) SortByOpsEnded()
SortByOpsEnded sorts the segments by the number of ops ended in segment. Lowest first.
func (Segments) SortByThroughput ¶
func (s Segments) SortByThroughput()
SortByThroughput sorts the segments by throughput. Slowest first.
func (Segments) SortByTime ¶
func (s Segments) SortByTime()
SortByTime sorts the segments by start time. Earliest first.
type SizeSegment ¶
type SizeSegment struct { Ops Operations Smallest int64 SmallestLog10 int Biggest int64 BiggestLog10 int }
SizeSegment is a size segment.
func (SizeSegment) SizeString ¶
func (s SizeSegment) SizeString() string
SizeString returns the size as a string.
func (SizeSegment) SizesString ¶ added in v0.3.0
func (s SizeSegment) SizesString() (lo, hi string)
SizesString returns the lower and upper limit as strings.
type Snowball ¶ added in v0.6.8
type Snowball struct { Common NumObjs int // Number objects in each snowball. WindowSize int Duplicate bool // Duplicate object content. Compress bool // Zstandard compress snowball. // contains filtered or unexported fields }
Snowball benchmarks snowball upload speed.
type Stat ¶
type Stat struct { Common // Default Stat options. StatOpts minio.StatObjectOptions ListPrefix string CreateObjects int Versions int ListExisting bool ListFlat bool // contains filtered or unexported fields }
Stat benchmarks HEAD speed.
type TTFB ¶
type TTFB struct { Average time.Duration Best time.Duration P25 time.Duration Median time.Duration P75 time.Duration P90 time.Duration P99 time.Duration Worst time.Duration StdDev time.Duration Percentiles [101]time.Duration `json:"percentiles_millis"` }
TTFB contains time to first byte stats.
type Throughput ¶
type Throughput float64
Throughput is the throughput as bytes/second.
func (Throughput) Float ¶ added in v0.3.0
func (t Throughput) Float() float64
Float returns a rounded (to 0.1) float value of the throughput.
func (Throughput) String ¶
func (t Throughput) String() string
type Versioned ¶ added in v0.3.16
type Versioned struct { Common Dist *VersionedDistribution GetOpts minio.GetObjectOptions StatOpts minio.StatObjectOptions CreateObjects int }
Versioned benchmarks mixed operations all inclusive.
type VersionedDistribution ¶ added in v0.3.16
type VersionedDistribution struct { // Operation -> distribution. Distribution map[string]float64 // contains filtered or unexported fields }
VersionedDistribution keeps track of operation distribution and currently available objects.
func (*VersionedDistribution) Generate ¶ added in v0.3.16
func (m *VersionedDistribution) Generate(allocObjs int) error
Generate versioned objects.
func (*VersionedDistribution) Objects ¶ added in v0.3.16
func (m *VersionedDistribution) Objects() generator.Objects