benchmark

package
v1.10.16 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 31, 2026 License: Apache-2.0 Imports: 17 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EqualRestoredTreeHashes 

func EqualRestoredTreeHashes(left, right map[string]string) (bool, string)

EqualRestoredTreeHashes returns (true, "") when left and right contain identical relative-path→SHA-256 mappings. On mismatch it returns false and a human-readable description of the first difference found (sorted by path for deterministic output).

func GenerateDataset 

func GenerateDataset(baseDir string, cfg DatasetConfig) error

GenerateDataset creates a deterministic benchmark dataset under baseDir.

Every generated file is named deterministically as: file_0001.bin, file_0002.bin, ...

func HashRestoredTree 

func HashRestoredTree(root string) (map[string]string, error)

HashRestoredTree walks the directory tree rooted at root deterministically (sorted by relative path) and returns a map of relative path → SHA-256 hex digest of file contents. Empty directories are ignored. This is used to verify that store→restore produces identical user-visible output across independent benchmark runs.

func RecordCPU added in v1.9.0 

func RecordCPU(operationType string, cpuTime time.Duration)

RecordCPU records CPU time spent in each operation phase. Duration should be the CPU time (not wall-clock time) for accurate overhead calculation. Calls made outside Measure are ignored.

func RecordMemory added in v1.9.0 

func RecordMemory(peakMemoryBytes int64, allocations int64)

RecordMemory records peak heap usage and allocation count. Calls made outside Measure are ignored.

func RecordProcessed 

func RecordProcessed(files int, bytes int64)

RecordProcessed increments files and bytes counters for the active Measure call. Calls made outside Measure are ignored.

func RecordStorage added in v1.9.0 

func RecordStorage(logicalBytes, compressedBytes, storedBytes int64)

RecordStorage records storage metrics: logical, compressed, and final stored sizes. Ratios are computed internally for stability across runs. Calls made outside Measure are ignored.

func RecordStructural added in v1.9.0 

func RecordStructural(compressedBlocks, uncompressedBlocks, storeIfSmallerFallbacks int64)

RecordStructural records compression effectiveness: blocks compressed, not compressed, and fallbacks. Calls made outside Measure are ignored.

func RecordThroughput added in v1.9.0 

func RecordThroughput(operationType string, mbps float64)

RecordThroughput records operation-specific throughput in MB/s. Calls made outside Measure are ignored.

func WriteDeterministicFile 

func WriteDeterministicFile(path string, size int64, seed int64) error

WriteDeterministicFile writes size bytes of deterministic pseudo-random content derived from seed to path. It is exported so that external validation helpers (e.g. restore-hash determinism checks) can produce the same byte sequence.

Types

type BenchmarkCase 

type BenchmarkCase struct {
	Name      string
	Run       func(ctx BenchmarkContext) error
	Execution execution.Options
}

BenchmarkCase defines one benchmark scenario execution unit.

func CoreScenarios 

func CoreScenarios(cfg ScenarioConfig) []BenchmarkCase

CoreScenarios returns the v1.7 Step 4 real-world benchmark cases.

type BenchmarkContext 

type BenchmarkContext struct {
	RepoPath string
	DataPath string
}

BenchmarkContext contains per-case isolated paths.

type CommandRunner 

type CommandRunner func(spec CommandSpec) error

CommandRunner executes one benchmark command.

type CommandSpec 

type CommandSpec struct {
	Executable string
	Args       []string
	WorkingDir string
	Env        []string
}

CommandSpec describes a coldkeep command invocation used by benchmark scenarios.

type CorpusBuilder added in v1.9.0 

type CorpusBuilder struct {
	// contains filtered or unexported fields
}

CorpusBuilder orchestrates reproducible corpus generation with versioning.

func NewCorpusBuilder added in v1.9.0 

func NewCorpusBuilder(baseDir string) *CorpusBuilder

NewCorpusBuilder creates a new builder for a corpus location.

func (*CorpusBuilder) GenerateCorpus added in v1.9.0 

func (cb *CorpusBuilder) GenerateCorpus(def CorpusDefinition) error

GenerateCorpus creates all files for a corpus definition with SHA256 validation.

func (*CorpusBuilder) ValidateCorpus added in v1.9.0 

func (cb *CorpusBuilder) ValidateCorpus(def CorpusDefinition) (bool, error)

ValidateCorpus checks all files in a generated corpus match expected hashes and sizes.

type CorpusContent added in v1.9.0 

type CorpusContent struct {
	Type string // "json", "logs", "source", "binary", "jpeg_sim", "zip_sim", "random", "encrypted"
	Seed int64
	// CompressionRatio is estimated compressed_size / original_size for benchmark corpus shaping.
	// Typical interpretation: < 0.5 highly compressible, 0.5-0.8 mixed, > 0.95 already compressed.
	CompressionRatio float64

	// JSON-specific
	JSONObjects int64

	// Log-specific
	LogLines int64
	LogSize  int64

	// Binary/Office-specific
	TextRatio float64 // % of file that is plain text (0.0 - 1.0)

	// Random/Encrypted
	EntropyLevel float64 // 0.0 = zero, 1.0 = maximum (random)
}

CorpusContent describes how to generate file content.

type CorpusDefinition added in v1.9.0 

type CorpusDefinition struct {
	Type        CorpusType
	Version     string // e.g., "v1.0"
	Name        string // descriptive name
	Files       []CorpusFile
	Seed        int64 // deterministic generation seed
	SHA256      string
	Description string
}

CorpusDefinition describes a specific benchmark corpus with stable content.

func StandardCorpora added in v1.9.0 

func StandardCorpora() []CorpusDefinition

StandardCorpora defines all stable benchmark corpora.

type CorpusFile added in v1.9.0 

type CorpusFile struct {
	Name    string // unique filename within corpus
	Size    int64  // size in bytes
	Content CorpusContent
	SHA256  string // perfile validation hash
}

CorpusFile describes a single file within a corpus.

type CorpusRegistry added in v1.9.0 

type CorpusRegistry struct {
	// contains filtered or unexported fields
}

CorpusRegistry manages benchmark corpora for reproducible performance testing.

func NewCorpusRegistry added in v1.9.0 

func NewCorpusRegistry(baseDir string) *CorpusRegistry

NewCorpusRegistry creates a new corpus registry at the specified base directory.

func (*CorpusRegistry) CleanupCorpora added in v1.9.0 

func (cr *CorpusRegistry) CleanupCorpora() error

CleanupCorpora removes all generated corpora to force regeneration.

func (*CorpusRegistry) EnsureCorpora added in v1.9.0 

func (cr *CorpusRegistry) EnsureCorpora() error

EnsureCorpora ensures all standard corpora are generated and validated.

func (*CorpusRegistry) GetCorpusFiles added in v1.9.0 

func (cr *CorpusRegistry) GetCorpusFiles(corpusType CorpusType, version string) ([]string, error)

GetCorpusFiles returns the list of files in a generated corpus.

func (*CorpusRegistry) GetCorpusManifest added in v1.9.0 

func (cr *CorpusRegistry) GetCorpusManifest(corpusType CorpusType, version string) (string, error)

GetCorpusManifest returns the manifest content for a corpus.

func (*CorpusRegistry) GetCorpusPath added in v1.9.0 

func (cr *CorpusRegistry) GetCorpusPath(corpusType CorpusType, version string) string

GetCorpusPath returns the directory path for a specific corpus.

func (*CorpusRegistry) GetCorpusStats added in v1.9.0 

func (cr *CorpusRegistry) GetCorpusStats(corpusType CorpusType) CorpusStats

GetCorpusStats returns statistics for a specific corpus.

func (*CorpusRegistry) PrintCorpusStats added in v1.9.0 

func (cr *CorpusRegistry) PrintCorpusStats()

PrintCorpusStats prints human-readable corpus statistics.

func (*CorpusRegistry) ValidateAllCorpora added in v1.9.0 

func (cr *CorpusRegistry) ValidateAllCorpora() []error

ValidateAllCorpora checks integrity of all generated corpora.

type CorpusStats added in v1.9.0 

type CorpusStats struct {
	Type      CorpusType
	Version   string
	Name      string
	FileCount int
	TotalSize int64
	AvgRatio  float64
	Generated bool
}

CorpusStats provides summary statistics about a corpus.

type CorpusType added in v1.9.0 

type CorpusType string

CorpusType represents different benchmark corpus classifications. 

const (
	// CorpusTypeHighlyCompressible: source code, JSON, logs, plaintext with high redundancy.
	CorpusTypeHighlyCompressible CorpusType = "highly_compressible"
	// CorpusMixedRealistic: office files, sqlite databases, binaries, mixed real-world files.
	CorpusTypeMixedRealistic CorpusType = "mixed_realistic"
	// CorpusTypeAlreadyCompressed: JPEG, MP4, ZIP, PDF - already optimized formats.
	CorpusTypeAlreadyCompressed CorpusType = "already_compressed"
	// CorpusTypeAdversarial: random bytes, encrypted blobs - should skip compression.
	CorpusTypeAdversarial CorpusType = "adversarial_random"
)

type DatasetConfig 

type DatasetConfig struct {
	NumFiles      int
	FileSizeBytes int
	Pattern       string // "random" | "repeated" | "mixed"
	Seed          int64
}

DatasetConfig controls synthetic dataset generation for benchmark runs.

type DatasetPreset 

type DatasetPreset string
const (
	DatasetPresetSmall  DatasetPreset = "small"
	DatasetPresetMedium DatasetPreset = "medium"
	DatasetPresetLarge  DatasetPreset = "large"
)

func ParseDatasetPreset 

func ParseDatasetPreset(raw string) (DatasetPreset, error)

type IterationReport 

type IterationReport struct {
	Iteration int      `json:"iteration"`
	Results   []Result `json:"results"`
}

type Metrics 

type Metrics struct {
	// Core operation metrics
	Duration       time.Duration
	FilesProcessed int
	BytesProcessed int64
	ThroughputMBps float64

	// Storage Metrics: distinguish logical vs physical data sizes
	LogicalBytes    int64 // input data size (uncompressed)
	CompressedBytes int64 // size after compression (before storage overhead)
	StoredBytes     int64 // final data on disk (storage + encryption + overhead)
	// CompressionRatio is the size fraction CompressedBytes / LogicalBytes.
	// This benchmark metric is intentionally <= 1.0 for compressible data.
	// Storage/read-path docs use the inverse term CompressionFactor.
	CompressionRatio float64
	PhysicalRatio    float64 // StoredBytes / LogicalBytes, includes all overhead

	// Throughput Metrics: operation-specific throughput
	StoreMBps   float64 // MB/s during store operation
	RestoreMBps float64 // MB/s during restore operation
	VerifyMBps  float64 // MB/s during verify operation

	// CPU Metrics: time spent in each operation phase
	CompressionCPUTime time.Duration // CPU time for compression phase
	RestoreCPUTime     time.Duration // CPU time for restore/decompression
	VerifyCPUTime      time.Duration // CPU time for verify operation

	// Memory Metrics: heap and allocation pressure
	PeakMemoryBytes int64 // peak heap usage during operation
	AllocationCount int64 // number of allocations (alloc churn indicator)

	// Structural Metrics: compression effectiveness tracking
	CompressedBlocks       int64 // blocks where compression was applied
	UncompressedBlocks     int64 // blocks stored as-is (compression ineffective or skipped)
	StoreIfSmallerFallback int64 // instances where uncompressed fallback was used
}

Metrics captures comprehensive benchmark run measurements across five categories: Storage, Throughput, CPU, Memory, and Structural metrics.

Storage Metrics distinguish logical input size from compressed and stored sizes, enabling analysis of compression effectiveness and storage transform overhead.

Throughput Metrics track MB/s for store, restore, and verify operations separately, allowing identification of CPU-intensive transformations.

CPU Metrics segment CPU time by operation phase to identify compression overhead.

Memory Metrics monitor peak heap usage and allocation churn from transforms.

Structural Metrics track compression decisions (blocks compressed vs not, fallbacks) for repositories with mixed stored/uncompressed content.

func Measure 

func Measure(fn func() error) (Metrics, error)

Measure executes fn, captures elapsed time, and computes throughput.

Throughput is calculated as: MB/s = bytes / duration

The fn can report processed counters via RecordProcessed, RecordStorage, RecordCPU, RecordMemory, and RecordStructural.

type Result 

type Result struct {
	Name      string
	Duration  time.Duration
	Metrics   Metrics
	Execution execution.Options
	ExecStats execution.ExecutionStats
	Success   bool
	Error     string
}

Result captures one benchmark case execution outcome.

func RunBenchmark 

func RunBenchmark(cases []BenchmarkCase) ([]Result, error)

RunBenchmark executes benchmark cases sequentially with isolated temp paths.

type RunReport 

type RunReport struct {
	GeneratedAtUTC string            `json:"generated_at_utc"`
	Dataset        DatasetPreset     `json:"dataset"`
	Repeat         int               `json:"repeat"`
	Iterations     []IterationReport `json:"iterations"`
}

func RunPreset 

func RunPreset(preset DatasetPreset, repeat int, base ScenarioConfig) (RunReport, error)

type ScenarioConfig 

type ScenarioConfig struct {
	ColdkeepExecutable     string
	Codec                  string
	Compression            string
	Execution              execution.Options
	Seed                   int64
	LargeFileSizeBytes     int64
	ManySmallFileCount     int
	ManySmallFileSizeBytes int
	MixedFileCount         int
	MixedMinFileSizeBytes  int
	MixedMaxFileSizeBytes  int
	RemoveEvery            int
	RunTag                 string
	ExtraEnv               map[string]string
	Runner                 CommandRunner
}

ScenarioConfig controls Step 4 core benchmark scenarios.

func PresetScenarioConfig 

func PresetScenarioConfig(preset DatasetPreset) (ScenarioConfig, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.