experiment

package
v3.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 23, 2022 License: MIT Imports: 17 Imported by: 2

Documentation

Overview

Package experiment defines standard evolutionary epochs evaluators and experimental data samples collectors.

Index

Constants

View Source 
const EmptyDuration = time.Duration(-1)

EmptyDuration is to return when average duration can not tbe estimated (empty trials or generations)

Variables

This section is empty.

Functions

This section is empty.

Types

type Experiment 

type Experiment struct {
	Id       int
	Name     string
	RandSeed int64
	Trials
	// The maximal allowed fitness score as defined by fitness function of experiment.
	// It is used to normalize fitness score value used in efficiency score calculation. If this value
	// is not set the fitness score will not be normalized during efficiency score estimation.
	MaxFitnessScore float64
}

An Experiment is a collection of trials for one experiment. It's helpful for analysis of a series of experiments.

func (*Experiment) AvgDiversity 

func (e *Experiment) AvgDiversity() Floats

AvgDiversity returns the average number of species in each trial

func (*Experiment) AvgEpochDuration 

func (e *Experiment) AvgEpochDuration() time.Duration

AvgEpochDuration Calculates average duration of evaluations among all generations of organism populations in this experiment

func (*Experiment) AvgGenerationsPerTrial 

func (e *Experiment) AvgGenerationsPerTrial() float64

AvgGenerationsPerTrial Calculates average number of generations evaluated per trial during this experiment. This can be helpful when estimating algorithm efficiency, because when winner organism is found the trial is terminated, i.e. less evaluations - more fast convergence.

func (*Experiment) AvgTrialDuration 

func (e *Experiment) AvgTrialDuration() time.Duration

AvgTrialDuration Calculates average duration of experiment's trial. Returns EmptyDuration for experiment with no trials. Note, that most trials finish after solution solved, so this metric can be used to represent how efficient the solvers was generated

func (*Experiment) AvgWinnerStatistics 

func (e *Experiment) AvgWinnerStatistics() (avgNodes, avgGenes, avgEvals, avgDiversity float64)

AvgWinnerStatistics calculates the average number of nodes, genes, organisms evaluations, and species diversity of winners among all trials, i.e. for all trials where winning solution was found.

func (*Experiment) BestComplexity 

func (e *Experiment) BestComplexity() Floats

BestComplexity finds the complexity values of the best organisms for each trial.

func (*Experiment) BestFitness 

func (e *Experiment) BestFitness() Floats

BestFitness finds the fitness values of the best organisms for each trial.

func (*Experiment) BestOrganism 

func (e *Experiment) BestOrganism(onlySolvers bool) (*genetics.Organism, int, bool)

BestOrganism Finds the most fit organism among all trials in this experiment. It's also possible to get the best organism only among the ones which was able to solve the experiment's problem. Returns the best fit organism in this experiment among with ID of trial where it was found and boolean value to indicate if search was successful.

func (*Experiment) BestSpeciesAge 

func (e *Experiment) BestSpeciesAge() Floats

BestSpeciesAge finds the age values of the species with the best organisms for each trial.

func (*Experiment) Decode 

func (e *Experiment) Decode(dec *gob.Decoder) error

Decode Decodes experiment data

func (*Experiment) EfficiencyScore 

func (e *Experiment) EfficiencyScore() float64

EfficiencyScore calculates the efficiency score of the found solution.

We are interested in efficient solver solution that take less time per epoch, fewer generations per trial, and produce less complicated winner genomes. At the same time it should have maximal fitness score and maximal success rate among trials. 

This value can only be compared against values obtained for the same type of experiments.

func (*Experiment) Encode 

func (e *Experiment) Encode(enc *gob.Encoder) error

Encode Encodes experiment with GOB encoding

func (*Experiment) EpochsPerTrial 

func (e *Experiment) EpochsPerTrial() Floats

EpochsPerTrial calculates the number of epochs per trial

func (*Experiment) Execute 

func (e *Experiment) Execute(ctx context.Context, startGenome *genetics.Genome, evaluator GenerationEvaluator, trialObserver TrialRunObserver) error

Execute is to run specific experiment using provided startGenome and specific evaluator for each epoch of the experiment

func (*Experiment) MostRecentTrialEvalTime 

func (e *Experiment) MostRecentTrialEvalTime() time.Time

MostRecentTrialEvalTime Returns the time of evaluation of the most recent trial

func (*Experiment) PrintStatistics 

func (e *Experiment) PrintStatistics()

PrintStatistics Prints experiment statistics

func (*Experiment) Read 

func (e *Experiment) Read(r io.Reader) error

Read is to read experiment data from provided reader and decodes it

func (*Experiment) Solved 

func (e *Experiment) Solved() bool

Solved is to check if solution was found in at least one trial

func (*Experiment) SuccessRate 

func (e *Experiment) SuccessRate() float64

SuccessRate calculates the success rate of the experiment as ratio of trials with successful solvers per total number of trials executed.

func (*Experiment) TrialsSolved 

func (e *Experiment) TrialsSolved() int

TrialsSolved calculates the number of trials solved

func (*Experiment) Write 

func (e *Experiment) Write(w io.Writer) error

Write is to write encoded experiment data into provided writer

func (*Experiment) WriteNPZ 

func (e *Experiment) WriteNPZ(w io.Writer) error

WriteNPZ Dumps experiment results to the NPZ file. The file has the following structure: - trials_fitness - the mean, variance of fitness scores per trial - trials_ages - the mean, variance of species ages per trial - trials_complexity - the mean, variance of genome complexity of the best organisms among species per trial - trial_[0...n]_epoch_mean_fitnesses - the mean fitness scores per epoch per trial - trial_[0...n]_epoch_best_fitnesses - the best fitness scores per epoch per trial the same for AGE and COMPLEXITY per epoch per trial - trial_[0...n]_epoch_diversity - the number of species per epoch per trial

type Experiments 

type Experiments []Experiment

Experiments is a sortable list of experiments by execution time and Id

func (Experiments) Len 

func (es Experiments) Len() int

func (Experiments) Less 

func (es Experiments) Less(i, j int) bool

func (Experiments) Swap 

func (es Experiments) Swap(i, j int)

type Floats 

type Floats []float64

Floats provides descriptive statistics on a slice of float64 values

func (Floats) Max 

func (x Floats) Max() float64

Max returns the greatest value in the slice

func (Floats) Mean 

func (x Floats) Mean() float64

Mean returns the average of the values in the slice

func (Floats) MeanVariance 

func (x Floats) MeanVariance() []float64

MeanVariance returns the sample mean and unbiased variance of the values in the slice

func (Floats) Median 

func (x Floats) Median() float64

Median returns the middle value in the slice (50% quantile)

func (Floats) Min 

func (x Floats) Min() float64

Min returns the smallest value in the slice

func (Floats) Q25 

func (x Floats) Q25() float64

Q25 is the 25% quantile

func (Floats) Q75 

func (x Floats) Q75() float64

Q75 is the 75% quantile

func (Floats) StdDev 

func (x Floats) StdDev() float64

StdDev returns the standard deviation of the values in the slice

func (Floats) Sum 

func (x Floats) Sum() float64

Sum returns the total of the values in the slice

func (Floats) Variance 

func (x Floats) Variance() float64

Variance returns the variance of the values in the slice

type Generation 

type Generation struct {
	// The generation ID for this epoch
	Id int
	// The time when epoch was evaluated
	Executed time.Time
	// The elapsed time between generation execution start and finish
	Duration time.Duration
	// The best organism of the best species (probably successful solver if Solved flag set).
	Champion *genetics.Organism
	// The flag to indicate whether experiment was solved in this epoch
	Solved bool

	// The list of the best organisms' fitness values per species in population
	Fitness Floats
	// The age of the best organisms' per species in population
	Age Floats
	// The list of the best organisms' complexities per species in population
	Complexity Floats

	// The number of species in population at the end of this epoch
	Diversity int

	// The number of evaluations done before winner (champion solver) found
	WinnerEvals int
	// The number of nodes in the genome of the winner (champion solver) or zero if not solved
	WinnerNodes int
	// The numbers of genes (links) in the genome of the winner (champion solver) or zero if not solved
	WinnerGenes int

	// The ID of Trial this Generation was evaluated in
	TrialId int
}

Generation the structure to represent execution results of one generation

func (*Generation) Average 

func (g *Generation) Average() (fitness, age, complexity float64)

Average the average fitness, age, and complexity among the best organisms of each species in the population at the end of this epoch

func (*Generation) Decode 

func (g *Generation) Decode(dec *gob.Decoder) error

func (*Generation) Encode 

func (g *Generation) Encode(enc *gob.Encoder) error

Encode is to encode the generation with provided GOB encoder

func (*Generation) FillPopulationStatistics 

func (g *Generation) FillPopulationStatistics(pop *genetics.Population)

FillPopulationStatistics Collects statistics about given population

type GenerationEvaluator 

type GenerationEvaluator interface {
	// GenerationEvaluate Invoked to evaluate one generation of population of organisms within given
	// execution context.
	GenerationEvaluate(ctx context.Context, pop *genetics.Population, epoch *Generation) error
}

GenerationEvaluator the interface describing evaluator for one epoch (generation) of the evolutionary process.

type Generations 

type Generations []Generation

Generations is a sortable collection of generations by execution time and Id

func (Generations) Len 

func (is Generations) Len() int

func (Generations) Less 

func (is Generations) Less(i, j int) bool

func (Generations) Swap 

func (is Generations) Swap(i, j int)

type Trial 

type Trial struct {
	// The trial number
	Id int
	// The results per generation in this trial
	Generations Generations
	// The winner generation
	WinnerGeneration *Generation

	// The elapsed time between trial start and finish
	Duration time.Duration
}

Trial The structure to hold statistics about one experiment run (trial)

func (*Trial) Average 

func (t *Trial) Average() (fitness, age, complexity Floats)

Average the average fitness, age, and complexity of the best organisms per species for each epoch in this trial

func (*Trial) AvgEpochDuration 

func (t *Trial) AvgEpochDuration() time.Duration

AvgEpochDuration Calculates average duration of evaluations among all generations of organism populations in this trial

func (*Trial) BestOrganism 

func (t *Trial) BestOrganism(onlySolvers bool) (*genetics.Organism, bool)

BestOrganism finds the most fit organism among all epochs in this trial. It's also possible to get the best organism only among successful solvers of the experiment's problem.

func (*Trial) ChampionSpeciesAges 

func (t *Trial) ChampionSpeciesAges() Floats

ChampionSpeciesAges returns the age of the species of the champion per generation in this trial

func (*Trial) ChampionsComplexities 

func (t *Trial) ChampionsComplexities() Floats

ChampionsComplexities returns the complexities of the champion organisms per generation in this trial

func (*Trial) ChampionsFitness 

func (t *Trial) ChampionsFitness() Floats

ChampionsFitness returns the fitness values of the champion organisms per generation in this trial

func (*Trial) Decode 

func (t *Trial) Decode(dec *gob.Decoder) error

Decode Decodes trial data

func (*Trial) Diversity 

func (t *Trial) Diversity() Floats

Diversity returns number of species for each epoch

func (*Trial) Encode 

func (t *Trial) Encode(enc *gob.Encoder) error

Encode is to encode this trial

func (*Trial) RecentEpochEvalTime 

func (t *Trial) RecentEpochEvalTime() time.Time

RecentEpochEvalTime is to get time of the epoch executed most recently within this trial

func (*Trial) Solved 

func (t *Trial) Solved() bool

func (*Trial) WinnerStatistics 

func (t *Trial) WinnerStatistics() (nodes, genes, evals, diversity int)

WinnerStatistics finds the number of nodes, genes of the winner genome as well as number of winner organism evaluations and species diversity in the population with successful solver.

type TrialRunObserver 

type TrialRunObserver interface {
	// TrialRunStarted invoked to notify that new trial run just started. Invoked before any epoch evaluation in that trial run
	TrialRunStarted(trial *Trial)
	// TrialRunFinished invoked to notify that the trial run just finished. Invoked after all epochs evaluated or successful solver found.
	TrialRunFinished(trial *Trial)
	// EpochEvaluated invoked to notify that evaluation of specific epoch completed.
	EpochEvaluated(trial *Trial, epoch *Generation)
}

TrialRunObserver defines observer to be notified about experiment's trial lifecycle methods

type Trials 

type Trials []Trial

Trials is a sortable collection of experiment runs (trials) by execution time and id

func (Trials) Len 

func (ts Trials) Len() int

func (Trials) Less 

func (ts Trials) Less(i, j int) bool

func (Trials) Swap 

func (ts Trials) Swap(i, j int)

Source Files

View all Source files

Directories

Path Synopsis
Package utils provides common utilities to be used by experiments.
 Package utils provides common utilities to be used by experiments.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.