datasets

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 19, 2025 License: Apache-2.0 Imports: 5 Imported by: 0

Documentation

Overview

Package datasets implements the Neurlang dataset type

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AnyTally 

type AnyTally interface {

	// Erase
	Init()

	// Global Premodulo
	IsGlobalPremodulo() bool
	SetGlobalPremodulo(mod uint32)
	GetGlobalSaltPremodulo() [2]uint32
	GetGlobalPremodulo() uint32

	// Pre tallying
	GetCellDecision(position int, feature uint32) (bool, bool)
	SetCellDecision(position int, feature uint32, output bool)

	// Tallying
	AddToCorrect(feature uint32, vote int8, improvement bool)
	AddToImprove(feature uint32, vote int8)
	AddToMapping(feature uint16, output uint64)

	// Get Dataset at
	DatasetAt(n int) Dataset
	GetImprovementPossible() bool

	// Len
	Len() (ret int)
}

func NewAnyTally 

func NewAnyTally(typ TallyType) AnyTally

type Datamap 

type Datamap map[uint16]uint64

Datamap holds keys mapped to values

func (*Datamap) Init 

func (d *Datamap) Init()

Init initializes the Datamap

func (Datamap) Reduce 

func (d Datamap) Reduce(whole bool) (o Datamap)

Reduce reduces the datamap

func (Datamap) Split 

func (d Datamap) Split() (o SplittedDataset)

Split splits datamap into a true set and a false set

type Dataset 

type Dataset map[uint32]bool

Dataset holds keys mapped to booleans

func (*Dataset) Init 

func (d *Dataset) Init()

Init initializes the Dataset

func (Dataset) Split 

func (d Dataset) Split() (o SplittedDataset)

Split splits dataset into a true set and a false set

type PreTally 

type PreTally struct {
	// contains filtered or unexported fields
}

PreTally stores distilled decisions for multiple cells with thread-safe access

func (*PreTally) AddToCorrect 

func (t *PreTally) AddToCorrect(feature uint32, vote int8, improvement bool)

AddToCorrect votes for feature which caused the overall result to be correct

func (*PreTally) AddToImprove 

func (t *PreTally) AddToImprove(feature uint32, vote int8)

AddToImprove votes for feature which caused the overall result to be correct

func (*PreTally) AddToMapping 

func (t *PreTally) AddToMapping(feature uint16, output uint64)

AddToMapping adds feature maps to this output votes to mapping

func (*PreTally) DatasetAt 

func (d *PreTally) DatasetAt(position int) Dataset

func (*PreTally) Free 

func (d *PreTally) Free()

func (*PreTally) GetCellDecision 

func (d *PreTally) GetCellDecision(position int, feature uint32) (bool, bool)

GetCellDecision returns the distilled output for a specific cell and feature

func (*PreTally) GetGlobalPremodulo 

func (t *PreTally) GetGlobalPremodulo() uint32

func (*PreTally) GetGlobalSaltPremodulo 

func (t *PreTally) GetGlobalSaltPremodulo() [2]uint32

func (*PreTally) GetImprovementPossible 

func (t *PreTally) GetImprovementPossible() bool

GetImprovementPossible reads improvementPossible

func (*PreTally) Init 

func (d *PreTally) Init()

Init resets pretally to be empty

func (*PreTally) IsGlobalPremodulo 

func (t *PreTally) IsGlobalPremodulo() bool

func (*PreTally) Len 

func (d *PreTally) Len() (ret int)

func (*PreTally) SetCellDecision 

func (d *PreTally) SetCellDecision(position int, feature uint32, output bool)

SetCellDecision stores a distilled decision for a specific cell and feature

func (*PreTally) SetGlobalPremodulo 

func (t *PreTally) SetGlobalPremodulo(mod uint32)

type SplittedDataset 

type SplittedDataset [2]map[uint32]struct{}

SplittedDataset is a dataset that has been split into the false set and true set

func BalanceDataset 

func BalanceDataset(d SplittedDataset) SplittedDataset

BalanceDataset fills the smaller set with random number until it matches the bigger set

func (*SplittedDataset) Init 

func (d *SplittedDataset) Init()

Init initializes a SplittedDataset

func (SplittedDataset) Split 

func (d SplittedDataset) Split() (o SplittedDataset)

Split splits SplittedDataset into a true set and a false set

type Splitter 

type Splitter interface {
	Split() (o SplittedDataset)
}

Splitter is a dataset that can be split into a SplittedDataset

type Tally 

type Tally struct {
	// contains filtered or unexported fields
}

Tally is used to count votes on dataset features and return the majority votes

func (*Tally) AddToCorrect 

func (t *Tally) AddToCorrect(feature uint32, vote int8, improvement bool)

Correct votes for feature which caused the overall result to be correct

func (*Tally) AddToImprove 

func (t *Tally) AddToImprove(feature uint32, vote int8)

Improve votes for feature which improved the overall result

func (*Tally) AddToMapAll 

func (t *Tally) AddToMapAll(feature uint16, output uint64, loss func(n uint32) uint32, max uint32)

AddToMapAll adds feature maps to all output votes to mapping

func (*Tally) AddToMapping 

func (t *Tally) AddToMapping(feature uint16, output uint64)

AddToMap adds feature maps to this output votes to mapping

func (*Tally) Dataset 

func (t *Tally) Dataset() Dataset

Dataset gets binary Dataset from tally

func (*Tally) DatasetAt 

func (t *Tally) DatasetAt(n int) Dataset

func (*Tally) Free 

func (t *Tally) Free()

Free frees the memory occupied by tally dataset structure

func (*Tally) GetCellDecision 

func (t *Tally) GetCellDecision(position int, feature uint32) (val bool, avail bool)

GetCellDecision returns the distilled output for a specific cell and feature

func (*Tally) GetGlobalPremodulo 

func (t *Tally) GetGlobalPremodulo() uint32

func (*Tally) GetGlobalSaltPremodulo 

func (t *Tally) GetGlobalSaltPremodulo() [2]uint32

func (*Tally) GetImprovementPossible 

func (t *Tally) GetImprovementPossible() bool

GetImprovementPossible reads improvementPossible

func (*Tally) Init 

func (t *Tally) Init()

Init initializes the tally dataset structure

func (*Tally) IsGlobalPremodulo 

func (t *Tally) IsGlobalPremodulo() bool

func (*Tally) Len 

func (t *Tally) Len() (o int)

Len estimates the size of tally

func (*Tally) SetCellDecision 

func (t *Tally) SetCellDecision(position int, feature uint32, output bool)

SetCellDecision stores a distilled decision for a specific cell and feature

func (*Tally) SetFinalization 

func (t *Tally) SetFinalization(final bool)

SetFinalization sets isFinalization and enables the final stage of training

func (*Tally) SetGlobalPremodulo 

func (t *Tally) SetGlobalPremodulo(mod uint32)

func (*Tally) Split 

func (t *Tally) Split() SplittedDataset

Split Splits the tally structure into a splitted dataset

type TallyType 

type TallyType byte
const FinTallyType TallyType = 1
const PreTallyType TallyType = 2

Source Files

View all Source files

Directories

Path Synopsis
Package isalnum implements the IsAlnum Dataset
 Package isalnum implements the IsAlnum Dataset
Package isvirus contains TLSH hashes of viruses and clean files dataset for machine learning (without leading "T1" characters)
 Package isvirus contains TLSH hashes of viruses and clean files dataset for machine learning (without leading "T1" characters)
Package MNIST is the 60000 + 10000 handwritten digits dataset
 Package MNIST is the 60000 + 10000 handwritten digits dataset

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.