efficiency

Documentation

Overview

Package efficiency offers utilities for generating read-only channels, merging and splitting streams, concurrent processing of channel items, and partitioning key-value stores using shards for scalability and performance.

Index

• func Generate[T any](in ...T) <-chan T
• func Merge[T any](in ...<-chan T) chan T
• func Process[IN, OUT any](in <-chan IN, fn service.Function[IN, OUT]) (<-chan OUT, <-chan error)
• func Split[T any](in <-chan T, num int) []<-chan T
• func WithCompression(next http.Handler) http.Handler
• type Shard
• type Sharding
  • func NewSharding[K comparable, V any](num int) Sharding[K, V]
  • func (a Sharding[K, V]) Delete(key K)
  • func (a Sharding[K, V]) Get(key K) (value V, exists bool)
  • func (a Sharding[K, V]) Put(key K, value V)
• type SparseSet
  • func NewSparseSet[T comparable](initialSize int) *SparseSet[T]
  • func (a *SparseSet[T]) Add(id int, item T)
  • func (a *SparseSet[T]) Densed() []T
  • func (a *SparseSet[T]) Get(id int) *T
  • func (a *SparseSet[T]) Remove(id int)

Functions

func Generate

func Generate[T any](in ...T) <-chan T

Generate takes a variadic input of any type T and returns a read-only channel of type T. It sends each input value into the returned channel in a separate goroutine.

func Merge

func Merge[T any](in ...<-chan T) chan T

Merge combines multiple input channels into a single output channel. It starts a goroutine for each input channel to forward its values to the output channel. Once all input channels are processed, the output channel is closed.

func Process

func Process[IN, OUT any](in <-chan IN, fn service.Function[IN, OUT]) (<-chan OUT, <-chan error)

Process concurrently processes items from the input channel using the provided function `fn`. It spawns a number of worker goroutines equal to the number of available CPU cores.

func Split

func Split[T any](in <-chan T, num int) []<-chan T

Split takes an input channel `in` and an integer `num`. It returns a slice of `num` output channels, where each channel will receive the same data from the input channel concurrently.

func WithCompression

func WithCompression(next http.Handler) http.Handler

WithCompression ...

Types

type Shard

type Shard[K comparable, V any] struct {
	// contains filtered or unexported fields
}

Shard represents a single partition of a sharded key-value store. It holds a map of keys (of type K) to values (of type V), and a mutex for thread-safety.

type Sharding

type Sharding[K comparable, V any] []Shard[K, V]

Sharding is a collection of Shard objects, each representing a separate partition of the key-value store. This allows for distributing keys across multiple shards.

func NewSharding

func NewSharding[K comparable, V any](num int) Sharding[K, V]

NewSharding creates an array of Shard objects with the given number of shards.

func (Sharding[K, V]) Delete

func (a Sharding[K, V]) Delete(key K)

Delete removes a key-value pair from the appropriate shard.

func (Sharding[K, V]) Get

func (a Sharding[K, V]) Get(key K) (value V, exists bool)

Get retrieves a value from the appropriate shard.

func (Sharding[K, V]) Put

func (a Sharding[K, V]) Put(key K, value V)

Put adds a key-value pair to the appropriate shard.

type SparseSet

type SparseSet[T comparable] struct {
	Dense  []T
	Sparse []int
	Size   int
}

SparseSet is a data structure that provides efficient storage and retrieval of elements.

func NewSparseSet

func NewSparseSet[T comparable](initialSize int) *SparseSet[T]

NewSparseSet creates a new SparseSet with the given initial Size.

func (*SparseSet[T]) Add

func (a *SparseSet[T]) Add(id int, item T)

Add adds an element to the SparseSet.

func (*SparseSet[T]) Densed

func (a *SparseSet[T]) Densed() []T

Densed returns the Dense representation of the SparseSet.

func (*SparseSet[T]) Get

func (a *SparseSet[T]) Get(id int) *T

Get returns the element at the given id.

func (*SparseSet[T]) Remove

func (a *SparseSet[T]) Remove(id int)

Remove removes an element from the SparseSet.