sharding

package
v0.0.0-...-9649366 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 21, 2019 License: Apache-2.0 Imports: 7 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// NoShardingSharderID is the sharder id used where no sharding is applicable.
	// It maps all inputs to a single shard.
	NoShardingSharderID = SharderID{/* contains filtered or unexported fields */}
)

Functions

This section is empty.

Types

type AggregatedShardFn

type AggregatedShardFn func(chunkedID id.ChunkedID, numShards int) uint32

AggregatedShardFn maps a chunked id to a shard.

type AggregatedSharder

type AggregatedSharder struct {
	// contains filtered or unexported fields
}

AggregatedSharder maps an aggregated metric to a shard.

func NewAggregatedSharder

func NewAggregatedSharder(sharderID SharderID) (AggregatedSharder, error)

NewAggregatedSharder creates a new aggregated sharder.

func (*AggregatedSharder) ID

func (s *AggregatedSharder) ID() SharderID

ID returns the sharder id.

func (*AggregatedSharder) Shard

func (s *AggregatedSharder) Shard(chunkedID id.ChunkedID) uint32

Shard maps a chunked id to a shard.

type HashType

type HashType string

HashType is the hashing type.

const (
	// Murmur32Hash represents the murmur3 hash.
	Murmur32Hash HashType = "murmur32"

	DefaultHash = Murmur32Hash
)

List of supported hashing types.

func (HashType) AggregatedShardFn

func (t HashType) AggregatedShardFn() (AggregatedShardFn, error)

AggregatedShardFn returns the sharding function for computing aggregated shards.

func (HashType) MustShardFn

func (t HashType) MustShardFn() ShardFn

MustShardFn returns the sharding function, or panics if an error is encountered.

func (HashType) ShardFn

func (t HashType) ShardFn() (ShardFn, error)

ShardFn returns the sharding function.

func (*HashType) UnmarshalYAML

func (t *HashType) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML unmarshals YAML object into a hash type.

type ShardFn

type ShardFn func(id []byte, numShards uint32) uint32

ShardFn maps a id to a shard.

type ShardSet

type ShardSet map[uint32]struct{}

ShardSet is a collection of shards organized as a set. The shards contained in the set can be discontinuous.

func MustParseShardSet

func MustParseShardSet(s string) ShardSet

MustParseShardSet parses a shard set from the input string, and panics if parsing is unsuccessful.

func ParseShardSet

func ParseShardSet(s string) (ShardSet, error)

ParseShardSet parses a shard set from the input string.

func (ShardSet) Add

func (ss ShardSet) Add(p uint32)

Add adds the shard to the set.

func (ShardSet) AddBetween

func (ss ShardSet) AddBetween(minInclusive, maxExclusive uint32)

AddBetween adds shards between the given min (inclusive) and max (exclusive).

func (ShardSet) Contains

func (ss ShardSet) Contains(p uint32) bool

Contains returns true if the shard set contains the given shard.

func (ShardSet) ParseRange

func (ss ShardSet) ParseRange(s string) error

ParseRange parses a range of shards and adds them to the set.

func (*ShardSet) UnmarshalYAML

func (ss *ShardSet) UnmarshalYAML(f func(interface{}) error) error

UnmarshalYAML unmarshals YAML into a shard set. The following formats are supported: * StartShard..EndShard, e.g., 0..63. * Single shard, e.g., 5. * Array containing shard ranges and single shards.

type SharderID

type SharderID struct {
	// contains filtered or unexported fields
}

SharderID uniquely identifies a sharder.

func NewSharderID

func NewSharderID(hashType HashType, numShards int) SharderID

NewSharderID creates a new sharder id.

func (SharderID) NumShards

func (sid SharderID) NumShards() int

NumShards returns the total number of shards.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL