Documentation
¶
Overview ¶
Package runtime provides the core processing engine that executes a topology. It handles source reading, processor chain execution, sink writing, and coordinates checkpointing and backpressure.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine executes a topology by reading from sources, processing through processor chains, and writing to sinks.
type EngineOption ¶
type EngineOption func(*Engine)
EngineOption configures the engine.
func WithBackpressure ¶
func WithBackpressure(ctrl *backpressure.Controller) EngineOption
WithBackpressure sets the backpressure controller.
func WithCoordinator ¶
func WithCoordinator(c checkpoint.Coordinator) EngineOption
WithCoordinator sets the checkpoint coordinator.
type PartitionAssigner ¶
type PartitionAssigner struct {
// contains filtered or unexported fields
}
PartitionAssigner distributes partitions among processing nodes using consistent hashing for minimal reassignment during scaling.
func NewPartitionAssigner ¶
func NewPartitionAssigner(partitions int) *PartitionAssigner
NewPartitionAssigner creates a partition assigner for the given number of partitions.
func (*PartitionAssigner) GetAssignment ¶
func (pa *PartitionAssigner) GetAssignment(partition int) string
GetAssignment returns which node owns the given partition.
func (*PartitionAssigner) GetNodePartitions ¶
func (pa *PartitionAssigner) GetNodePartitions(nodeID string) []int
GetNodePartitions returns all partitions assigned to a given node.
func (*PartitionAssigner) PartitionForKey ¶
func (pa *PartitionAssigner) PartitionForKey(key []byte) int
PartitionForKey determines the partition for a given key using FNV-1a hashing.
func (*PartitionAssigner) SetNodes ¶
func (pa *PartitionAssigner) SetNodes(nodes []string)
SetNodes updates the set of processing nodes and rebalances partitions.
Source Files