streaming

package
v0.1.9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 10, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Reduce

func Reduce[T, R any](s Stream[T], fn func(<-chan T) (R, error)) (R, error)

Reduce hands s's source channel to fn, drains any remaining items after fn returns, and joins fn's returned error with the stream error state.

func SendContext

func SendContext[T any](ctx context.Context, pipe chan<- T, item T) bool

SendContext sends item to pipe unless ctx has already been canceled.

func SetWorkerErrorWrapper

func SetWorkerErrorWrapper(wrap func(error) error)

SetWorkerErrorWrapper installs the root-owned worker error wrapper used by concurrent stream operations.

Types

type Group

type Group[K comparable, T any] struct {
	Key   K
	Items []T
}

Group holds one grouping key plus the items assigned to that key.

type Option

type Option = config.Option

type Stream

type Stream[T any] struct {
	// contains filtered or unexported fields
}

Stream is a lazy sequence of values backed by a channel plus shared error state that records upstream worker failures.

func Chunk

func Chunk[T any](s Stream[T], n int) Stream[[]T]

Chunk groups items into slices of size n, emitting a final short chunk when the source ends.

func Concat

func Concat[T any](s Stream[T], others ...Stream[T]) Stream[T]

Concat merges s with others and returns a stream that emits items from all inputs as they arrive.

func DistinctBy

func DistinctBy[T any, K comparable](s Stream[T], fn func(T) K) Stream[T]

DistinctBy keeps the first item for each key produced by fn.

func DistinctByCount

func DistinctByCount[T any, K comparable](s Stream[T], n int, fn func(T) K) Stream[T]

DistinctByCount keeps the first item for each key within windows of n input items, resetting the seen-key set after every window.

func DistinctByWindow

func DistinctByWindow[T any, K comparable](ctx context.Context, s Stream[T], every time.Duration, fn func(T) K) Stream[T]

DistinctByWindow keeps the first item for each key within a time window that starts when the first item in that window arrives.

func FlatMap

func FlatMap[T, U any](s Stream[T], fn func(T, chan<- U), opts ...Option) Stream[U]

FlatMap calls fn for each item and lets fn emit zero or more output values.

func FlatMapContext

func FlatMapContext[T, U any](ctx context.Context, s Stream[T], fn func(context.Context, T, chan<- U), opts ...Option) Stream[U]

FlatMapContext is FlatMap with a caller-provided context passed into each worker.

func FlatMapContextErr

func FlatMapContextErr[T, U any](ctx context.Context, s Stream[T], fn func(context.Context, T, chan<- U) error, opts ...Option) Stream[U]

FlatMapContextErr is FlatMapErr with a caller-provided context passed into each worker.

func FlatMapErr

func FlatMapErr[T, U any](s Stream[T], fn func(T, chan<- U) error, opts ...Option) Stream[U]

FlatMapErr calls fn for each item and records any returned worker error in the stream state.

func FlatStage

func FlatStage[I, O any](ctx context.Context, in Stream[I], fn func(context.Context, I, chan<- O), opts ...Option) Stream[O]

FlatStage applies fn to each item in in and lets fn emit zero or more output values.

func FlatStageErr

func FlatStageErr[I, O any](ctx context.Context, in Stream[I], fn func(context.Context, I, chan<- O) error, opts ...Option) Stream[O]

FlatStageErr applies fn to each item in in, lets fn emit zero or more output values, and records returned worker errors in the stream state.

func From

func From[T any](generate func(chan<- T)) Stream[T]

From adapts a producer callback into a stream. Panics from generate are captured in the stream state and surfaced by terminal operations.

func FromChan

func FromChan[T any](source <-chan T) Stream[T]

FromChan wraps source as a Stream without changing its production semantics.

func GroupBy

func GroupBy[T any, K comparable](s Stream[T], fn func(T) K) Stream[[]T]

GroupBy drains s, groups items by fn, and emits groups in first-seen key order.

func GroupByCount

func GroupByCount[T any, K comparable](s Stream[T], n int, fn func(T) K) Stream[Group[K, T]]

GroupByCount groups items by fn within windows of n input items and emits one Group per key in first-seen order for each window.

func GroupByWindow

func GroupByWindow[T any, K comparable](ctx context.Context, s Stream[T], every time.Duration, fn func(T) K) Stream[Group[K, T]]

GroupByWindow groups items by fn within a time window that starts when the first item in that window arrives and flushes on timer tick or source close.

func Map

func Map[T, U any](s Stream[T], fn func(T) U, opts ...Option) Stream[U]

Map applies fn to each item in s and emits the mapped values.

func MapContext

func MapContext[T, U any](ctx context.Context, s Stream[T], fn func(context.Context, T) U, opts ...Option) Stream[U]

MapContext is Map with a caller-provided context passed into each worker.

func MapContextErr

func MapContextErr[T, U any](ctx context.Context, s Stream[T], fn func(context.Context, T) (U, error), opts ...Option) Stream[U]

MapContextErr is MapErr with a caller-provided context passed into each worker.

func MapErr

func MapErr[T, U any](s Stream[T], fn func(T) (U, error), opts ...Option) Stream[U]

MapErr applies fn to each item in s and records any returned error in the stream state.

func New

func New[T any](source <-chan T) Stream[T]

New wraps source with a fresh local error state.

func Stage

func Stage[I, O any](ctx context.Context, in Stream[I], fn func(context.Context, I) O, opts ...Option) Stream[O]

Stage applies fn to each item in in and emits the mapped values.

func StageErr

func StageErr[I, O any](ctx context.Context, in Stream[I], fn func(context.Context, I) (O, error), opts ...Option) Stream[O]

StageErr applies fn to each item in in and records returned worker errors in the stream state.

func Tap

func Tap[T any](ctx context.Context, in Stream[T], fn func(context.Context, T) error, opts ...Option) Stream[T]

Tap runs fn for each item in in and re-emits the original item when fn succeeds.

func Values

func Values[T any](items ...T) Stream[T]

Values returns a stream that emits items in order and then closes.

func WithState

func WithState[T any](source <-chan T, handle state.Handle) Stream[T]

WithState wraps source with handle, creating a fresh state when nil is provided.

func WithStateAndLink[T any](source <-chan T, handle state.Handle, linkMeter *link.Meter) Stream[T]

WithStateAndLink wraps source with handle and optional outbound link meter.

func (Stream[T]) AllMatch

func (s Stream[T]) AllMatch(predicate func(T) bool) bool

AllMatch reports whether every item satisfies predicate. It drains the remainder of the stream after the first mismatch so delayed fail-fast errors can still surface.

func (Stream[T]) AllMatchErr

func (s Stream[T]) AllMatchErr(predicate func(T) bool) (bool, error)

AllMatchErr reports whether every item satisfies predicate and returns the current error state when it short-circuits.

func (Stream[T]) AnyMatch

func (s Stream[T]) AnyMatch(predicate func(T) bool) bool

AnyMatch reports whether any item satisfies predicate. It drains the remainder of the stream after the first match so delayed fail-fast errors can still surface.

func (Stream[T]) AnyMatchErr

func (s Stream[T]) AnyMatchErr(predicate func(T) bool) (bool, error)

AnyMatchErr reports whether any item satisfies predicate and returns the current error state when it short-circuits.

func (Stream[T]) Buffer

func (s Stream[T]) Buffer(n int) Stream[T]

Buffer inserts a channel buffer of size n between s and the returned stream.

func (Stream[T]) Collect

func (s Stream[T]) Collect() []T

Collect drains the stream into a slice and panics on fail-fast errors.

func (Stream[T]) CollectErr

func (s Stream[T]) CollectErr() ([]T, error)

CollectErr drains the stream into a slice and returns the final error state.

func (Stream[T]) Concat

func (s Stream[T]) Concat(others ...Stream[T]) Stream[T]

Concat merges s with others while preserving per-stream item order. Items from different input streams may interleave based on runtime scheduling.

func (Stream[T]) Count

func (s Stream[T]) Count() (count int)

Count drains the stream and returns the number of items it produced.

func (Stream[T]) CountErr

func (s Stream[T]) CountErr() (int, error)

CountErr drains the stream, returns the item count, and returns the final error state.

func (Stream[T]) Done

func (s Stream[T]) Done()

Done drains the stream and panics if a fail-fast error was recorded.

func (Stream[T]) DoneErr

func (s Stream[T]) DoneErr() error

DoneErr drains the stream and returns the final error state.

func (Stream[T]) Err

func (s Stream[T]) Err() error

Err returns the currently accumulated stream error without draining the stream.

func (Stream[T]) Filter

func (s Stream[T]) Filter(fn func(T) bool, opts ...Option) Stream[T]

Filter keeps only the items for which fn returns true.

func (Stream[T]) First

func (s Stream[T]) First() (T, bool)

First returns the first item and then drains the rest of the stream so any delayed fail-fast error is observed before the call returns.

func (Stream[T]) FirstErr

func (s Stream[T]) FirstErr() (T, bool, error)

FirstErr returns the first item and the current error state, then drains the remaining source asynchronously.

func (Stream[T]) ForAll

func (s Stream[T]) ForAll(fn func(<-chan T))

ForAll hands the raw source channel to fn, then drains any leftovers and applies fail-fast panic behavior.

func (Stream[T]) ForAllErr

func (s Stream[T]) ForAllErr(fn func(<-chan T)) error

ForAllErr hands the raw source channel to fn, then drains any leftovers and returns the final error state.

func (Stream[T]) ForEach

func (s Stream[T]) ForEach(fn func(T))

ForEach calls fn for every item in the stream and panics if a fail-fast error was recorded.

func (Stream[T]) ForEachErr

func (s Stream[T]) ForEachErr(fn func(T)) error

ForEachErr calls fn for every item in the stream and returns the final error state.

func (Stream[T]) Head

func (s Stream[T]) Head(n int64) Stream[T]

Head returns a stream containing at most the first n items from s. The upstream source is drained after the head is satisfied so producers can exit.

func (Stream[T]) Last

func (s Stream[T]) Last() (T, bool)

Last drains the stream and returns the last item it observed.

func (Stream[T]) LastErr

func (s Stream[T]) LastErr() (T, bool, error)

LastErr drains the stream, returns the last item it observed, and returns the final error state.

func (Stream[T]) Max

func (s Stream[T]) Max(less func(T, T) bool) (T, bool)

Max drains the stream and returns the greatest item according to less.

func (Stream[T]) MaxErr

func (s Stream[T]) MaxErr(less func(T, T) bool) (T, bool, error)

MaxErr drains the stream, returns the greatest item according to less, and returns the final error state.

func (Stream[T]) Min

func (s Stream[T]) Min(less func(T, T) bool) (T, bool)

Min drains the stream and returns the least item according to less.

func (Stream[T]) MinErr

func (s Stream[T]) MinErr(less func(T, T) bool) (T, bool, error)

MinErr drains the stream, returns the least item according to less, and returns the final error state.

func (Stream[T]) NoneMatch

func (s Stream[T]) NoneMatch(predicate func(T) bool) bool

NoneMatch reports whether no item satisfies predicate. It drains the remainder of the stream after the first match so delayed fail-fast errors can still surface.

func (Stream[T]) NoneMatchErr

func (s Stream[T]) NoneMatchErr(predicate func(T) bool) (bool, error)

NoneMatchErr reports whether no item satisfies predicate and returns the current error state when it short-circuits.

func (Stream[T]) Parallel

func (s Stream[T]) Parallel(fn func(T), opts ...Option)

Parallel applies fn to each item using the same worker machinery as the transform operators and panics on fail-fast errors.

func (Stream[T]) ParallelErr

func (s Stream[T]) ParallelErr(fn func(T) error, opts ...Option) error

ParallelErr applies fn to each item using worker options and returns the final error state.

func (Stream[T]) Reverse

func (s Stream[T]) Reverse() Stream[T]

Reverse drains s, reverses the collected items, and replays them as a new stream.

func (Stream[T]) Skip

func (s Stream[T]) Skip(n int64) Stream[T]

Skip discards the first n items from s and emits the remainder.

func (Stream[T]) Sort

func (s Stream[T]) Sort(less func(T, T) bool) Stream[T]

Sort drains s, sorts all items with less, and then replays them as a new stream.

func (Stream[T]) Tail

func (s Stream[T]) Tail(n int64) Stream[T]

Tail returns a stream containing the last n items from s in original order.

func (Stream[T]) Tap

func (s Stream[T]) Tap(ctx context.Context, fn func(context.Context, T) error, opts ...Option) Stream[T]

Tap runs fn for each item in s and re-emits the original item when fn succeeds.

func (Stream[T]) Through

func (s Stream[T]) Through(ctx context.Context, fn func(context.Context, T) T, opts ...Option) Stream[T]

Through applies fn to each item in s and returns another Stream[T].

func (Stream[T]) ThroughErr

func (s Stream[T]) ThroughErr(ctx context.Context, fn func(context.Context, T) (T, error), opts ...Option) Stream[T]

ThroughErr applies fn to each item in s, records returned worker errors in the stream state, and returns another Stream[T].

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL