iterkit

package
v0.300.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 3, 2025 License: Apache-2.0 Imports: 12 Imported by: 2

README

iterkit

Go Reference

iterkit is a Go package that streamlines working with data sequences, prioritising an intuitive developer experience. The package's helper functions are designed with streaming in mind, ensuring mindful memory usage. It offers various iterator implementations and utilities that enable developers to process, transform, and manage data efficiently.

Installation

To integrate iterkit into your Go project, run:

go get go.llib.dev/frameless/pkg/iterkit

Working with Failable Iterators

iterkit aims to make it easier to work with iterators that interact with external resources that can fail. These are represented by the iterkit.SeqE[T] type, which is an alias for iter.Seq2[T, error]. This represents an iterator sequence that may potentially fail at any time. The name draws inspiration from the standard library's Seq2, but with a key distinction – the suffix "E" highlights the possibility of errors occurring during iteration.

type SeqE[T any] = iter.Seq2[T, error]

For more information on how we chose this approach, see our detailed explanation.

Features

Stream Operations:

iter.Seq iter.Seq2 iterkit.SeqE
Collect the values of an iterator Collect Collect2,
Collect2Map,
Collect2KV		 CollectE
Convert slice to iterator Slice1 SliceE
Create an empty iterator Empty Empty2 EmptyE
Transform values between types Map Map2 MapE
Collect all values Collect Collect2,
Collect2KV,
Collect2Map		 CollectE
Filter unwanted values out from a data sequence Filter Filter2 Filter
Merge multiple data sequence into one Merge Merge2 MergeE
Turn the data stream into a chunked/batch data stream Batch BatchE
Make an data stream only consumable once Once Once2 OnceE
Limit number of items Limit,
Head		 Limit2,
Head2		 LimitE,
HeadE
Take N values Take,
TakeAll		 Take2,
TakeAll2		 TakeE,
TakeAllE
Count all elements Count Count2 CountE
Enable FanOut/FanIn with iterators Sync Sync2 SyncE
Create iterator from paginated data source FromPages
Reduce a data sequence into a result value Reduce,
ReduceErr		 Reduce2 ReduceE,
ReduceEErr
,Reduce,
ReduceErr
Consume an iterator to count its elements Count Count2 CountE
Limit the maximum returned element count Limit Limit2 LimitE
Offset what will be the first element in a sequence Offset Offset2 OffsetE
Get the first element from a data sequence First First2 FirstE
Get the last element from a data sequence Last Last2 LastE
Work with a SeqE[T] like it is simple Seq[T] OnSeqEValue

Constructors:

iterkit.SeqE
Create failable data sequence in a clean and idiomatic way From SeqE[T]
Turn a bufio scanner's stream into a data sequence BufioScanner SingleUseSeqE[T]
Create int range between a given boundary IntRange,
IntRangeE		 Seq[int],
SeqE[int]
Create a character range between a given boundary CharRange,
CharRangeE		 Seq[rune],
SeqE[rune]
Turn a Channel into a data sequence Chan,
ChanE		 Seq[T],
SeqE[T]
Create sequence that represent a persistent error Error,
ErrorF		 SeqE[T]
Express a single value as a data sequence Of,
Of2,
OfE		 Seq[T],
Seq2[K, V],
SeqE[T]
Cast a sequence into a SeqE ToSeqE SeqE[T]
Split a SeqE into a Seq and a closer func SplitSeqE Seq[T] + func() error
Express a data sequence as a channel value ToChan chan T
Create a stdlib iterator out from a stateful OOP pull iterator FromPullIter SeqE[T]

Pull Operations:

iter.Seq iter.Seq2 iterkit.SeqE
Take the first N element from a pull iterator Take Take2 TakeE
Take all the elements from a pull iterator TakeAll TakeAll2 TakeAllE
Collect all elements from a pull iterator and close it CollectPull CollectEPull
Convert back a pull iterator into a single use Sequence FromPull FromPull2 FromPullE

Usage

Below is an example demonstrating the use of iterkit to filter and transform a slice:

package main

import (
	"fmt"
	"slices"

	"go.llib.dev/frameless/pkg/iterkit"
)

func main() {
	// Create an iterator from a slice
	numbers := slices.Values([]int{1, 2, 3, 4, 5, 6})

	// Filter even numbers
	evens := iterkit.Filter(numbers, func(n int) bool {
		return n%2 == 0
	})

	// Square each even number
	squares := iterkit.Map(evens, func(n int) int {
		return n * n
	})

	// Collect results into a slice
	result := iterkit.Collect(squares)

	fmt.Println(result) // Output: [4 16 36]
}

Documentation

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Batch 

func Batch[T any](i iter.Seq[T], opts ...BatchOption) iter.Seq[[]T]
Example 
src := iterkit.IntRange(0, 1000)

batched := iterkit.Batch(src)

for vs := range batched {
	fmt.Printf("%#v\n", vs)
}
Example (WithSize) 
src := iterkit.IntRange(0, 1000)

batched := iterkit.Batch(src, iterkit.BatchSize(100))

for vs := range batched {
	fmt.Printf("%#v\n", vs)
}
Example (WithWaitLimit) 
slowIterSeq := iterkit.IntRange(0, 1000)

batched := iterkit.Batch(slowIterSeq, iterkit.BatchWaitLimit(time.Second))

// Batching will occure either when the batching size reached
// or when the wait limit duration passed
for vs := range batched {
	fmt.Printf("%#v\n", vs)
}

func Chan 

func Chan[T any](ch <-chan T) iter.Seq[T]

Chan creates an iterator out from a channel

Example 
ch := make(chan int)

i := iterkit.Chan(ch)

go func() {
	defer close(ch)
	ch <- 42
}()

for v := range i {
	fmt.Println(v) // 42 once
}

func CharRange 

func CharRange(begin, end rune) iter.Seq[rune]

CharRange returns an iterator that will range between the specified `begin“ and the `end` rune.

Example 
for char := range iterkit.CharRange('A', 'Z') {
	// prints characters between A and Z
	// A, B, C, D... Z
	fmt.Println(string(char))
}

func Collect 

func Collect[T any](i iter.Seq[T]) []T
Example 
var itr iter.Seq[int]

ints := iterkit.Collect(itr)
_ = ints

func Collect2 

func Collect2[K, V, KV any](i iter.Seq2[K, V], m kvMapFunc[KV, K, V]) []KV
Example 
var itr iter.Seq2[string, int]

type T struct {
	S string
	I int
}

ints := iterkit.Collect2(itr, func(s string, i int) T {
	return T{S: s, I: i}
})
_ = ints

func Collect2Map added in v0.294.0 

func Collect2Map[K comparable, V any](i iter.Seq2[K, V]) map[K]V

Collect2Map will collect2 an iter.Seq2 into a map.

Example 
var values iter.Seq2[string, int] = func(yield func(string, int) bool) {
	if !yield("foo", 42) {
		return
	}
	if !yield("bar", 7) {
		return
	}
	if !yield("baz", 13) {
		return
	}
}

vs := iterkit.Collect2Map(values)

_ = vs // map[string]int{"foo": 42, "bar": 7, "baz": 13}

func CollectE added in v0.297.0 

func CollectE[T any](i iter.Seq2[T, error]) ([]T, error)
Example 
var itr iter.Seq2[int, error] = func(yield func(int, error) bool) {
	for i := 0; i < 42; i++ {
		if !yield(i, nil) {
			return
		}
	}
}

vs, err := iterkit.CollectE(itr)
_, _ = vs, err

func CollectEPull added in v0.297.0 

func CollectEPull[T any](next func() (T, error, bool), stops ...func()) ([]T, error)
Example 
var itr iter.Seq2[int, error] = iterkit.ToSeqE(iterkit.IntRange(1, 10))
vs, err := iterkit.CollectEPull(iter.Pull2(itr))
_ = vs
_ = err

func CollectErrIter deprecated 

func CollectErrIter[T any](i iter.Seq2[T, error]) ([]T, error)

CollectErrIter is a temporal alias to CollectErr

Deprecated: use iterkit.CollectErr instead

func CollectPull 

func CollectPull[T any](next func() (T, bool), stops ...func()) []T
Example 
var itr iter.Seq[int] = iterkit.IntRange(1, 10)
vs := iterkit.CollectPull(iter.Pull(itr))
_ = vs

func CollectPullIter 

func CollectPullIter[T any](itr PullIter[T]) ([]T, error)

func Count 

func Count[T any](i iter.Seq[T]) int

Count will iterate over and count the total iterations number

Good when all you want is count all the elements in an iterator but don't want to do anything else.

Example 
i := iterkit.Slice1[int]([]int{1, 2, 3})
total := iterkit.Count[int](i)
_ = total // 3

func Count2 

func Count2[K, V any](i iter.Seq2[K, V]) int
Example 
itr := maps.All(map[string]int{
	"foo": 2,
	"bar": 4,
	"baz": 8,
})
iterkit.Count2(itr) // 3

func CountE added in v0.297.0 

func CountE[T any](i SeqE[T]) (int, error)
Example 
i := iterkit.SliceE([]int{1, 2, 3})

n, err := iterkit.CountE(i)
_, _ = n, err

func Empty 

func Empty[T any]() iter.Seq[T]

Empty iterator is used to represent nil result with Null object pattern

Example 
_ = iterkit.Empty[any]()

func Empty2 

func Empty2[T1, T2 any]() iter.Seq2[T1, T2]

Empty2 iterator is used to represent nil result with Null object pattern

func Filter 

func Filter[T any, Iter i1[T]](i Iter, filter func(T) bool) Iter
Example 
var i iter.Seq[int]
i = iterkit.Slice1([]int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9})
i = iterkit.Filter[int](i, func(n int) bool { return n > 2 })

for v := range i {
	fmt.Println(v)
}
Example (WithSeqE) 
numbers := iterkit.IntRangeE(1, 100)
evens := iterkit.Filter(numbers, func(n int) bool {
	return n%2 == 0
})

_ = evens

func Filter2 added in v0.293.0 

func Filter2[K, V any](i iter.Seq2[K, V], filter func(k K, v V) bool) iter.Seq2[K, V]

func First 

func First[T any](i iter.Seq[T]) (T, bool)

First decode the first next value of the iterator and close the iterator

func First2 

func First2[K, V any](i iter.Seq2[K, V]) (K, V, bool)

First2 decode the first next value of the iterator and close the iterator

Example 
var itr iter.Seq2[string, int] = func(yield func(string, int) bool) {
	for i := 0; i < 42; i++ {
		if !yield(strconv.Itoa(i), i) {
			return
		}
	}
}

k, v, ok := iterkit.First2(itr)
_, _, _ = k, v, ok

func FirstE added in v0.297.0 

func FirstE[T any](i SeqE[T]) (T, bool, error)

FirstE will find the first value in a ErrSequence, and return it in a idiomatic tuple return value order.

Example 
var itr iter.Seq2[string, error] = func(yield func(string, error) bool) {
	for i := 0; i < 42; i++ {
		if !yield(strconv.Itoa(i), nil) {
			return
		}
	}
}

v, ok, err := iterkit.FirstE(itr)
_, _, _ = v, ok, err

func FromErrIter deprecated 

func FromErrIter[T any](i SeqE[T]) (iter.Seq[T], func() error)

FromErrIter is a temporal alias to SplitErrSeq

Deprecated: use iterkit.SplitErrSeq instead

func FromKV added in v0.295.0 

func FromKV[K, V any](kvs []KV[K, V]) iter.Seq2[K, V]

func FromPull 

func FromPull[T any](next func() (T, bool), stops ...func()) iter.Seq[T]

func FromPull2 

func FromPull2[K, V any](next func() (K, V, bool), stops ...func()) iter.Seq2[K, V]
func Head[T any](i iter.Seq[T], n int) iter.Seq[T]

Head takes the first n element, similarly how the coreutils "head" app works.

Example 
inf42 := func(yield func(int) bool) {
	for /* infinite */ {
		if !yield(42) {
			return
		}
	}
}

i := iterkit.Head[int](inf42, 3)

vs := iterkit.Collect(i)
_ = vs // []{42, 42, 42}, nil

func Head2 added in v0.294.0 

func Head2[K, V any](i iter.Seq2[K, V], n int) iter.Seq2[K, V]

Head2 takes the first n element, similarly how the coreutils "head" app works.

Example 
inf42 := func(yield func(int, int) bool) {
	for /* infinite */ {
		if !yield(42, 24) {
			return
		}
	}
}

i := iterkit.Head2[int](inf42, 3)

vs := iterkit.Collect2Map(i)
_ = vs // map[int]int{42:24, 42:24, 42:24}, nil

func IntRange 

func IntRange(begin, end int) iter.Seq[int]

IntRange returns an iterator that will range between the specified `begin“ and the `end` int.

Example 
for n := range iterkit.IntRange(1, 9) {
	// prints characters between 1 and 9
	// 1, 2, 3, 4, 5, 6, 7, 8, 9
	fmt.Println(n)
}

func Last 

func Last[T any](i iter.Seq[T]) (T, bool)
Example 
itr := iterkit.IntRange(0, 10)

n, ok := iterkit.Last(itr)
_ = ok // true
_ = n  // 10

func Last2 added in v0.287.0 

func Last2[K, V any](i iter.Seq2[K, V]) (K, V, bool)
Example 
var itr iter.Seq2[int, string] = func(yield func(int, string) bool) {
	for n := range iterkit.IntRange(0, 10) {
		if !yield(n, strconv.Itoa(n)) {
			return
		}
	}
}

num, str, ok := iterkit.Last2(itr)
_ = ok  // true
_ = num // 10
_ = str // "10"

func LastE added in v0.297.0 

func LastE[T any](i SeqE[T]) (T, bool, error)

LastE will find the last value in a ErrSeq, and return it in a idiomatic way. If an error occurs during execution, it will be immediately returned.

Example 
itr := iterkit.IntRangeE(0, 10)

n, ok, err := iterkit.LastE(itr)
_ = err // nil
_ = ok  // true
_ = n   // 10

func Limit 

func Limit[T any](i iter.Seq[T], n int) iter.Seq[T]

func Limit2 added in v0.297.0 

func Limit2[K, V any](i iter.Seq2[K, V], n int) iter.Seq2[K, V]

func Map 

func Map[To any, From any](i iter.Seq[From], transform func(From) To) iter.Seq[To]

Map allows you to do additional transformation on the values. This is useful in cases, where you have to alter the input value, or change the type all together. Like when you read lines from an input stream, and then you map the line content to a certain data structure, in order to not expose what steps needed in order to deserialize the input stream, thus protect the business rules from this information.

Example 
rawNumbers := iterkit.Slice1([]string{"1", "2", "42"})
numbers := iterkit.Map[int](rawNumbers, func(v string) int {
	return len(v)
})
_ = numbers

func Map2 added in v0.294.0 

func Map2[OKey, OVal, IKey, IVal any](i iter.Seq2[IKey, IVal], transform func(IKey, IVal) (OKey, OVal)) iter.Seq2[OKey, OVal]
Example 
itr := maps.All(map[int]string{1: "1", 2: "2", 3: "42"})

numbers := iterkit.Map2[int, int](itr, func(k int, v string) (int, int) {
	return k, len(v)
})

_ = numbers

func Merge 

func Merge[T any](is ...iter.Seq[T]) iter.Seq[T]

func Merge2 

func Merge2[K, V any](is ...iter.Seq2[K, V]) iter.Seq2[K, V]

func Of added in v0.297.0 

func Of[T any](v T) iter.Seq[T]

Of creates an iterator that can return the value of v.

func Of2 added in v0.297.0 

func Of2[K, V any](k K, v V) iter.Seq2[K, V]

Of creates an iterator that can return the value of v.

func Offset 

func Offset[T any](i iter.Seq[T], offset int) iter.Seq[T]
Example 
_ = iterkit.OffsetE(iterkit.IntRangeE(2, 6), 2)

func Offset2 added in v0.297.0 

func Offset2[K, V any](i iter.Seq2[K, V], offset int) iter.Seq2[K, V]
Example 
_ = iterkit.Offset2(iterkit.IntRangeE(2, 6), 2)

func Reduce 

func Reduce[R, T any, I i1[T]](i I, initial R, fn func(R, T) R) (R, error)
Example 
raw := iterkit.Slice1([]int{1, 2, 42})

_, _ = iterkit.Reduce(raw, nil, func(vs []int, v int) []int {
	return append(vs, v)
})

func Reduce1 added in v0.297.0 

func Reduce1[R, T any](i iter.Seq[T], initial R, fn func(R, T) R) R
Example 
raw := iterkit.Slice1([]int{1, 2, 42})

_ = iterkit.Reduce1[[]int](raw, nil, func(vs []int, v int) []int {
	return append(vs, v)
})

func Reduce2 added in v0.297.0 

func Reduce2[R, T any](i iter.Seq[T], initial R, fn func(R, T) R) R

func ReduceE added in v0.297.0 

func ReduceE[R, T any](i SeqE[T], initial R, fn func(R, T) R) (R, error)
Example 
raw := iterkit.SliceE([]int{1, 2, 42})

_, _ = iterkit.ReduceE(raw, nil, func(vs []int, v int) []int {
	return append(vs, v)
})

func ReduceEErr added in v0.297.0 

func ReduceEErr[R, T any](i SeqE[T], initial R, fn func(R, T) (R, error)) (result R, rErr error)

func ReduceErr 

func ReduceErr[R, T any, I i1[T]](i I, initial R, fn func(R, T) (R, error)) (result R, rErr error)
Example 
raw := iterkit.Slice1([]string{"1", "2", "42"})

_, _ = iterkit.ReduceErr[[]int](raw, nil, func(vs []int, raw string) ([]int, error) {

	v, err := strconv.Atoi(raw)
	if err != nil {
		return nil, err
	}
	return append(vs, v), nil

})

func Slice1 added in v0.297.0 

func Slice1[T any](vs []T) iter.Seq[T]

func SplitErrSeq deprecated added in v0.295.0 

func SplitErrSeq[T any](i SeqE[T]) (iter.Seq[T], func() error)

SplitErrSeq is a temporal alias.

Deprecated: use iterkit.SplitSeqE instead

func SplitSeqE added in v0.297.0 

func SplitSeqE[T any](i SeqE[T]) (iter.Seq[T], func() error)

SplitSeqE will split an iter.Seq2[T, error] iterator into a iter.Seq[T] iterator plus an error retrival func.

Example 
var sourceSeqE iter.Seq2[int, error]

i, errFunc := iterkit.SplitSeqE(sourceSeqE)
for v := range i {
	fmt.Println(v)
}
if err := errFunc(); err != nil {
	fmt.Println(err.Error())
}

func Take 

func Take[T any](next func() (T, bool), n int) []T

Take will take the next N value from a pull iterator.

Example 
i := iterkit.Slice1([]int{1, 2, 3})
next, stop := iter.Pull(i)
defer stop()
vs := iterkit.Take(next, 2)
_ = vs // []int{1, 2}

func Take2 added in v0.295.0 

func Take2[KV any, K, V any](next func() (K, V, bool), n int, m kvMapFunc[KV, K, V]) []KV

Take will take the next N value from a pull iterator.

Example 
kvs := maps.All(map[string]int{
	"foo": 42,
	"bar": 7,
	"baz": 13,
})

next, stop := iter.Pull2(kvs)
defer stop()

type E struct {
	Key   string
	Value int
}

es := iterkit.Take2[E](next, 3, func(k string, v int) E {
	return E{Key: k, Value: v}
})

_ = len(es) // 3

func TakeAll 

func TakeAll[T any](next func() (T, bool)) []T

TakeAll will take all the remaining values from a pull iterator.

Example 
i := iterkit.Slice1([]int{1, 2, 3, 4, 5})
next, stop := iter.Pull(i)
defer stop()
vs := iterkit.TakeAll(next)
_ = vs // []int{1, 2, 3, 4, 5}

func TakeAll2 added in v0.297.0 

func TakeAll2[KV any, K, V any](next func() (K, V, bool), m kvMapFunc[KV, K, V]) []KV

TakeAll will take all the remaining values from a pull iterator.

Example 
var i iter.Seq2[int, int] = func(yield func(int, int) bool) {
	for i := range 3 {
		if !yield(i, i*2) {
			return
		}
	}
}

next, stop := iter.Pull2(i)
defer stop()

vs := iterkit.TakeAll2(next, func(n int, v int) int {
	return n + v
})

_ = vs

func TakeAllE added in v0.297.0 

func TakeAllE[T any](next func() (T, error, bool)) ([]T, error)

TakeEAll will take all the remaining values from a pull iterator.

func TakeE added in v0.297.0 

func TakeE[T any](next func() (T, error, bool), n int) ([]T, error)

TakeE will take the next N value from a pull iterator.

Example 
i := iterkit.SliceE([]int{1, 2, 3})
next, stop := iter.Pull2(i)
defer stop()
vs, err := iterkit.TakeE(next, 2)
_ = err // nil
_ = vs  // []int{1, 2}

func ToChan 

func ToChan[T any](itr iter.Seq[T]) (_ <-chan T, cancel func())

Types

type BatchConfig added in v0.295.0 

type BatchConfig struct {
	Size      int
	WaitLimit time.Duration
}

func (BatchConfig) Configure added in v0.295.0 

func (c BatchConfig) Configure(t *BatchConfig)

type BatchOption added in v0.295.0 

type BatchOption option.Option[BatchConfig]

func BatchSize added in v0.295.0 

func BatchSize(n int) BatchOption

func BatchWaitLimit added in v0.295.0 

func BatchWaitLimit(d time.Duration) BatchOption

type EagerSeq added in v0.300.0 

type EagerSeq[T any] = iter.Seq[T]

EagerSeq is an iter.Seq[T] that eagerly loads its content, thus not safe to use with infinite streams.

EagerSeq iterators load all data upfront, meaning they cannot be used with infinite or unbounded sequences. This type is intended for scenarios where the entire sequence must be loaded into memory at once, which is impractical for infinite streams. Using EagerSeq with such streams may lead to memory exhaustion or unexpected behavior.

If an iterator Sequence is eager, it should either have comments for functions or methods that return eager iterators or use the EagerSeq to clearly express it with a return type.

func Reverse 

func Reverse[T any](i iter.Seq[T]) EagerSeq[T]

Reverse will reverse the iteration order.

WARNING, it is not possible to reverse an iteration without first fully consuming it.

Example 
var i iter.Seq[int]
i = iterkit.Reverse(i)

type EagerSeq2 added in v0.300.0 

type EagerSeq2[K, V any] = iter.Seq2[K, V]

EagerSeq2 is an iter.Seq2[K, V] that eagerly loads its content, thus not safe to use with infinite streams.

EagerSeq2 iterators load all data upfront, meaning they cannot be used with infinite or unbounded sequences. This type is intended for scenarios where the entire sequence must be loaded into memory at once, which is impractical for infinite streams. Using EagerSeq2 with such streams may lead to memory exhaustion or unexpected behavior.

If an iterator Sequence is eager, it should either have comments for functions or methods that return eager iterators or use the EagerSeq2 to clearly express it with a return type.

func Reverse2 added in v0.300.0 

func Reverse2[K, V any](i iter.Seq2[K, V]) EagerSeq2[K, V]

Reverse2 will reverse the iteration order.

WARNING, it is not possible to reverse an iteration without first fully consuming it.

Example 
var i iter.Seq2[string, int]
i = iterkit.Reverse2(i)

type EagerSeqE added in v0.300.0 

type EagerSeqE[T any] = SeqE[T]

EagerSeqE is an iter.Seq2[T, error] that eagerly loads its content, thus not safe to use with infinite streams.

EagerSeqE iterators load all data upfront, meaning they cannot be used with infinite or unbounded sequences. This type is intended for scenarios where the entire sequence must be loaded into memory at once, which is impractical for infinite streams. Using EagerSeqE with such streams may lead to memory exhaustion or unexpected behavior.

If an iterator Sequence is eager, it should either have comments for functions or methods that return eager iterators or use the EagerSeqE to clearly express it with a return type.

func ReverseE added in v0.300.0 

func ReverseE[T any](i SeqE[T]) EagerSeqE[T]

ReverseE will reverse the iteration order.

WARNING, it is not possible to reverse an iteration without first fully consuming it.

Example 
var i iter.Seq2[string, error]
i = iterkit.ReverseE(i)

type ErrIter deprecated 

type ErrIter[T any] = SeqE[T]

ErrIter is a temporal alias to ErrSeq for backward compability purposes.

Deprecated: use iterkit.SeqE[T] instead.

type ErrSeq deprecated added in v0.295.0 

type ErrSeq[T any] = SeqE[T]

ErrSeq is a temporal alias to SeqE for backward compability purposes.

Deprecated: use iterkit.SeqE[T] instead.

type KV 

type KV[K, V any] struct {
	K K
	V V
}

func Collect2KV added in v0.297.0 

func Collect2KV[K, V any](i iter.Seq2[K, V]) []KV[K, V]
Example 
var itr iter.Seq2[string, int]

ints := iterkit.Collect2KV(itr)
_ = ints

type NoMore added in v0.295.0 

type NoMore struct{}

func (NoMore) Error added in v0.297.0 

func (NoMore) Error() string

type PullIter 

type PullIter[V any] interface {
	// Next will ensure that Value returns the next item when executed.
	// If the next value is not retrievable, Next should return false and ensure Err() will return the error cause.
	Next() bool
	// Value returns the current value in the iterator.
	// The action should be repeatable without side effects.
	Value() V
	// Closer is required to make it able to cancel iterators where resources are being used behind the scene
	// for all other cases where the underling io is handled on a higher level, it should simply return nil
	io.Closer
	// Err return the error cause.
	Err() error
}

PullIter define a separate object that encapsulates accessing and traversing an aggregate object. Clients use an iterator to access and traverse an aggregate without knowing its representation (data structures). Interface design inspirited by https://golang.org/pkg/encoding/json/#Decoder https://en.wikipedia.org/wiki/Iterator_pattern

func ToPullIter 

func ToPullIter[T any](itr SeqE[T]) PullIter[T]

type SeqE added in v0.297.0 

type SeqE[T any] = iter.Seq2[T, error]

SeqE is an iterator sequence that represents the iteration of external resources, which may potentially fail. The name draws inspiration from the standard library's `Seq2`, but with a key distinction: the suffix "E" highlights the possibility of errors occurring during iteration.

Examples of such resources include gRPC streams, HTTP streams, and database query result iterations.

func BatchE added in v0.297.0 

func BatchE[T any](i SeqE[T], opts ...BatchOption) SeqE[[]T]
Example (WithSize) 
src := iterkit.IntRangeE(0, 1000)

batched := iterkit.BatchE(src, iterkit.BatchSize(100))

for vs := range batched {
	fmt.Printf("%#v\n", vs)
}
Example (WithWaitLimit) 
slowIterSeq := iterkit.IntRangeE(0, 1000)

batched := iterkit.BatchE(slowIterSeq, iterkit.BatchWaitLimit(time.Second))

// Batching will occure either when the batching size reached
// or when the wait limit duration passed
for vs := range batched {
	fmt.Printf("%#v\n", vs)
}

func ChanE added in v0.297.0 

func ChanE[T any](ch <-chan T) SeqE[T]

ChanE creates an iterator out from a channel

Example 
ch := make(chan int)

i := iterkit.ChanE(ch)

go func() {
	defer close(ch)
	ch <- 42
}()

for v := range i {
	fmt.Println(v) // 42 once
}

func CharRangeE added in v0.297.0 

func CharRangeE(begin, end rune) SeqE[rune]

CharRange1 returns an iterator that will range between the specified `begin“ and the `end` rune.

Example 
for char, err := range iterkit.CharRangeE('A', 'Z') {
	_ = err // nil
	// prints characters between A and Z
	// A, B, C, D... Z
	fmt.Println(string(char))
}

func EmptyE added in v0.297.0 

func EmptyE[T any]() SeqE[T]

EmptyE iterator is used to represent nil result with Null object pattern

Example 
_ = iterkit.EmptyE[any]()

func Error 

func Error[T any](err error) SeqE[T]

Error returns an Interface that only can do is returning an Err and never have next element

func ErrorF 

func ErrorF[T any](format string, a ...any) SeqE[T]

ErrorF behaves exactly like fmt.ErrorF but returns the error wrapped as iterator

func From added in v0.297.0 

func From[T any](fn func(yield func(T) bool) error) SeqE[T]

From creates a ErrSeq with a function that feels similar than creating an iter.Seq.

Example 
src := iterkit.From(func(yield func(int) bool) error {
	for v := range 42 {
		if !yield(v) {
			return nil
		}
	}
	return nil
})

_ = src

func FromPages added in v0.295.0 

func FromPages[T any](next func(offset int) (values []T, _ error)) SeqE[T]

FromPages will create an iter.Seq[T] which can be used like any other iterator, Under the hood the "more" function will be used to dynamically retrieve more values when the previously called values are already used up.

If the more function has a hard-coded true for the "has next page" return value, then the pagination will interpret an empty result as "no more pages left".

Example 
ctx := context.Background()

fetchMoreFoo := func(offset int) ([]Foo, error) {
	const limit = 10
	query := url.Values{}
	query.Set("limit", strconv.Itoa(limit))
	query.Set("offset", strconv.Itoa(offset))
	resp, err := http.Get("https://api.mydomain.com/v1/foos?" + query.Encode())
	if err != nil {
		return nil, err
	}

	var values []FooDTO
	defer resp.Body.Close()
	dec := json.NewDecoder(resp.Body)
	dec.DisallowUnknownFields()
	if err := dec.Decode(&values); err != nil {
		return nil, err
	}

	vs, err := dtokit.Map[[]Foo](ctx, values)
	if err != nil {
		return nil, err
	}
	if len(vs) < limit {
		return vs, iterkit.NoMore{}
	}
	return vs, nil
}

foos := iterkit.FromPages(fetchMoreFoo)
_ = foos // foos can be called like any iterator,
// and under the hood, the fetchMoreFoo function will be used dynamically,
// to retrieve more values when the previously called values are already used up.

func FromPullE added in v0.297.0 

func FromPullE[T any](next func() (T, error, bool), stops ...func()) SeqE[T]

func FromPullIter 

func FromPullIter[T any](itr PullIter[T]) SeqE[T]

func HeadE added in v0.297.0 

func HeadE[T any](i SeqE[T], n int) SeqE[T]

Head1 takes the first n element, similarly how the coreutils "head" app works.

Example 
inf42 := func(yield func(int, error) bool) {
	for /* infinite */ {
		if !yield(42, nil) {
			return
		}
	}
}

i := iterkit.HeadE(inf42, 3)
vs, err := iterkit.CollectE(i)
_ = err // nil
_ = vs  // []{42, 42, 42}, nil

func IntRangeE added in v0.297.0 

func IntRangeE(begin, end int) SeqE[int]

IntRangeE returns an iterator that will range between the specified `begin“ and the `end` int.

Example 
for n, _ := range iterkit.IntRangeE(1, 9) {
	// prints characters between 1 and 9
	// 1, 2, 3, 4, 5, 6, 7, 8, 9
	fmt.Println(n)
}

func LimitE added in v0.297.0 

func LimitE[T any](i SeqE[T], n int) SeqE[T]

func MapE added in v0.297.0 

func MapE[To any, From any, Iter i1[From]](i Iter, transform func(From) (To, error)) SeqE[To]
Example 
rawNumbers := iterkit.Slice1([]string{"1", "2", "42"})
numbers := iterkit.MapE[int](rawNumbers, strconv.Atoi)
_ = numbers

func MergeE added in v0.297.0 

func MergeE[T any](is ...SeqE[T]) SeqE[T]

func OfE added in v0.297.0 

func OfE[T any](v T) SeqE[T]

OfE creates an SeqE iterator that can return the value of v.

func OffsetE added in v0.297.0 

func OffsetE[T any](i SeqE[T], offset int) SeqE[T]
Example 
_ = iterkit.Offset(iterkit.IntRange(2, 6), 2)

func OnErrIterValue deprecated 

func OnErrIterValue[To any, From any](itr SeqE[From], pipeline func(itr iter.Seq[From]) iter.Seq[To]) SeqE[To]

OnErrIterValue is a temporal alias to OnErrSeqValue

Deprecated: use iterkit.OnErrSeqValue instead

func OnErrSeqValue deprecated added in v0.295.0 

func OnErrSeqValue[To any, From any](itr SeqE[From], pipeline func(itr iter.Seq[From]) iter.Seq[To]) SeqE[To]

OnErrSeqValue is a temporal alias.

Deprecated: use iterkit.OnSeqEValue instead

func OnSeqEValue added in v0.297.0 

func OnSeqEValue[To any, From any](itr SeqE[From], pipeline func(itr iter.Seq[From]) iter.Seq[To]) SeqE[To]

OnSeqEValue will apply a iterator pipeline on a given ErrSeq

Example 
var (
	input  iter.Seq2[int, error]
	output iter.Seq2[string, error]
)

output = iterkit.OnSeqEValue(input, func(itr iter.Seq[int]) iter.Seq[string] {
	// we receive an iterator without the error second value
	// we do our iterator manipulation like it doesn't have an error
	// then we return it back
	itr = iterkit.Map(itr, func(v int) int { return v * 3 })
	itr = iterkit.Filter(itr, func(i int) bool { return i%2 == 0 })
	return iterkit.Map(itr, strconv.Itoa)
})

// the returned iter have the pipeline applied,
// but the elements still contain the potential error value in case something went wrong.
_ = output

func SliceE added in v0.297.0 

func SliceE[T any](vs []T) SeqE[T]
Example 
var vs = []int{1, 2, 3}
var i = iterkit.SliceE(vs)

for v, err := range i {
	_, _ = v, err
}

func ToSeqE added in v0.297.0 

func ToSeqE[T any](i iter.Seq[T]) SeqE[T]

ToSeqE will turn a iter.Seq[T] into an iter.Seq2[T, error] iterator, and use the error function to yield potential issues with the iteration.

Example 
seq1Iter := iterkit.Slice1([]int{1, 2, 3})
errIter := iterkit.ToSeqE(seq1Iter)
for v, err := range errIter {
	if err != nil {
		// will be always nil for the []int slice
	}
	_ = v // 1, 2, 3...
}

type SingleUseErrSeq deprecated added in v0.295.0 

type SingleUseErrSeq[T any] = SingleUseSeqE[T]

ErrSeq is a temporal alias to SeqE for backward compability purposes.

Deprecated: use iterkit.SingleUseSeqE[T] instead.

type SingleUseSeq added in v0.287.0 

type SingleUseSeq[T any] = iter.Seq[T]

SingleUseSeq is an iter.Seq[T] that can only iterated once. After iteration, it is expected to yield no more values.

Most iterators provide the ability to walk an entire sequence: when called, the iterator does any setup necessary to start the sequence, then calls yield on successive elements of the sequence, and then cleans up before returning. Calling the iterator again walks the sequence again.

SingleUseSeq iterators break that convention, providing the ability to walk a sequence only once. These “single-use iterators” typically report values from a data stream that cannot be rewound to start over. Calling the iterator again after stopping early may continue the stream, but calling it again after the sequence is finished will yield no values at all.

If an iterator Sequence is single use, it should either has comments for functions or methods that it return single-use iterators or it should use the SingleUseSeq to clearly express it with a return type.

func Once 

func Once[T any](i iter.Seq[T]) SingleUseSeq[T]

func Sync added in v0.286.0 

func Sync[T any](i iter.Seq[T]) (SingleUseSeq[T], func())

Sync ensures that an iterator can be safely used by multiple goroutines at the same time.

Example 
src := iterkit.IntRange(0, 100)
itr, cancel := iterkit.Sync(src)
defer cancel()

var g tasker.JobGroup[tasker.FireAndForget]
for range 2 {
	g.Go(func(ctx context.Context) error {
		for v := range itr {
			_ = v // use v
		}
		return nil
	})
}

g.Join()

type SingleUseSeq2 added in v0.287.0 

type SingleUseSeq2[K, V any] = iter.Seq2[K, V]

SingleUseSeq2 is an iter.Seq2[K, V] that can only iterated once. After iteration, it is expected to yield no more values. For more information on single use sequences, please read the documentation of SingleUseSeq.

func Once2 

func Once2[K, V any](i iter.Seq2[K, V]) SingleUseSeq2[K, V]

func Sync2 added in v0.286.0 

func Sync2[K, V any](i iter.Seq2[K, V]) (SingleUseSeq2[K, V], func())

Sync2 ensures that an iterator can be safely used by multiple goroutines at the same time.

Example 
src := iterkit.IntRange(0, 100)
itr, cancel := iterkit.Sync2(iterkit.ToSeqE(src))
defer cancel()

var g tasker.JobGroup[tasker.FireAndForget]
for range 2 {
	g.Go(func(ctx context.Context) error {
		for v, err := range itr {
			_ = err // handle err
			_ = v   // use v
		}
		return nil
	})
}

g.Join()

type SingleUseSeqE added in v0.297.0 

type SingleUseSeqE[T any] = SeqE[T]

SingleUseSeqE is an iter.Seq2[T, error] that can only iterated once. After iteration, it is expected to yield no more values. For more information on single use sequences, please read the documentation of SingleUseSeq.

func BufioScanner 

func BufioScanner[T string | []byte](s *bufio.Scanner, closer io.Closer) SingleUseSeqE[T]

func OnceE added in v0.297.0 

func OnceE[T any](i SeqE[T]) SingleUseSeqE[T]

func SyncE added in v0.297.0 

func SyncE[T any](i SeqE[T]) (SingleUseSeqE[T], func())

SyncE ensures that an iterator can be safely used by multiple goroutines at the same time.

Example 
src := iterkit.IntRangeE(0, 100)
itr, cancel := iterkit.SyncE(src)
defer cancel()

var g tasker.JobGroup[tasker.FireAndForget]
for range 2 {
	g.Go(func(ctx context.Context) error {
		for v := range itr {
			_ = v // use v
		}
		return nil
	})
}

g.Join()

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.