Help us shape the future of Go! Take the 2026 Go Developer Survey

stream

package module
v1.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 5, 2022 License: GPL-3.0 Imports: 4 Imported by: 9

README

stream

Install

go get github.com/wushilin/stream

Documentation

$ godoc -http=":16666"

Browse http://localhost:16666

Usage

Stream API

What is a stream? Stream is a concept of collection of items, from various sources.

Creating Stream

There are couple of ways of creating streams. Please read throught code below.

import "github.com/wushilin/stream"

// create stream from static list
s := stream.Of("1","2","3")

// Create stream from slice
s := stream.FromArray([]string{"1","2","3"})

// Create stream from a file
s := stream.FromFileLines("/tmp/input.txt")
// Note that s.Close() must be called for file streams - it will close 
// the file handle

// Create stream from a receiver channel
c := make(chan int)
s := stream.FromChannel(c)
// Note here the other go routines must send to the channel and close the channel

// Create stream from a generator function
s := stream.Generate(func() int {
  return 5
})
// This is pretty much useless, it returns a infinite stream of integer 5.
// But you can combine it with other features like Limit, Skip, and Sum

s := stream.Generate(func() int {
  return rand.Intn(12)
})
// This will create a infinite stream of random integers betwee 0 and 12

// Create stream from iterative function
s := stream.Iterate(1, func(i int) int {
  return i + 2
})
// This will generate a stream of even integers, starting from 1.

seed := 1
fibonaci_func := func(i int) int {
    result := i + seed
    seed = i
    return result
}
stream.Iterate(1, fibonaci).Limit(30)
// This creates a stream of Fibonacci numbers, limited to first 30 items

// Create stream from a range
s := stream.Range(1, 100)
fmt.Println(s.Sum())
// You will get 4950, true <- true means there is a sum for retrieval

// Create stream from Iterator
// dummy type implements Iterator, which has a Next() function 
// Next() function returns the next value and whether the value exists.
type dummy struct {
}

func (v dummy) Next() (int, bool) {
  return 5, true
}

s := stream.FromIterator(dummy{}) // this is same as static generator function

// Create stream from Map keys and values
mp := make(map[string]string)
mp["1"] = "Jessy"
mp["2"] = "Steve"
sk := stream.FromMapKeys(mp)
sv := stream.FromMapValues(mp)
se := stream.FromMapEntries(mp)
Stream operators
Concat - consumes first stream first, when exhausted, consume the second
s1 := stream.Range(0,5)
s2 := stream.Range(5,10)
s3 := s1.Concat(s2) 
// s3 is the same as stream.Range(0,10)
// this applies for all kinds of streams, not only ranges
Map - Map a transformation to existing stream
// Due to go type system, Stream[T1].Map(func (T1) T2) => Stream[T2] won't work. We have use a static function like this unfortunately.
s1 := stream.Range(0, 5) // this stream, when consumed, will produce 0,1,2,3,4
s2 := stream.Map(s1, func(i int) int {
  return i + 5
})
// s2, when consumed, will consume s1, and produce 5,6,7,8,9
s3 := s2.Map(func(i int) string {
  return fmt.Sprintf("Student number %d", i)
}
// s3, when consumed, will consume s2 and produce 
// "Student number 5", "Student number 6",...,"Student number 9"
Reduce - reduce a stream to a single object

The result is a stream.Optional object. The optional object has a Value() function to retrieve its value

s1 := stream.Of(1,2,3,4,5)
sum := 0
reduce_func := func(i,j int) int { return i + j }
fmt.Println(s1.Reduce(reduce_func).Value())
// This will produce 15, true (15 is the result, true means there is a result)
// stream.Range(5, -1).Reduce(reduce_func).Value() will produce nil, false
// since stream.Range(5, -1) contains no elements.
Limit - limit the number of elements to emit.

The remaining items of upper stream is not consumed

s1 := stream.Generate(func() int { return 5 }) // s1 is infinite number of 5s
s2 := s1.Limit(20)
fmt.Println(s2.Sum().Value()) // will produce 100, true
Count - consume all elements and return the count
fmt.Println(stream.Of("1","2").Count()) // produces 2
fmt.Println(stream.Range(0,100).Count()) // produces 100
fmt.Println(stream.Generate(func() int{ return 5 })) // never returns, infinite loop
Filter - filter elements based on predict
s1 := stream.FromFileLines("/tmp/test.txt")
defer s1.Close()
s2 := s1.Filter(func(line string) bool {
  return len(line) < 100
})
// s2 is stream of strings, where the lines are shorter than 100 characters
Each - do a func for each of elements, doesn't expect return result
stream.Range(0, 5).Each(func(i int) {
    fmt.Println(i, "* 2 =", i*2)
})
// Produces
// 0 * 2 = 0
// 1 * 2 = 2
// ...
// 4 * 2 = 8
Close - close the stream, and trigger OnClose function

Streams by default, has no close handler. so Close() does nothing. The only one has a default close handler is the stream.FromFileLines("filename") When it is closed, the file gets closed. You can attach a close handler, so the handler is called when stream is closed.

s1 := stream.Range(0, 100).OnClose(func() {
  fmt.Println("I am closed, good job man, I will call some other functions")
}).OnClose(func() {
  fmt.Println("I am closing some database connection")
})
defer s1.Close()
// Then you will see the two OnClose functions are called in order they are attached. (earlier OnClose func got called first).

Iterator - return iterator of the stream

streams are implemented using iterators. So it simply returns the underlying iterator

iter := stream.Range(0,100).Iterator()
for val, has_next := iter.Next(); val, has_next = iter.Next(); has_next {
  // do something
}
Skip - skip N elements
s1 := stream.Range(0, 100).Skip(50) // 50, 51, 52,...., 99
CollectTo - collect elements to slice, up to stream's availability and buffer capacity
buffer := make([]int, 100)
collect_count := stream.Range(0, 100000).CollectTo(buffer)
// collect_count will be 100, and buffer will contain 0~99

collect_count1 := stream.Range(0,1).CollectTo(buffer)
// collect_count1 will be 1, and buffer will contain 0. Rest of values untouched
MaxCmp - Do reduce use the less comparator to find max value
max_cmp_func := func(i, j int) bool {
  return i < j
}
fmt.Println(stream.Range(0, 100).MaxCmp(max_cmp_func)) // 99, true
MinCmp - Do reduce using less comparator to find min value
max_cmp_func := func(i, j int) bool {
  return i < j
}
fmt.Println(stream.Range(0, 100).MinCmp(max_cmp_func)) // 99, true
Max - do natural comparison max value. Builtin support for all numbers and strings
fmt.Println(stream.Range(0, 100).MaxCmp(func(i, j int) int {
	if i < j {
		return -1
	}
  if i == j {
  	return 0
  }
  return 1
}).Value()) // 99, true
Min - do natural comparison min value. Builtin support for all numbers and strings
fmt.Println(stream.Of(1.1, 1.2, 1.3).MinCmp(func(j, j int) int {
  if i < j {
    return -1
  }
  if i == j {
    return 0
  }
  return 1
}).Value()) // 1.1, true
Peek - return a stream that contains same element, but attach a trigger on consumption
count := 0
fmt.Println(stream.Range(0, 5).Peek(func(i int) {
  count = count + i
}).Count())
fmt.Println(count)
// Produces 5 (the count), 10 (the sum)
WithIndex - index the number when consuming
stream.WithIndex(stream.Range(6, 100)).Each(func(i stream.NumberedItem) {
  fmt.Println("Object index is:", i.Index)
  fmt.Println("Object is:", i.Item)
})
SendTo - send stream to a channel
destc := make(chan int)
done := make(chan bool)
go func() {
    for next_int := range(destc) {
      fmt.Println("Received:", next_int)
    }
    done <- true
}()

stream.Range(0, 100).SendTo(destc)
close(destc)
<- done
// This will send 0~99 (100 numbers) to the channel for another go routine to consume.
Pack2, Pack3, Pack4, Pack5 Package stream items in pairs, triples, quadruples, and quintuples
stream.Pack2(stream.Of(1,2,3,4,5)) => [1,2], [3,4]
stream.Pack3(stream.Of(1,2,3,4,5,6)) => [1,2,3], [4,5,6]
stream.Pack4(stream.Of(1,2,3,4,5,6)) => [1,2,3,4]
Flatten, expand elements in place
	s13 := Of("1,2", "3,4,5", "", "6,7,8,9", "10")
	s14 := Flatten(s13, func(i string) []int {
		tokens := strings.Split(i, ",")
		result := []int{}
		for _, str := range tokens {
			if str == "" {
				continue
			}
			num, _ := strconv.Atoi(str)
			result = append(result, num)
		}
		return result
	})
        //s14-> [1,2,3,4,5,6,7,8,9,10]

Documentation

Overview

Define stream just like java

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CollectAll

func CollectAll[T any](iter Iterator[T]) []T

Same as CollecctN, but no limit

func CollectN

func CollectN[T any](iter Iterator[T], count int) []T

Collect at most count elements from Iterator object. Returns a slice of object collected. If count is 0, empty slice is returned. If count is less than 0, Maximum number of objects are collected (similar to Integer.MAX_VALUE in java)

Note: Collected objects might be less than count if Iterator reached end
Note: Not thread safe

Types

type ChannelIterator

type ChannelIterator[T any] struct {
	// contains filtered or unexported fields
}

Utility struct to for channel iteration Iterate through channels requires explicit channel closure. Failing to do so will result indefinite waiting.

Note: thread safe

func NewChannelIterator

func NewChannelIterator[T any](target chan T) *ChannelIterator[T]

Create Iterator on a channel. Target must be a channel

Note: Channel sender need to close channel or Iterator's last Next() call will hang forever
Multiple Iterator on same channel is possible since each Next() call will receive from the channel

func (*ChannelIterator[T]) Next

func (v *ChannelIterator[T]) Next() (result T, ok bool)

Implements Iterator. result will be channel's corresponding value type, requires casting

Note: thread safe

type Comparator added in v1.1.0

type Comparator[T any] func(arg1, arg2 T) int

Comparator returns -1 if arg1 < arg2, returns 0 if arg1 == arg2 and returns 1 if arg1 > arg2

type GenFunc

type GenFunc[T any] func() T

GenFunc is a function generate values, It is also a ProducerFunc

type Iterator

type Iterator[T any] interface {
	/*
		Next() returns the next element. If an element is available, val is the element and ok is true.
		If an element is not available, val is zero value of the type, and ok is set as false

		When OK is false, the value, if returned, should be ignored.

		A common way of iterating:
		for next, ok := iter.Next(); ok; next, ok = iter.Next() {
		   do_some_thing_with(next)
		}
	*/
	Next() (result T, ok bool)
}

Defined a iterator for generic purpose

func NewArrayIterator

func NewArrayIterator[T any](target []T) Iterator[T]

Return a new iterator based on array. target must be an array, or a slice, or a pointer to an array, or a pointer to a slice It can't be pointer to pointer to an array/slice Note: Not thread safe

type MapEntry

type MapEntry[K, V any] struct {
	Key   K
	Value V
}

Map entry iteration: Just like java.util.Map.Entry class

type MapEntryIterator

type MapEntryIterator[K comparable, V any] struct {
	// contains filtered or unexported fields
}

Utility struct for Map Entry iteration. All map keys are obtained when iterator is constructed. Thus no new key's value added after construction will be returned. However, values modified after iteration construction, new key/value will be returned

Note: Not thread safe

func NewMapEntryIterator

func NewMapEntryIterator[K comparable, V any](target map[K]V) *MapEntryIterator[K, V]

Create Iterator to iterate over a map's entries Target must be a map Multiple Iterator on same map is possible.

func (*MapEntryIterator[K, V]) Next

func (v *MapEntryIterator[K, V]) Next() (result MapEntry[K, V], ok bool)

Implements Iterator. Will return MapEntry on each call.

Note: Not thread safe

type MapKeyIterator

type MapKeyIterator[K comparable, V any] struct {
	// contains filtered or unexported fields
}

Utility struct for map key iteration

Note: Not thread safe

func NewMapKeyIterator

func NewMapKeyIterator[K comparable, V any](target map[K]V) *MapKeyIterator[K, V]

Create Iterator to iterate over a map's keys. Target must be a map or code may panic Multiple Iterator on same map is possible

func (*MapKeyIterator[K, V]) Next

func (v *MapKeyIterator[K, V]) Next() (result K, ok bool)

Implements Iterator. Returns a map key on each Next() call All map keys are obtained when iterator is constructed. Thus no new key added after construction will be returned.

Note: Not thread safe

type MapValueIterator

type MapValueIterator[K comparable, V any] struct {
	// contains filtered or unexported fields
}

Utility struct for map value iteration

Note: Not thread safe

func NewMapValueIterator

func NewMapValueIterator[K comparable, V any](target map[K]V) *MapValueIterator[K, V]

Create Iterator to iterate over a map's values. Target must be a map Multiple Iterator on same map is possible

func (*MapValueIterator[K, V]) Next

func (v *MapValueIterator[K, V]) Next() (result V, ok bool)

Implements Iterator. Returns a map value on each Next() call All map keys are obtained when iterator is constructed. Thus no new key's value added after construction will be returned. However, values modified after iteration construction, new value will be returned If a value is removed after the iterator was created, the nil value for that key will be returned when iterating

Note: Not thread safe

type Number added in v1.1.0

type Number interface {
	~int8 | ~int16 | ~int32 | ~int64 | ~uint8 | ~uint16 | ~uint32 | ~uint64 | ~string | ~int | ~float32 | ~float64
}

type NumberedItem

type NumberedItem[T any] struct {
	Index int64
	Item  T
}

type Optional

type Optional[T any] interface {
	// Return the value and if value is present
	// Optional(1).Value() -> 1, true
	// Optional(None).Value() -> <zero value>, false
	// Caller need to check value exist before the value can be trusted
	Value() (T, bool)

	// Return another optional value, when call Value() on the new optional,
	// return this object's value if present
	// If not, return other's Value()
	// Optional(1).Or(Optional(2)).Value() -> 1, true
	// Optional(None).Or(Optional(2)).Value() -> 2, true
	// Optional(None).Or(Optional(None)).Value() -> 0, false
	Or(other Optional[T]) Optional[T]

	// Return this object's value if present, or the defaultValue supplied
	// Optional(1).OrValue(2) -> 1
	// Optional(None).OrValue(2) -> 2
	OrValue(defaultV T) T
}

Optional value may or may not have a value

func NewEmptyOptional

func NewEmptyOptional[T any]() Optional[T]

func NewOptional

func NewOptional[T any](val T) Optional[T]

Return an optional has a value call to result.Value() will return val, true

func Sum added in v1.1.0

func Sum[T Number](v Stream[T]) Optional[T]

type Pair added in v1.1.1

type Pair[T1, T2 any] interface {
	First() T1
	Second() T2
	Value() (T1, T2)
}

A pair holds 2 values, potentially with same or different types you can get the value both with Value() call, or get First() and Second() separately

func NewPair added in v1.1.1

func NewPair[T1, T2 any](first T1, second T2) Pair[T1, T2]

Create a new Pair with 2 values

type Quadruple added in v1.1.1

type Quadruple[T1, T2, T3, T4 any] interface {
	First() T1
	Second() T2
	Third() T3
	Forth() T4
	Value() (T1, T2, T3, T4)
}

Same as pair, but 4 values

func NewQuadruple added in v1.1.1

func NewQuadruple[T1, T2, T3, T4 any](first T1, second T2, third T3, forth T4) Quadruple[T1, T2, T3, T4]

Create a new Quodruple with 4 values

type Quintuple added in v1.1.1

type Quintuple[T1, T2, T3, T4, T5 any] interface {
	First() T1
	Second() T2
	Third() T3
	Forth() T4
	Fifth() T5
	Value() (T1, T2, T3, T4, T5)
}

Same as pair, but 5 values

func NewQuintuple added in v1.1.1

func NewQuintuple[T1, T2, T3, T4, T5 any](first T1, second T2, third T3, forth T4, fifth T5) Quintuple[T1, T2, T3, T4, T5]

Create a new Quintuple with 5 values

type Stream

type Stream[T any] interface {
	// Due to Golang generic limitation, the map only takes interface{} as result
	// If you want to have the type parameterized, use stream.Map(src Stream[F], func(F) T) Stream[T]
	// For each element of the stream, apply a map function, and return the
	// mapped result stream. It is lazy so only operated when you use a terminal operator
	// e.g.
	//   stream.Range(0, 10).Map(func(a interface{}) interface{} {
	//     return a.(int) + 1
	//   }
	// will return a lazy stream, from 1~10 (not range(0,10) is 0~9)
	Map(f func(in T) interface{}) Stream[interface{}]

	// Use a reduce function to reduce elements in the stream and return reduced resumt.
	// e.g.
	//   stream.Range(0, 10).Reduce(func(a, b interface{}) interface{} {
	//     if a.(int) > b.(int) {
	//       return a
	//     }
	//     return b
	//   }
	// will return an optional, value is 9. This essentially reduce using Max(a,b) function
	// This is a terminal operator
	Reduce(f func(arg1, arg2 T) T) Optional[T]

	// Lazily limit the elements to process to number
	Limit(number int) Stream[T]

	// Count elements in stream.
	// This is terminal operator
	Count() int

	// Similar to Map, but skip if the predict func did not pass the test.
	// Return a lazy stream
	// e.g.
	//   stream.Range(0, 10).Filer(func(a interface{}) bool) {
	//      return a.(int) > 5
	//   }
	// will return a stream from [6 ~ 9]
	Filter(f func(arg T) bool) Stream[T]

	// For each of the stream, do the function
	// e.g.
	// var sum := 0
	//   stream.Of(1, 2, 3).Each(func(i interface{}) {
	//     sum = sum + i.(int)
	//   })
	// will return 6
	//   Note this is a terminal operator
	Each(f func(T))

	// Same as Each, but depend on the bool value to continue
	// returns true => move to next
	// returns false => aborts
	EachCondition(func(T) bool)

	// Close the stream. If your stream if from files, you have to close it.
	// Any OnClose handler previously attached will be called
	// If your stream is a result of operation on upstream streams, closing it
	// will also close upstream stream
	// e.g.
	// s1 = stream.FromFileLines("test.txt")
	// s2 = s1.Limit(5)
	// s2.Close() => s1 will be closed too!
	Close()

	// Attach another close handler to the stream.
	// Previous handlers are still there
	// Upon closing,
	// This handler will be called after the previous handler had been called
	// If you attach multiple handlers of same instance, it will be called multiple times
	OnClose(closefunc func()) Stream[T]

	// Return an iterator of elements from the stream
	Iterator() Iterator[T]

	// Concat the other stream to end of current stream and return the new stream.
	// Both original stream and other stream itself are not modified.
	// The new stream's close handler will call this stream's close handler,
	// then call the other stream's close handler
	Concat(other Stream[T]) Stream[T]

	// Collect elements to target. Target must be array/slice of same type as the elements.
	// The max number if elements to collect is len(target).
	// Returns the number of elements collected
	// When number of elements collected equal to slice/array cap, there might be more elemnts to be collected
	// otherwise, it is guaranteed no more elements left in stream
	CollectTo(target []T) int

	// Get max in the stream using a less comparator function
	MaxCmp(cmp Comparator[T]) Optional[T]

	// Get min in the stream using a less comparator function
	MinCmp(cmp Comparator[T]) Optional[T]

	// Skip N elements in the stream, returning a new stream
	Skip(number int) Stream[T]

	// Return a new stream of itself, but when elements are consumed, the peek function is called
	// e.g.
	//   sum := 0
	//   fmt.Println(stream.Range(0, 10).Peek(func(i interface) {
	//     sum = sum + i.(int)
	//   }).Count())
	//   fmt.Println("Sum is", sum)
	// This will count the elements, as well as add a sum
	Peek(f func(T)) Stream[T]

	// Stream all elements to channel. May block the caller if
	// the channel is full
	SendTo(chan T)
}

Stream defines java like stream magics

func Flatten added in v1.1.1

func Flatten[T1, T2 any](v Stream[T1], expander func(i T1) []T2) Stream[T2]

Flattern a stream with expander [1,2], [3], [4, 5],[] -> Flatten() -> [1,2,3,4,5]

func FromArray

func FromArray[T any](it []T) Stream[T]

Return stream from array. It is safe to call multiple FromArray on same array The call doesn't modify source array

func FromChannel

func FromChannel[T any](it chan T) Stream[T]

Return stream from channel. If you use same channel create multiple streams all streams will share the channel and may see part of the data Sender of channel must close or stream's reduce function/map function may not terminate

func FromFileLines

func FromFileLines(filepath string) (Stream[string], error)

Return a stream from file lines. Caller must call close on return stream. If file open fails, the stream will be nil and error will be returned.

func FromIterator

func FromIterator[T any](it Iterator[T]) Stream[T]

Return stream from iterator. stream's reduce function will consume all items in iterator

func FromMapEntries

func FromMapEntries[K comparable, V any](it map[K]V) Stream[MapEntry[K, V]]

Create stream from map's key value pairs. Note the iterator is a snapshot of map subsequent modification after Stream is created won't be visiable to stream

func FromMapKeys

func FromMapKeys[K comparable, V any](it map[K]V) Stream[K]

Create stream from map's keys. Note the iterator is a snapshot of map subsequent modification after Stream is created won't be visiable to stream

func FromMapValues

func FromMapValues[K comparable, V any](it map[K]V) Stream[V]

Create stream from map's values. Note the iterator is a snapshot of map subsequent modification after Stream is created won't be visiable to stream

func FromReader

func FromReader(r io.Reader, delimeter rune) Stream[string]

Create new stream from a reader, with the delimiter for tokenizer

func Generate

func Generate[T any](f func() T) Stream[T]

Generate will use the GenFunc to generate a infinite stream e.g.

stream.Generate(func() interface{} {
  return 5
}

will generate a infinite stream of 5. Note it is lazy so do not count infinite stream, it will not complete Similarly, do not Reduce infinite stream You can limit first before reducing

func Iterate

func Iterate[T any](seed T, f func(T) T) Stream[T]

Generate a infinite stream, using seed as first element, then use MapFun and the previously returned value to generate the new value e.g.

func add1(i int) int {
  return i.(int) + 1
}
stream.Iterate(1, add1).Limit(3) <= will produce the same as
stream.Of(1, 2, 3)

func Map added in v1.1.0

func Map[F any, T any](src Stream[F], f func(in F) T) Stream[T]

For each of the stream element, use a mapping functiont to transform it

stream.Map(Stream.of("1", "2", "3"), func(i string) int {
    parse int here
}) => [1,2,3]

func Of

func Of[T any](vars ...T) Stream[T]

Create a stream from a set of values. e.g.

stream.Of(1,2,3,4,5).Sum().Value() => 15, true

15 is the result, true means the optional actually have a value

stream.Of().Sum().Value() => nil, false

nil is default result, false means the stream is empty, so sum is non-existent

func Pack2 added in v1.1.1

func Pack2[T any](v Stream[T]) Stream[Pair[T, T]]

Compacts the stream of pairs 1,2,3,4,5,6,7,8,9,10 => [1,2], [3,4], [5,6], [7, 8], [9, 10] 1,2,3,4,5 => [1,2], [3, 4]. If you need the last value, it is returned as [5, 0], false

func Pack3 added in v1.1.1

func Pack3[T any](v Stream[T]) Stream[Triple[T, T, T]]

func Pack4 added in v1.1.1

func Pack4[T any](v Stream[T]) Stream[Quadruple[T, T, T, T]]

func Pack5 added in v1.1.1

func Pack5[T any](v Stream[T]) Stream[Quintuple[T, T, T, T, T]]

func Range

func Range(low, high int) Stream[int]

Return Stream of range from low (inclusive) to high(exclusive). values are all int. Again, it is lazy, don't worry about the pre-allocation of memory. It is safe to do stream.Range(0, 10000000)

func WithIndex

func WithIndex[T any](v Stream[T]) Stream[NumberedItem[T]]

func WrapStream added in v1.1.1

func WrapStream[T1, T2 any](src Stream[T1], iter Iterator[T2]) Stream[T2]

type Triple added in v1.1.1

type Triple[T1, T2, T3 any] interface {
	First() T1
	Second() T2
	Third() T3
	Value() (T1, T2, T3)
}

Same as pair, but for 3 values

func NewTriple added in v1.1.1

func NewTriple[T1, T2, T3 any](first T1, second T2, third T3) Triple[T1, T2, T3]

Create a new triple with 3 values

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL