kv

Documentation

Overview

Package kv contains Pair for holding <K,V> pairs as byte slices, and Codec for transforming <K,V> pairs into Pair and vice versa.

Index

• type Codec
  • func NewCodec(kType, vType reflect.Type) *Codec
  • func (codec *Codec) Decode(p Pair) (k, v any, err error)
  • func (codec *Codec) Encode(k, v any) (Pair, error)
  • func (codec *Codec) Setup() error
• type DecodeFn
  • func NewDecodeFn(kT, vT typex.FullType) *DecodeFn
  • func (fn *DecodeFn) ProcessElement(c Pair) (beam.T, beam.V)
  • func (fn *DecodeFn) Setup()
• type EncodeFn
  • func NewEncodeFn(kT, vT typex.FullType) *EncodeFn
  • func (fn *EncodeFn) ProcessElement(k beam.T, v beam.V) Pair
  • func (fn *EncodeFn) Setup()
• type Pair

Types

type Codec

type Codec struct {
	KType beam.EncodedType
	VType beam.EncodedType
	// contains filtered or unexported fields
}

Codec provides functions for encoding a <K,V> pair into a Pair and decoding a Pair into a <K,V> pair. It can be used for performing <K,V> transforms. Whenever a Codec is used inside a structural doFn, it should be an exported field in order to be serialized.

After initialization, Setup function should be called before any calls are made to Encode/Decode.

func NewCodec

func NewCodec(kType, vType reflect.Type) *Codec

NewCodec returns a new Codec with specified <K,V> type.

func (*Codec) Decode

func (codec *Codec) Decode(p Pair) (k, v any, err error)

Decode transforms a Pair into a <K,V> pair.

func (*Codec) Encode

func (codec *Codec) Encode(k, v any) (Pair, error)

Encode transforms a <K,V> pair into a Pair.

func (*Codec) Setup

func (codec *Codec) Setup() error

Setup initializes the encoders and decoders. This functions needs to be called once before any calls are made to Encode/Decode.

type DecodeFn

type DecodeFn struct {
	KType beam.EncodedType
	VType beam.EncodedType
	// contains filtered or unexported fields
}

DecodeFn transforms a PCollection<codedKV> into a PCollection<K,V>.

func NewDecodeFn

func NewDecodeFn(kT, vT typex.FullType) *DecodeFn

NewDecodeFn returns a DecodeFn from given types.

func (*DecodeFn) ProcessElement

func (fn *DecodeFn) ProcessElement(c Pair) (beam.T, beam.V)

ProcessElement decodes a Pair into a <K,V>.

func (*DecodeFn) Setup

func (fn *DecodeFn) Setup()

Setup initializes the decoders.

type EncodeFn

type EncodeFn struct {
	KType beam.EncodedType
	VType beam.EncodedType
	// contains filtered or unexported fields
}

EncodeFn transforms a PCollection<K,V> into a PCollection<Pair>.

func NewEncodeFn

func NewEncodeFn(kT, vT typex.FullType) *EncodeFn

NewEncodeFn returns an EncodeFn from given types.

func (*EncodeFn) ProcessElement

func (fn *EncodeFn) ProcessElement(k beam.T, v beam.V) Pair

ProcessElement encodes a <K,V> as a Pair.

func (*EncodeFn) Setup

func (fn *EncodeFn) Setup()

Setup initializes the encoders.

type Pair

type Pair struct {
	K []byte
	V []byte
}

Pair contains a KV<K,V> pair, both values encoded as []byte.