streams

Documentation

Overview

Package streams defines types used for the data object streams section. The streams section holds a list of streams present in the data object.

Index

• func Iter(ctx context.Context, dec encoding.Decoder) result.Seq[Stream]
• func IterSection(ctx context.Context, dec encoding.StreamsDecoder, section *filemd.SectionInfo) result.Seq[Stream]
• type Metrics
  • func NewMetrics() *Metrics
  • func (m *Metrics) Register(reg prometheus.Registerer) error
  • func (m *Metrics) Unregister(reg prometheus.Registerer)
• type Stream
  • func Decode(columns []*streamsmd.ColumnDesc, row dataset.Row) (Stream, error)
  • func (s *Stream) Reset()
• type Streams
  • func New(metrics *Metrics, pageSize int) *Streams
  • func (s *Streams) EncodeTo(enc *encoding.Encoder) error
  • func (s *Streams) EstimatedSize() int
  • func (s *Streams) Record(streamLabels labels.Labels, ts time.Time, recordSize int64) int64
  • func (s *Streams) Reset()
  • func (s *Streams) StreamID(streamLabels labels.Labels) int64
  • func (s *Streams) TimeRange() (time.Time, time.Time)

Functions

func Iter

func Iter(ctx context.Context, dec encoding.Decoder) result.Seq[Stream]

Iter iterates over streams in the provided decoder. All streams sections are iterated over in order.

func IterSection

func IterSection(ctx context.Context, dec encoding.StreamsDecoder, section *filemd.SectionInfo) result.Seq[Stream]

Types

type Metrics

type Metrics struct {
	// contains filtered or unexported fields
}

Metrics instruments the streams section.

func NewMetrics

func NewMetrics() *Metrics

NewMetrics creates a new set of metrics for the streams section.

func (*Metrics) Register

func (m *Metrics) Register(reg prometheus.Registerer) error

Register registers metrics to report to reg.

func (*Metrics) Unregister

func (m *Metrics) Unregister(reg prometheus.Registerer)

Unregister unregisters metrics from the provided Registerer.

type Stream

type Stream struct {
	// ID to uniquely represent a stream in a data object. Valid IDs start at 1.
	// IDs are used to track streams across multiple sections in the same data
	// object.
	ID int64

	Labels           labels.Labels // Stream labels.
	MinTimestamp     time.Time     // Minimum timestamp in the stream.
	MaxTimestamp     time.Time     // Maximum timestamp in the stream.
	UncompressedSize int64         // Uncompressed size of the log lines and structured metadata values in the stream.
	Rows             int           // Number of rows in the stream.
}

A Stream is an individual stream within a data object.

func Decode

func Decode(columns []*streamsmd.ColumnDesc, row dataset.Row) (Stream, error)

Decode decodes a stream from a dataset.Row, using the provided columns to determine the column type. The list of columns must match the columns used to create the row.

func (*Stream) Reset

func (s *Stream) Reset()

Reset zeroes all values in the stream struct so it can be reused.

type Streams

type Streams struct {
	// contains filtered or unexported fields
}

Streams tracks information about streams in a data object.

func New

func New(metrics *Metrics, pageSize int) *Streams

New creates a new Streams section. The pageSize argument specifies how large pages should be.

func (*Streams) EncodeTo

func (s *Streams) EncodeTo(enc *encoding.Encoder) error

EncodeTo encodes the list of recorded streams to the provided encoder.

EncodeTo may generate multiple sections if the list of streams is too big to fit into a single section.

Streams.Reset is invoked after encoding, even if encoding fails.

func (*Streams) EstimatedSize

func (s *Streams) EstimatedSize() int

EstimatedSize returns the estimated size of the Streams section in bytes.

func (*Streams) Record

func (s *Streams) Record(streamLabels labels.Labels, ts time.Time, recordSize int64) int64

Record a stream record within the Streams section. The provided timestamp is used to track the minimum and maximum timestamp of a stream. The number of calls to Record is used to track the number of rows for a stream. The recordSize is used to track the uncompressed size of the stream.

The stream ID of the recorded stream is returned.

func (*Streams) Reset

func (s *Streams) Reset()

Reset resets all state, allowing Streams to be reused.

func (*Streams) StreamID

func (s *Streams) StreamID(streamLabels labels.Labels) int64

StreamID returns the stream ID for the provided streamLabels. If the stream has not been recorded, StreamID returns 0.

func (*Streams) TimeRange

func (s *Streams) TimeRange() (time.Time, time.Time)

TimeRange returns the minimum and maximum timestamp across all streams.