logstream

Documentation

Overview

Package logstream provides beam utilities to interact with LogStream and LogStreamState datastore entities.

Index

• func BackfillExpireAtFromCreated(s beam.Scope, cloudProject string, logStreamKeys beam.PCollection, ...) beam.PCollection
• type BackfillOptions
• type RandomizedExponentialBackoff
  • func (b *RandomizedExponentialBackoff) Next(ctx context.Context, err error) time.Duration

Functions

func BackfillExpireAtFromCreated

func BackfillExpireAtFromCreated(s beam.Scope, cloudProject string, logStreamKeys beam.PCollection, opts BackfillOptions) beam.PCollection

BackfillExpireAtFromCreated takes a PCollection<KeyBatch> and updates all the entities' ExpireAt timestamp from its Created timestamp according to expiry.

Types

type BackfillOptions

type BackfillOptions struct {
	// DryRun controls whether the datastore updates should be applied.
	DryRun bool
	// Workers controls the number of go worker in each beam worker.
	Workers int
	// BatchSize controls the number of elements each go worker attempt to process
	// at a time.
	BatchSize int
	// SkipCreatedAfter entities created after this timestamp will be skipped.
	// Older entities are always finalized. This allow us to do updates without
	// using a transaction since this is the only process that will update those
	// entities.
	SkipCreatedAfter time.Time
	// Expiry is the duration added to entities creation time to obtain the expiry
	// date.
	Expiry time.Duration
	// RetryCount is the number of times a failed batch will be retried.
	RetryCount int
}

type RandomizedExponentialBackoff

type RandomizedExponentialBackoff struct {
	retry.ExponentialBackoff
	MaxIncreaseRatio float64
}

RandomizedExponentialBackoff is similar to ExponentialBackoff but the actual delay is a random duration between [thisDelay, thisDelay * (1+MaxIncreaseRatio)).

func (*RandomizedExponentialBackoff) Next

func (b *RandomizedExponentialBackoff) Next(ctx context.Context, err error) time.Duration

Next implements Iterator.