sequencer

package
Version: v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 3, 2020 License: Apache-2.0 Imports: 26 Imported by: 2

Documentation

Overview

Package sequencer reads mutations and applies them to the Trillian Map.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func PeriodicallyRun

func PeriodicallyRun(ctx context.Context, tickch <-chan time.Time, f func(ctx context.Context))

PeriodicallyRun executes f once per tick until ctx is closed. Closing ctx will also stop any in-flight operation mid-way through.

Types

type Batcher

type Batcher interface {
	// WriteBatchSources saves the (low, high] boundaries used for each log in making this revision.
	WriteBatchSources(ctx context.Context, dirID string, rev int64, meta *spb.MapMetadata) error
	// ReadBatch returns the batch definitions for a given revision.
	ReadBatch(ctx context.Context, directoryID string, rev int64) (*spb.MapMetadata, error)
	// HighestRev returns the highest defined revision number for directoryID.
	HighestRev(ctx context.Context, directoryID string) (int64, error)
}

Batcher writes batch definitions to storage.

type LogsReader

type LogsReader interface {
	// HighWatermark counts up to `batchSize` entries in the specified log,
	// located at or after the given `start` watermark. Returns the number of
	// entries found, and the watermark "just beyond" the last entry found.
	//
	// Guarantees:
	// - The returned `count` is normally between 0 and `batchSize`. It can be
	//   more if the storage supports batching multiple entries with the same
	//   watermark key. The `batchSize` is a hint rather than a hard limit.
	// - The returned `high` watermark is at least equal to `start`.
	// - There are exactly `count` entries in the [`start`, `high`) range.
	// - The content of the [`start`, `high`) range will never change. The caller
	//   can take it as a promise, and set `start` for the next HighWatermark
	//   call to be the returned `high`.
	// - If `high` == `start` then there are no entries found, i.e. `count` is 0.
	//   Note that it's also possible that `high` > `start` but there are still
	//   no entries found.
	HighWatermark(ctx context.Context, directoryID string, logID int64,
		start water.Mark, batchSize int32) (count int32, high water.Mark, err error)

	// ListLogs returns the logIDs associated with directoryID that have their write bits set,
	// or all logIDs associated with directoryID if writable is false.
	ListLogs(ctx context.Context, directoryID string, writable bool) ([]int64, error)

	// ReadLog returns up to `batchSize` lowest messages in the [low, high)
	// watermarks range of the specified log. Some implementations may return
	// more because there can be multiple entries with the same watermark, but
	// different local IDs. The entries are ordered by (watermark, local ID).
	ReadLog(ctx context.Context, directoryID string, logID int64, low, high water.Mark,
		batchSize int32) ([]*mutator.LogMessage, error)
}

LogsReader reads messages in multiple logs.

type MapClient

type MapClient struct {
	*tclient.MapClient
	// contains filtered or unexported fields
}

MapClient interacts with the Trillian Map and verifies its responses.

func (*MapClient) GetAndVerifyLatestMapRoot

func (c *MapClient) GetAndVerifyLatestMapRoot(ctx context.Context) (*tpb.SignedMapRoot, *types.MapRootV1, error)

GetAndVerifyLatestMapRoot verifies and returns the latest map root.

func (*MapClient) GetAndVerifyMapRootByRevision

func (c *MapClient) GetAndVerifyMapRootByRevision(ctx context.Context,
	rev int64) (*tpb.SignedMapRoot, *types.MapRootV1, error)

GetAndVerifyMapRootByRevision verifies and returns a specific map root.

func (*MapClient) GetMapLeavesByRevisionNoProof

func (c *MapClient) GetMapLeavesByRevisionNoProof(ctx context.Context, revision int64, indexes [][]byte) ([]*tpb.MapLeaf, error)

GetMapLeavesByRevisionNoProof returns the requested map leaves at a specific revision. indexes may not contain duplicates.

type MapWriteClient

type MapWriteClient struct {
	MapID int64
	// contains filtered or unexported fields
}

func (*MapWriteClient) GetLeavesByRevision

func (c *MapWriteClient) GetLeavesByRevision(ctx context.Context, rev int64, indexes [][]byte) ([]*tpb.MapLeaf, error)

func (*MapWriteClient) WriteLeaves

func (c *MapWriteClient) WriteLeaves(ctx context.Context, rev int64, leaves []*tpb.MapLeaf) error

type Sequencer

type Sequencer struct {
	// contains filtered or unexported fields
}

Sequencer processes mutations and sends them to the trillian map.

func New

func New(
	sequencerClient spb.KeyTransparencySequencerClient,
	directories directory.Storage,
	tracker *election.Tracker,
) *Sequencer

New creates a new instance of the signer.

func (*Sequencer) AddAllDirectories

func (s *Sequencer) AddAllDirectories(ctx context.Context) error

AddAllDirectories adds all directories to the set of resources this sequencer attempts to obtain mastership for.

func (*Sequencer) AddDirectory

func (s *Sequencer) AddDirectory(dirIDs ...string)

AddDirectory adds dirIDs to the set of resources this sequencer attempts to obtain mastership for.

func (*Sequencer) ApplyRevisionsForAllMasterships

func (s *Sequencer) ApplyRevisionsForAllMasterships(ctx context.Context) error

ApplyRevisionsForAllMasterships runs KeyTransparencySequencerClient's ApplyRevisions method on all directories that this sequencer is currently master for.

func (*Sequencer) DefineRevisionsForAllMasterships

func (s *Sequencer) DefineRevisionsForAllMasterships(ctx context.Context, batchSize int32) error

DefineRevisionsForAllMasterships runs KeyTransparencySequencerClient's DefineRevisions method on all directories that this sequencer is currently master for.

func (*Sequencer) ForAllMasterships

func (s *Sequencer) ForAllMasterships(ctx context.Context, f func(ctx context.Context, dirID string) error) error

ForAllMasterships runs f once for all directories this server is master for.

func (*Sequencer) PublishLogForAllMasterships

func (s *Sequencer) PublishLogForAllMasterships(ctx context.Context) error

PublishLogForAllMasterships runs KeyTransparencySequencer.PublishRevisions on all directories this sequencer is currently master for.

func (*Sequencer) TrackMasterships

func (s *Sequencer) TrackMasterships(ctx context.Context)

TrackMasterships monitors resources for mastership.

type Server

type Server struct {
	BatchSize              int32
	ApplyRevisionBatchSize uint64
	LogPublishBatchSize    uint64
	// contains filtered or unexported fields
}

Server implements KeyTransparencySequencerServer.

func NewServer

func NewServer(
	directories directory.Storage,
	tlog tpb.TrillianLogClient,
	tmap tpb.TrillianMapClient,
	twrite tpb.TrillianMapWriteClient,
	batcher Batcher,
	logs LogsReader,
	loopback spb.KeyTransparencySequencerClient,
	metricsFactory monitoring.MetricFactory,
) *Server

NewServer creates a new KeyTransparencySequencerServer.

func (*Server) ApplyRevision

ApplyRevision applies the supplied mutations to the current map revision and creates a new revision.

func (*Server) ApplyRevisions

func (s *Server) ApplyRevisions(ctx context.Context, in *spb.ApplyRevisionsRequest) (*empty.Empty, error)

ApplyRevisions builds multiple outstanding revisions of a single directory's map by integrating the corresponding mutations.

func (*Server) DefineRevisions

DefineRevisions returns the set of outstanding revisions that have not been applied, after optionally defining a new revision of outstanding mutations.

func (*Server) EstimateBacklog

EstimateBacklog updates the log_entryunapplied metric for directoryID

func (*Server) GetDefinedRevisions

GetDefinedRevisions returns the range of defined and unapplied revisions.

func (*Server) HighWatermarks

func (s *Server) HighWatermarks(ctx context.Context, directoryID string, lastMeta *spb.MapMetadata,
	batchSize int32) (int32, *spb.MapMetadata, error)

HighWatermarks returns the total count across all logs and the highest watermark for each log. batchSize is a limit on the total number of items represented by the returned watermarks. TODO(gbelvin): Block until a minBatchSize has been reached or a timeout has occurred.

func (*Server) PublishRevisions

PublishRevisions copies the MapRoots of all known map revisions into the Log of MapRoots.

type Trillian

type Trillian struct {
	// contains filtered or unexported fields
}

Trillian contains Trillian gRPC clients and metadata about them.

func (*Trillian) LogClient

func (t *Trillian) LogClient(ctx context.Context, dirID string) (trillianLog, error)

LogClient returns a verifying LogClient.

func (*Trillian) MapClient

func (t *Trillian) MapClient(ctx context.Context, dirID string) (trillianMap, error)

MapClient returns a verifying MapClient

func (*Trillian) MapWriteClient

func (t *Trillian) MapWriteClient(ctx context.Context, dirID string) (*MapWriteClient, error)

MapWriteClient returns a connection to the map write API.

type Watermarks

type Watermarks map[int64]int64

Watermarks is a map of watermarks by logID.

Directories

Path Synopsis
Package mapper contains a transformation pipeline from log messages to map revisions.
Package mapper contains a transformation pipeline from log messages to map revisions.
Package metadata helps enforce a consistent standard of meaning around the map metadata object.
Package metadata helps enforce a consistent standard of meaning around the map metadata object.
Package runner executes the mapper pipeline.
Package runner executes the mapper pipeline.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL