sequencer

package
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 29, 2020 License: Apache-2.0 Imports: 26 Imported by: 2

Documentation

Overview

Package sequencer reads mutations and applies them to the Trillian Map.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func PeriodicallyRun 

func PeriodicallyRun(ctx context.Context, tickch <-chan time.Time, f func(ctx context.Context))

PeriodicallyRun executes f once per tick until ctx is closed. Closing ctx will also stop any in-flight operation mid-way through.

Types

type Batcher 

type Batcher interface {
	// WriteBatchSources saves the (low, high] boundaries used for each log in making this revision.
	WriteBatchSources(ctx context.Context, dirID string, rev int64, meta *spb.MapMetadata) error
	// ReadBatch returns the batch definitions for a given revision.
	ReadBatch(ctx context.Context, directoryID string, rev int64) (*spb.MapMetadata, error)
	// HighestRev returns the highest defined revision number for directoryID.
	HighestRev(ctx context.Context, directoryID string) (int64, error)
}

Batcher writes batch definitions to storage.

type LogsReader 

type LogsReader interface {
	// HighWatermark counts up to `batchSize` entries in the specified log,
	// located at or after the given `start` watermark. Returns the number of
	// entries found, and the watermark "just beyond" the last entry found.
	//
	// Guarantees:
	// - The returned `count` is normally between 0 and `batchSize`. It can be
	//   more if the storage supports batching multiple entries with the same
	//   watermark key. The `batchSize` is a hint rather than a hard limit.
	// - The returned `high` watermark is at least equal to `start`.
	// - There are exactly `count` entries in the [`start`, `high`) range.
	// - The content of the [`start`, `high`) range will never change. The caller
	//   can take it as a promise, and set `start` for the next HighWatermark
	//   call to be the returned `high`.
	// - If `high` == `start` then there are no entries found, i.e. `count` is 0.
	//   Note that it's also possible that `high` > `start` but there are still
	//   no entries found.
	HighWatermark(ctx context.Context, directoryID string, logID int64,
		start water.Mark, batchSize int32) (count int32, high water.Mark, err error)

	// ListLogs returns the logIDs associated with directoryID that have their write bits set,
	// or all logIDs associated with directoryID if writable is false.
	ListLogs(ctx context.Context, directoryID string, writable bool) ([]int64, error)

	// ReadLog returns up to `batchSize` lowest messages in the [low, high)
	// watermarks range of the specified log. Some implementations may return
	// more because there can be multiple entries with the same watermark, but
	// different local IDs. The entries are ordered by (watermark, local ID).
	ReadLog(ctx context.Context, directoryID string, logID int64, low, high water.Mark,
		batchSize int32) ([]*mutator.LogMessage, error)
}

LogsReader reads messages in multiple logs.

type MapClient 

type MapClient struct {
	*tclient.MapClient
}

MapClient interacts with the Trillian Map and verifies its responses.

func (*MapClient) GetAndVerifyLatestMapRoot 

func (c *MapClient) GetAndVerifyLatestMapRoot(ctx context.Context) (*tpb.SignedMapRoot, *types.MapRootV1, error)

GetAndVerifyLatestMapRoot verifies and returns the latest map root.

func (*MapClient) GetAndVerifyMapRootByRevision 

func (c *MapClient) GetAndVerifyMapRootByRevision(ctx context.Context,
	rev int64) (*tpb.SignedMapRoot, *types.MapRootV1, error)

GetAndVerifyMapRootByRevision verifies and returns a specific map root.

func (*MapClient) GetMapLeavesByRevisionNoProof 

func (c *MapClient) GetMapLeavesByRevisionNoProof(ctx context.Context, revision int64, indexes [][]byte) ([]*tpb.MapLeaf, error)

GetMapLeavesByRevisionNoProof returns the requested map leaves at a specific revision. indexes may not contain duplicates.

type MapWriteClient 

type MapWriteClient struct {
	MapID int64
	// contains filtered or unexported fields
}

func (*MapWriteClient) GetLeavesByRevision 

func (c *MapWriteClient) GetLeavesByRevision(ctx context.Context, rev int64, indexes [][]byte) ([]*tpb.MapLeaf, error)

func (*MapWriteClient) WriteLeaves 

func (c *MapWriteClient) WriteLeaves(ctx context.Context, rev int64, leaves []*tpb.MapLeaf) error

type Sequencer 

type Sequencer struct {
	// contains filtered or unexported fields
}

Sequencer processes mutations and sends them to the trillian map.

func New 

func New(
	sequencerClient spb.KeyTransparencySequencerClient,
	directories directory.Storage,
	tracker *election.Tracker,
) *Sequencer

New creates a new instance of the signer.

func (*Sequencer) AddAllDirectories 

func (s *Sequencer) AddAllDirectories(ctx context.Context) error

AddAllDirectories adds all directories to the set of resources this sequencer attempts to obtain mastership for.

func (*Sequencer) AddDirectory 

func (s *Sequencer) AddDirectory(dirIDs ...string)

AddDirectory adds dirIDs to the set of resources this sequencer attempts to obtain mastership for.

func (*Sequencer) ApplyRevisionsForAllMasterships 

func (s *Sequencer) ApplyRevisionsForAllMasterships(ctx context.Context) error

ApplyRevisionsForAllMasterships runs KeyTransparencySequencerClient's ApplyRevisions method on all directories that this sequencer is currently master for.

func (*Sequencer) DefineRevisionsForAllMasterships 

func (s *Sequencer) DefineRevisionsForAllMasterships(ctx context.Context, batchSize int32) error

DefineRevisionsForAllMasterships runs KeyTransparencySequencerClient's DefineRevisions method on all directories that this sequencer is currently master for.

func (*Sequencer) ForAllMasterships 

func (s *Sequencer) ForAllMasterships(ctx context.Context, f func(ctx context.Context, dirID string) error) error

ForAllMasterships runs f once for all directories this server is master for.

func (*Sequencer) PublishLogForAllMasterships 

func (s *Sequencer) PublishLogForAllMasterships(ctx context.Context) error

PublishLogForAllMasterships runs KeyTransparencySequencer.PublishRevisions on all directories this sequencer is currently master for.

func (*Sequencer) TrackMasterships 

func (s *Sequencer) TrackMasterships(ctx context.Context)

TrackMasterships monitors resources for mastership.

type Server 

type Server struct {
	BatchSize              int32
	ApplyRevisionBatchSize uint64
	LogPublishBatchSize    uint64
	// contains filtered or unexported fields
}

Server implements KeyTransparencySequencerServer.

func NewServer 

func NewServer(
	directories directory.Storage,
	tlog tpb.TrillianLogClient,
	tmap tpb.TrillianMapClient,
	twrite tpb.TrillianMapWriteClient,
	batcher Batcher,
	logs LogsReader,
	loopback spb.KeyTransparencySequencerClient,
	metricsFactory monitoring.MetricFactory,
) *Server

NewServer creates a new KeyTransparencySequencerServer.

func (*Server) ApplyRevision 

func (s *Server) ApplyRevision(ctx context.Context, in *spb.ApplyRevisionRequest) (*spb.ApplyRevisionResponse, error)

ApplyRevision applies the supplied mutations to the current map revision and creates a new revision.

func (*Server) ApplyRevisions 

func (s *Server) ApplyRevisions(ctx context.Context, in *spb.ApplyRevisionsRequest) (*empty.Empty, error)

ApplyRevisions builds multiple outstanding revisions of a single directory's map by integrating the corresponding mutations.

func (*Server) DefineRevisions 

func (s *Server) DefineRevisions(ctx context.Context,
	in *spb.DefineRevisionsRequest) (*spb.DefineRevisionsResponse, error)

DefineRevisions returns the set of outstanding revisions that have not been applied, after optionally defining a new revision of outstanding mutations.

func (*Server) EstimateBacklog 

func (s *Server) EstimateBacklog(ctx context.Context, in *spb.EstimateBacklogRequest) (*spb.EstimateBacklogResponse, error)

EstimateBacklog updates the log_entryunapplied metric for directoryID

func (*Server) GetDefinedRevisions 

func (s *Server) GetDefinedRevisions(ctx context.Context,
	in *spb.GetDefinedRevisionsRequest) (*spb.GetDefinedRevisionsResponse, error)

GetDefinedRevisions returns the range of defined and unapplied revisions.

func (*Server) HighWatermarks 

func (s *Server) HighWatermarks(ctx context.Context, directoryID string, lastMeta *spb.MapMetadata,
	batchSize int32) (int32, *spb.MapMetadata, error)

HighWatermarks returns the total count across all logs and the highest watermark for each log. batchSize is a limit on the total number of items represented by the returned watermarks. TODO(gbelvin): Block until a minBatchSize has been reached or a timeout has occurred.

func (*Server) PublishRevisions 

func (s *Server) PublishRevisions(ctx context.Context,
	in *spb.PublishRevisionsRequest) (*spb.PublishRevisionsResponse, error)

PublishRevisions copies the MapRoots of all known map revisions into the Log of MapRoots.

type Trillian 

type Trillian struct {
	// contains filtered or unexported fields
}

Trillian contains Trillian gRPC clients and metadata about them.

func (*Trillian) LogClient 

func (t *Trillian) LogClient(ctx context.Context, dirID string) (trillianLog, error)

LogClient returns a verifying LogClient.

func (*Trillian) MapClient 

func (t *Trillian) MapClient(ctx context.Context, dirID string) (trillianMap, error)

MapClient returns a verifying MapClient

func (*Trillian) MapWriteClient 

func (t *Trillian) MapWriteClient(ctx context.Context, dirID string) (*MapWriteClient, error)

MapWriteClient returns a connection to the map write API.

type Watermarks 

type Watermarks map[int64]int64

Watermarks is a map of watermarks by logID.

Source Files

View all Source files

Directories

Path Synopsis
Package mapper contains a transformation pipeline from log messages to map revisions.
 Package mapper contains a transformation pipeline from log messages to map revisions.
Package metadata helps enforce a consistent standard of meaning around the map metadata object.
 Package metadata helps enforce a consistent standard of meaning around the map metadata object.
Package runner executes the mapper pipeline.
 Package runner executes the mapper pipeline.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.