Version: v0.3.0 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Jul 3, 2020 License: Apache-2.0 Imports: 26 Imported by: 2



Package sequencer reads mutations and applies them to the Trillian Map.



This section is empty.


This section is empty.


func PeriodicallyRun

func PeriodicallyRun(ctx context.Context, tickch <-chan time.Time, f func(ctx context.Context))

PeriodicallyRun executes f once per tick until ctx is closed. Closing ctx will also stop any in-flight operation mid-way through.


type Batcher

type Batcher interface {
	// WriteBatchSources saves the (low, high] boundaries used for each log in making this revision.
	WriteBatchSources(ctx context.Context, dirID string, rev int64, meta *spb.MapMetadata) error
	// ReadBatch returns the batch definitions for a given revision.
	ReadBatch(ctx context.Context, directoryID string, rev int64) (*spb.MapMetadata, error)
	// HighestRev returns the highest defined revision number for directoryID.
	HighestRev(ctx context.Context, directoryID string) (int64, error)

Batcher writes batch definitions to storage.

type LogsReader

type LogsReader interface {
	// HighWatermark counts up to `batchSize` entries in the specified log,
	// located at or after the given `start` watermark. Returns the number of
	// entries found, and the watermark "just beyond" the last entry found.
	// Guarantees:
	// - The returned `count` is normally between 0 and `batchSize`. It can be
	//   more if the storage supports batching multiple entries with the same
	//   watermark key. The `batchSize` is a hint rather than a hard limit.
	// - The returned `high` watermark is at least equal to `start`.
	// - There are exactly `count` entries in the [`start`, `high`) range.
	// - The content of the [`start`, `high`) range will never change. The caller
	//   can take it as a promise, and set `start` for the next HighWatermark
	//   call to be the returned `high`.
	// - If `high` == `start` then there are no entries found, i.e. `count` is 0.
	//   Note that it's also possible that `high` > `start` but there are still
	//   no entries found.
	HighWatermark(ctx context.Context, directoryID string, logID int64,
		start water.Mark, batchSize int32) (count int32, high water.Mark, err error)

	// ListLogs returns the logIDs associated with directoryID that have their write bits set,
	// or all logIDs associated with directoryID if writable is false.
	ListLogs(ctx context.Context, directoryID string, writable bool) ([]int64, error)

	// ReadLog returns up to `batchSize` lowest messages in the [low, high)
	// watermarks range of the specified log. Some implementations may return
	// more because there can be multiple entries with the same watermark, but
	// different local IDs. The entries are ordered by (watermark, local ID).
	ReadLog(ctx context.Context, directoryID string, logID int64, low, high water.Mark,
		batchSize int32) ([]*mutator.LogMessage, error)

LogsReader reads messages in multiple logs.

type MapClient

type MapClient struct {
	// contains filtered or unexported fields

MapClient interacts with the Trillian Map and verifies its responses.

func (*MapClient) GetAndVerifyLatestMapRoot

func (c *MapClient) GetAndVerifyLatestMapRoot(ctx context.Context) (*tpb.SignedMapRoot, *types.MapRootV1, error)

GetAndVerifyLatestMapRoot verifies and returns the latest map root.

func (*MapClient) GetAndVerifyMapRootByRevision

func (c *MapClient) GetAndVerifyMapRootByRevision(ctx context.Context,
	rev int64) (*tpb.SignedMapRoot, *types.MapRootV1, error)

GetAndVerifyMapRootByRevision verifies and returns a specific map root.

func (*MapClient) GetMapLeavesByRevisionNoProof

func (c *MapClient) GetMapLeavesByRevisionNoProof(ctx context.Context, revision int64, indexes [][]byte) ([]*tpb.MapLeaf, error)

GetMapLeavesByRevisionNoProof returns the requested map leaves at a specific revision. indexes may not contain duplicates.

type MapWriteClient

type MapWriteClient struct {
	MapID int64
	// contains filtered or unexported fields

func (*MapWriteClient) GetLeavesByRevision

func (c *MapWriteClient) GetLeavesByRevision(ctx context.Context, rev int64, indexes [][]byte) ([]*tpb.MapLeaf, error)

func (*MapWriteClient) WriteLeaves

func (c *MapWriteClient) WriteLeaves(ctx context.Context, rev int64, leaves []*tpb.MapLeaf) error

type Sequencer

type Sequencer struct {
	// contains filtered or unexported fields

Sequencer processes mutations and sends them to the trillian map.

func New

func New(
	sequencerClient spb.KeyTransparencySequencerClient,
	directories directory.Storage,
	tracker *election.Tracker,
) *Sequencer

New creates a new instance of the signer.

func (*Sequencer) AddAllDirectories

func (s *Sequencer) AddAllDirectories(ctx context.Context) error

AddAllDirectories adds all directories to the set of resources this sequencer attempts to obtain mastership for.

func (*Sequencer) AddDirectory

func (s *Sequencer) AddDirectory(dirIDs ...string)

AddDirectory adds dirIDs to the set of resources this sequencer attempts to obtain mastership for.

func (*Sequencer) ApplyRevisionsForAllMasterships

func (s *Sequencer) ApplyRevisionsForAllMasterships(ctx context.Context) error

ApplyRevisionsForAllMasterships runs KeyTransparencySequencerClient's ApplyRevisions method on all directories that this sequencer is currently master for.

func (*Sequencer) DefineRevisionsForAllMasterships

func (s *Sequencer) DefineRevisionsForAllMasterships(ctx context.Context, batchSize int32) error

DefineRevisionsForAllMasterships runs KeyTransparencySequencerClient's DefineRevisions method on all directories that this sequencer is currently master for.

func (*Sequencer) ForAllMasterships

func (s *Sequencer) ForAllMasterships(ctx context.Context, f func(ctx context.Context, dirID string) error) error

ForAllMasterships runs f once for all directories this server is master for.

func (*Sequencer) PublishLogForAllMasterships

func (s *Sequencer) PublishLogForAllMasterships(ctx context.Context) error

PublishLogForAllMasterships runs KeyTransparencySequencer.PublishRevisions on all directories this sequencer is currently master for.

func (*Sequencer) TrackMasterships

func (s *Sequencer) TrackMasterships(ctx context.Context)

TrackMasterships monitors resources for mastership.

type Server

type Server struct {
	BatchSize              int32
	ApplyRevisionBatchSize uint64
	LogPublishBatchSize    uint64
	// contains filtered or unexported fields

Server implements KeyTransparencySequencerServer.

func NewServer

func NewServer(
	directories directory.Storage,
	tlog tpb.TrillianLogClient,
	tmap tpb.TrillianMapClient,
	twrite tpb.TrillianMapWriteClient,
	batcher Batcher,
	logs LogsReader,
	loopback spb.KeyTransparencySequencerClient,
	metricsFactory monitoring.MetricFactory,
) *Server

NewServer creates a new KeyTransparencySequencerServer.

func (*Server) ApplyRevision

ApplyRevision applies the supplied mutations to the current map revision and creates a new revision.

func (*Server) ApplyRevisions

func (s *Server) ApplyRevisions(ctx context.Context, in *spb.ApplyRevisionsRequest) (*empty.Empty, error)

ApplyRevisions builds multiple outstanding revisions of a single directory's map by integrating the corresponding mutations.

func (*Server) DefineRevisions

DefineRevisions returns the set of outstanding revisions that have not been applied, after optionally defining a new revision of outstanding mutations.

func (*Server) EstimateBacklog

EstimateBacklog updates the log_entryunapplied metric for directoryID

func (*Server) GetDefinedRevisions

GetDefinedRevisions returns the range of defined and unapplied revisions.

func (*Server) HighWatermarks

func (s *Server) HighWatermarks(ctx context.Context, directoryID string, lastMeta *spb.MapMetadata,
	batchSize int32) (int32, *spb.MapMetadata, error)

HighWatermarks returns the total count across all logs and the highest watermark for each log. batchSize is a limit on the total number of items represented by the returned watermarks. TODO(gbelvin): Block until a minBatchSize has been reached or a timeout has occurred.

func (*Server) PublishRevisions

PublishRevisions copies the MapRoots of all known map revisions into the Log of MapRoots.

type Trillian

type Trillian struct {
	// contains filtered or unexported fields

Trillian contains Trillian gRPC clients and metadata about them.

func (*Trillian) LogClient

func (t *Trillian) LogClient(ctx context.Context, dirID string) (trillianLog, error)

LogClient returns a verifying LogClient.

func (*Trillian) MapClient

func (t *Trillian) MapClient(ctx context.Context, dirID string) (trillianMap, error)

MapClient returns a verifying MapClient

func (*Trillian) MapWriteClient

func (t *Trillian) MapWriteClient(ctx context.Context, dirID string) (*MapWriteClient, error)

MapWriteClient returns a connection to the map write API.

type Watermarks

type Watermarks map[int64]int64

Watermarks is a map of watermarks by logID.


Path Synopsis
Package mapper contains a transformation pipeline from log messages to map revisions.
Package mapper contains a transformation pipeline from log messages to map revisions.
Package metadata helps enforce a consistent standard of meaning around the map metadata object.
Package metadata helps enforce a consistent standard of meaning around the map metadata object.
Package runner executes the mapper pipeline.
Package runner executes the mapper pipeline.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL