ftmap

Documentation

Overview

Package ftmap contains Beam pipeline library functions for the FT verifiable map.

Index

• Constants
• func Aggregate(s beam.Scope, treeID int64, fws, annotationMalwares beam.PCollection) (beam.PCollection, beam.PCollection)
• func MakeReleaseLogs(s beam.Scope, treeID int64, logEntries beam.PCollection) (beam.PCollection, beam.PCollection)
• func NodeHash(left, right []byte) []byte
• func RecordHash(data []byte) []byte
• type InputLog
• type InputLogLeaf
• type InputLogMetadata
• type MapBuilder
  • func NewMapBuilder(source InputLog, treeID int64, prefixStrata int) MapBuilder
  • func (b *MapBuilder) Create(s beam.Scope, size int64) (*PipelineResult, error)
• type MapDB
  • func NewMapDB(location string) (*MapDB, error)
  • func (d *MapDB) Aggregation(revision int, fwLogIndex uint64) (api.AggregatedFirmware, error)
  • func (d *MapDB) Init() error
  • func (d *MapDB) LatestRevision() (rev int, logroot types.LogRootV1, count int64, err error)
  • func (d *MapDB) NextWriteRevision() (int, error)
  • func (d *MapDB) Tile(revision int, path []byte) (*batchmap.Tile, error)
  • func (d *MapDB) Versions(revision int, module string) ([]string, error)
  • func (d *MapDB) WriteRevision(rev int, logCheckpoint []byte, count int64) error
• type NoRevisionsFound
• type PipelineResult

Constants

const (
	// FirmwareMetaPartition is the partition index for partition containing FirmwareMetadata
	FirmwareMetaPartition = iota
	// MalwareStatementPartition is the partition index for partition containing MalwareStatement
	MalwareStatementPartition
	// UnclassifiedStatementPartition is the partition index for partition containing anything not classified above
	UnclassifiedStatementPartition
	// MaxPartitions is the total number of supported partitions. Add new partitions here.
	MaxPartitions
)

Functions

func Aggregate

func Aggregate(s beam.Scope, treeID int64, fws, annotationMalwares beam.PCollection) (beam.PCollection, beam.PCollection)

Aggregate will output an entry for each firmware entry in the input log, which includes the firmware metadata, along with an aggregated representation of the annotations for it. The rules are:

• AnnotationMalware: `Good` is true providing there are no malware annotations that claim the firmware is bad.

func MakeReleaseLogs

func MakeReleaseLogs(s beam.Scope, treeID int64, logEntries beam.PCollection) (beam.PCollection, beam.PCollection)

MakeReleaseLogs takes all firmwareLogEntrys and processes these by DeviceID in order to create logs of release revisions. The versions for each DeviceID are sorted (by ID in the original log), and a log is constructed for each device. This method returns two PCollections: 1. the first is of type Entry; the key/value data to include in the map 2. the second is of type DeviceReleaseLog.

func NodeHash

func NodeHash(left, right []byte) []byte

NodeHash returns the hash for an interior tree node with the given left and right hashes.

func RecordHash

func RecordHash(data []byte) []byte

RecordHash returns the content hash for the given record data.

Types

type InputLog

type InputLog interface {
	// Head returns the metadata of available entries.
	// The log checkpoint is a serialized LogRootV1.
	Head() (checkpoint []byte, count int64, err error)
	// Entries returns a PCollection of InputLogLeaf, containing entries in range [start, end).
	Entries(s beam.Scope, start, end int64) beam.PCollection
}

InputLog allows access to entries from the FT Log.

type InputLogLeaf

type InputLogLeaf struct {
	Seq  int64
	Data []byte
}

InputLogLeaf is a leaf in an input log, with its sequence index and data.

type InputLogMetadata

type InputLogMetadata struct {
	Checkpoint []byte
	Entries    int64
}

InputLogMetadata describes the provenance information of the input log to be passed around atomically.

type MapBuilder

type MapBuilder struct {
	// contains filtered or unexported fields
}

MapBuilder contains the static configuration for a map, and allows maps at different log sizes to be built using its methods.

func NewMapBuilder

func NewMapBuilder(source InputLog, treeID int64, prefixStrata int) MapBuilder

NewMapBuilder returns a MapBuilder for a map with the given configuration.

func (*MapBuilder) Create

func (b *MapBuilder) Create(s beam.Scope, size int64) (*PipelineResult, error)

Create builds a map from scratch, using the first `size` entries in the input log. If there aren't enough entries then it will fail. The results of the pipeline on success are returned as a PipelineResult.

type MapDB

type MapDB struct {
	// contains filtered or unexported fields
}

MapDB provides read/write access to the generated Map tiles.

func NewMapDB

func NewMapDB(location string) (*MapDB, error)

NewMapDB creates a MapDB using a file at the given location. If the file doesn't exist it will be created.

func (*MapDB) Aggregation

func (d *MapDB) Aggregation(revision int, fwLogIndex uint64) (api.AggregatedFirmware, error)

Aggregation gets the aggregation for the firmware at the given log index.

func (*MapDB) Init

func (d *MapDB) Init() error

Init creates the database tables if needed.

func (*MapDB) LatestRevision

func (d *MapDB) LatestRevision() (rev int, logroot types.LogRootV1, count int64, err error)

LatestRevision gets the metadata for the last completed write.

func (*MapDB) NextWriteRevision

func (d *MapDB) NextWriteRevision() (int, error)

NextWriteRevision gets the revision that the next generation of the map should be written at.

func (*MapDB) Tile

func (d *MapDB) Tile(revision int, path []byte) (*batchmap.Tile, error)

Tile gets the tile at the given path in the given revision of the map.

func (*MapDB) Versions

func (d *MapDB) Versions(revision int, module string) ([]string, error)

Versions gets the log of versions for the given module in the given map revision.

func (*MapDB) WriteRevision

func (d *MapDB) WriteRevision(rev int, logCheckpoint []byte, count int64) error

WriteRevision writes the metadata for a completed run into the database. If this method isn't called then the tiles may be written but this revision will be skipped by sensible readers because the provenance information isn't available.

type NoRevisionsFound

type NoRevisionsFound = error

NoRevisionsFound is returned when the DB appears valid but has no revisions in it.

type PipelineResult

type PipelineResult struct {
	MapTiles           beam.PCollection
	DeviceLogs         beam.PCollection
	AggregatedFirmware beam.PCollection
	Metadata           InputLogMetadata
}

PipelineResult is returned on successful run of the pipeline. It primarily exists to name the output and aid readability, as PCollections are untyped in code, so having them as named fields at least aids a little.