digest

package
v0.0.0-...-932836e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 23, 2020 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// EmptySet is an instance of Set that contains zero elements.
	EmptySet = Set{}
)
View Source 
var (
	// SupportedDigestFunctions is the list of digest functions
	// supported by digest.Digest, using the enumeration values that
	// are part of the Remote Execution protocol.
	SupportedDigestFunctions = []remoteexecution.DigestFunction_Value{
		remoteexecution.DigestFunction_MD5,
		remoteexecution.DigestFunction_SHA1,
		remoteexecution.DigestFunction_SHA256,
		remoteexecution.DigestFunction_SHA384,
		remoteexecution.DigestFunction_SHA512,
	}
)

Functions

func GetDifferenceAndIntersection 

func GetDifferenceAndIntersection(setA Set, setB Set) (onlyA, both Set, onlyB Set)

GetDifferenceAndIntersection partitions the elements stored in sets A and B across three resulting sets: one containing the elements present only in A, one containing the elements present in both A and B, and one containing thelements present only in B.

Types

type Digest 

type Digest struct {
	// contains filtered or unexported fields
}

Digest holds the identification of an object stored in the Content Addressable Storage (CAS) or Action Cache (AC). The use of this object is preferred over remoteexecution.Digest for a couple of reasons.

  • Instances of these objects are guaranteed not to contain any degenerate values. The hash has already been decoded from hexadecimal to binary. The size is non-negative.
  • They keep track of the instance as part of the digest, which allows us to keep function signatures across the codebase simple.
  • They provide utility functions for deriving new digests from them. This ensures that outputs of build actions automatically use the same instance name and hashing algorithm.

Because Digest objects are frequently used as keys (as part of caching data structures or to construct sets without duplicate values), this implementation immediately constructs a key representation upon creation. All functions that extract individual components (e.g., GetInstanceName(), GetHash*() and GetSizeBytes()) operate directly on the key format. 

var (
	// BadDigest is a default instance of Digest. It can, for
	// example, be used as a function return value for error cases.
	BadDigest Digest
)

func MustNewDigest 

func MustNewDigest(instanceName string, hash string, sizeBytes int64) Digest

MustNewDigest constructs a Digest similar to NewDigest, but never returns an error. Instead, execution will abort if the resulting instance would be degenerate. Useful for unit testing.

func NewDigestFromByteStreamReadPath 

func NewDigestFromByteStreamReadPath(path string) (Digest, error)

NewDigestFromByteStreamReadPath creates a Digest from a string having the following format: ${instanceName}/blobs/${hash}/${size}. This notation is used to read files through the ByteStream service.

func NewDigestFromByteStreamWritePath 

func NewDigestFromByteStreamWritePath(path string) (Digest, error)

NewDigestFromByteStreamWritePath creates a Digest from a string having the following format: ${instanceName}/uploads/${uuid}/blobs/${hash}/${size}/${path}. This notation is used to write files through the ByteStream service.

func (Digest) GetByteStreamReadPath 

func (d Digest) GetByteStreamReadPath() string

GetByteStreamReadPath converts the Digest to a string having the following format: ${instanceName}/blobs/${hash}/${size}. This notation is used to read files through the ByteStream service.

func (Digest) GetByteStreamWritePath 

func (d Digest) GetByteStreamWritePath(uuid uuid.UUID) string

GetByteStreamWritePath converts the Digest to a string having the following format: ${instanceName}/uploads/${uuid}/blobs/${hash}/${size}/${path}. This notation is used to write files through the ByteStream service.

func (Digest) GetHashBytes 

func (d Digest) GetHashBytes() []byte

GetHashBytes returns the hash of the object as a slice of bytes.

func (Digest) GetHashString 

func (d Digest) GetHashString() string

GetHashString returns the hash of the object as a string.

func (Digest) GetInstanceName 

func (d Digest) GetInstanceName() InstanceName

GetInstanceName returns the instance name of the object.

func (Digest) GetKey 

func (d Digest) GetKey(format KeyFormat) string

GetKey generates a string representation of the digest object that may be used as keys in hash tables.

func (Digest) GetProto 

func (d Digest) GetProto() *remoteexecution.Digest

GetProto encodes the digest into the format used by the remote execution protocol, so that it may be stored in messages returned to the client.

func (Digest) GetSizeBytes 

func (d Digest) GetSizeBytes() int64

GetSizeBytes returns the size of the object, in bytes.

func (Digest) NewGenerator 

func (d Digest) NewGenerator() *Generator

NewGenerator creates a writer that may be used to compute digests of newly created files.

func (Digest) NewHasher 

func (d Digest) NewHasher() hash.Hash

NewHasher creates a standard hash.Hash object that may be used to compute a checksum of data. The hash.Hash object uses the same algorithm as the one that was used to create the digest, making it possible to validate data against a digest.

func (Digest) String 

func (d Digest) String() string

type ExistenceCache 

type ExistenceCache struct {
	// contains filtered or unexported fields
}

ExistenceCache is a cache of digests, where entries expire once a certain duration of time has passed. It is used by ExistenceCachingBlobAccess to keep track of which objects may be omitted from FindMissing() calls.

It is safe to access ExistenceCache concurrently.

func NewExistenceCache 

func NewExistenceCache(clock clock.Clock, keyFormat KeyFormat, cacheSize int, cacheDuration time.Duration, evictionSet eviction.Set) *ExistenceCache

NewExistenceCache creates a new ExistenceCache that is empty.

func NewExistenceCacheFromConfiguration 

func NewExistenceCacheFromConfiguration(configuration *pb.ExistenceCacheConfiguration, keyFormat KeyFormat, name string) (*ExistenceCache, error)

NewExistenceCacheFromConfiguration is identical to NewExistenceCache(), except that it takes a specification for the object to be created from a configuration file message.

func (*ExistenceCache) Add 

func (ec *ExistenceCache) Add(digests Set)

Add digests to the cache. These digests will automatically be removed once the duration provided to NewExistenceCache passes.

func (*ExistenceCache) RemoveExisting 

func (ec *ExistenceCache) RemoveExisting(digests Set) Set

RemoveExisting removes digests from a provided set that are present in the cache.

type Generator 

type Generator struct {
	// contains filtered or unexported fields
}

Generator is a writer that may be used to compute digests of newly created files.

func (*Generator) Sum 

func (dg *Generator) Sum() Digest

Sum creates a new digest based on the data written into the Generator.

func (*Generator) Write 

func (dg *Generator) Write(p []byte) (int, error)

Write a chunk of data from a newly created file into the state of the Generator.

type InstanceName 

type InstanceName struct {
	// contains filtered or unexported fields
}

InstanceName is a simple container around REv2 instance name strings. Because instance names are embedded in URLs, the REv2 protocol places some restrictions on which instance names are valid. This type can only be instantiated for values that are valid.

func MustNewInstanceName 

func MustNewInstanceName(value string) InstanceName

MustNewInstanceName is identical to NewInstanceName, except that it panics in case the instance name is invalid. This function can be used as part of unit tests.

func NewInstanceName 

func NewInstanceName(value string) (InstanceName, error)

NewInstanceName creates a new InstanceName object that can be used to parse digests.

func (InstanceName) NewDigest 

func (in InstanceName) NewDigest(hash string, sizeBytes int64) (Digest, error)

NewDigest constructs a Digest object from an instance name, hash and object size. The object returned by this function is guaranteed to be non-degenerate.

func (InstanceName) NewDigestFromProto 

func (in InstanceName) NewDigestFromProto(digest *remoteexecution.Digest) (Digest, error)

NewDigestFromProto constructs a Digest object from an instance name and a protocol-level digest object. The object returned by this function is guaranteed to be non-degenerate.

func (InstanceName) String 

func (in InstanceName) String() string

type KeyFormat 

type KeyFormat int

KeyFormat is an enumeration type that determines the format of object keys returned by Digest.GetKey(). 

const (
	// KeyWithoutInstance lets Digest.GetKey() return a key that
	// does not include the name of the instance; only the hash and
	// the size.
	KeyWithoutInstance KeyFormat = iota
	// KeyWithInstance lets Digest.GetKey() return a key that
	// includes the hash, size and instance name.
	KeyWithInstance
)

type Set 

type Set struct {
	// contains filtered or unexported fields
}

Set of digests. Sets are immutable and can be created using SetBuilder.

func GetUnion 

func GetUnion(sets []Set) Set

GetUnion merges all of the elements stored in a list of sets into a single resulting set. This implementation uses a k-way merging algorithm.

func (Set) Empty 

func (s Set) Empty() bool

Empty returns true if the set contains zero elements.

func (Set) First 

func (s Set) First() (Digest, bool)

First returns the first element stored in the set. The boolean return value denotes whether the operation was successful (i.e., the set is non-empty).

func (Set) Items 

func (s Set) Items() []Digest

Items returns a sorted list of all elements stored within the set.

func (Set) Length 

func (s Set) Length() int

Length returns the number of elements stored in the set.

func (Set) RemoveEmptyBlob 

func (s Set) RemoveEmptyBlob() Set

RemoveEmptyBlob returns a copy of the set that has all of the entries corresponding with the empty blob removed.

type SetBuilder 

type SetBuilder struct {
	// contains filtered or unexported fields
}

SetBuilder is a builder for Set objects.

func NewSetBuilder 

func NewSetBuilder() SetBuilder

NewSetBuilder creates a SetBuilder that contains no initial elements.

func (SetBuilder) Add 

func (sb SetBuilder) Add(digest Digest) SetBuilder

Add a single element to the Set that is being built by the SetBuilder.

func (SetBuilder) Build 

func (sb SetBuilder) Build() Set

Build the Set containing the Digests provided to Add().

func (SetBuilder) Length 

func (sb SetBuilder) Length() int

Length returns the number of elements that the Set would contain if built.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.