cache

README

Caching

All caches are created from a global manager. By default this is a bypassedCache, which performs no caching. One benefit of this is that tests don't need to worry about caching causing issues unless they explicitly need to test the cache and can opt-in using the cache.TestCache(t) helper.

Syft sets a filesystemCache when the cache options are loaded.

When using the filesystemCache all items are stored on disk under a root directory, generally in the form of:

<rootDir>/<named-cache>/<data-version>/path/to/data

Using the cache

The easiest and preferred method to use the cache is a cache.Resolver, which automatically creates a <data-version> based on the structure of the provided type. If the structure changes in any way it will end up with a new version key and all will re populate this new key, ignoring cached values from older, different versions. The resolver will store items using the json package to serialize/deserialize values, so to save space it is encouraged to use omitempty. For example:

type myCacheItem struct {
	Name string `json:"name",omitempty`
}

It is possible to use core types such as pkg.Package as long as they support the standard json serialization, but this is discouraged in order to decouple changes to them from affecting the information stored in the cache.

To get a cache for this type:

resolver := cache.GetResolver[myCacheItem]("myCacheName", "v1")

Using the resolver is a single call, which manages checking for items in the cache, expiry times, and if not found invoking the callback to populate the cache and return a value:

data := resolver.Resolve("some/cache/key", func() (myCacheItem, error) {
	// do things to return a myCacheItem or error
})

If it is common that checking for an item will result in errors, and you do not want to re-run the resolve function when errors are encountered, instead of using GetResolver, you can use GetResolverCachingErrors, which is useful for things such as resolving artifacts over a network, where a number of them will not be resolved, and you do not want to continue to have the expense of running the network resolution. This should be used when it is acceptable a network outage and cached errors is an acceptable risk.

An example can be seen in the golang cataloger fetching remote licenses.

Documentation

Index

• func SetManager(m Manager)
• type Cache
• type Manager
  • func GetManager() Manager
  • func NewFromDir(dir string, ttl time.Duration) (Manager, error)
  • func NewInMemory(ttl time.Duration) Manager
• type ReaderAtCloser
• type Resolver
  • func GetResolver[T any](name, version string) Resolver[T]
  • func GetResolverCachingErrors[T any](name, version string) Resolver[T]

Functions

func SetManager

func SetManager(m Manager)

SetManager sets the global cache manager, which is used to instantiate all caches. Setting this to nil disables caching.

Types

type Cache

type Cache interface {
	// Read returns a reader for the cache value, if found and not expired
	// or errors when unable to find / expired
	Read(key string) (ReaderAtCloser, error)

	// Write writes the contents of the reader to the cache
	// and closes it, if the reader implements io.Closer
	Write(key string, contents io.Reader) error
}

Cache is what the application interacts with to get and set cached data

type Manager

type Manager interface {
	// GetCache returns a cache scoped to the given named, versioned data
	GetCache(name, version string) Cache

	// RootDirs returns any root directories this cache manager uses
	RootDirs() []string
}

Manager is responsible for managing cache data and instantiating all caches

func GetManager

func GetManager() Manager

GetManager returns the global cache manager, which is used to instantiate all caches

func NewFromDir

func NewFromDir(dir string, ttl time.Duration) (Manager, error)

NewFromDir creates a new cache manager which returns caches stored on disk, rooted at the given directory

func NewInMemory

func NewInMemory(ttl time.Duration) Manager

NewInMemory returns an in-memory only cache manager

type ReaderAtCloser

type ReaderAtCloser interface {
	io.Reader
	io.ReaderAt
	io.Closer
}

ReaderAtCloser is an amalgamation of: io.Reader, io.ReaderAt, and io.Closer

type Resolver

type Resolver[T any] interface {
	// Resolve attempts to resolve the given key from cache and convert it to the type of the cache,
	// or calls the resolver function if unable to resolve a cached value
	Resolve(key string, resolver resolverFunc[T]) (T, error)
}

Resolver interface provides a single Resolve method, which will return from cache or call the provided resolve function to get the value if not available in cache

func GetResolver

func GetResolver[T any](name, version string) Resolver[T]

GetResolver returns a cache resolver for persistent cached data across Syft runs, stored in a unique location based on the provided name and versioned by the type

func GetResolverCachingErrors

func GetResolverCachingErrors[T any](name, version string) Resolver[T]

GetResolverCachingErrors returns a Resolver that caches errors and will return them instead of continuing to call the provided resolve functions