rebuild

package
v0.0.0-...-ad8c3d5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 18, 2024 License: Apache-2.0 Imports: 40 Imported by: 0

Documentation

Overview

Package rebuild provides functionality to rebuild packages.

Package rebuild contains interfaces implementing generic rebuild functionality.

Index

Constants

View Source 
const (
	RetainArtifactsID ctxKey = iota
	AssetDirID
	UploadArtifactsPathID
	DebugStoreID
	RepoCacheClientID
	HTTPBasicClientID
	InvocationID
	TimewarpID
	RunID
	GCSClientOptionsID
)

Variables

View Source 
var (
	// ErrNoUploadPath is an error indicating that no upload path was provided and a DebugStorer couldn't be constructed.
	ErrNoUploadPath = errors.New("no artifact upload path provided")
	// ErrAssetNotFound indicates the asset requested to be read could not be found.
	ErrAssetNotFound = errors.New("asset not found")
)

Functions

func AssetCopy 

func AssetCopy(ctx context.Context, to, from AssetStore, a Asset) (string, error)

AssetCopy copies an asset from one store to another and returns the URI of the destination.

func BasicSourceSetup 

func BasicSourceSetup(s Location, env *BuildEnv) (string, error)

BasicSourceSetup provides a common source setup script.

func DoContext 

func DoContext(ctx context.Context, req *http.Request) (*http.Response, error)

DoContext attempts to make an external HTTP request using the gateway client, if configured.

func ExecuteScript 

func ExecuteScript(ctx context.Context, dir string, script string) (string, error)

ExecuteScript executes a single step of the strategy and returns the output regardless of error.

func FindTagMatch 

func FindTagMatch(pkg, version string, repo *git.Repository) (commit string, err error)

FindTagMatch searches a repositories tags for a possible version match and returns the commit hash.

func LoadRepo 

func LoadRepo(ctx context.Context, pkg string, s storage.Storer, fs billy.Filesystem, opt git.CloneOptions) (*git.Repository, error)

LoadRepo attempts to either reuse the local or load the remote repo specified in CloneOptions.

If rebuild.RepoCacheClientID is present, a Git cache service will be used instead of the remote defined in CloneOptions.URL.

func MatchTag 

func MatchTag(tag, pkg, version string) (strict bool, approx bool)

MatchTag evaluates whether the given tag is likely to refer to a package version.

func PopulateTemplate 

func PopulateTemplate(tmpl string, data any) (string, error)

PopulateTemplate is a helper to execute a template string using a data object.

func RebuildOne 

func RebuildOne(ctx context.Context, r Rebuilder, input Input, mux RegistryMux, rcfg *RepoConfig, fs billy.Filesystem, s storage.Storer, assets AssetStore) (*Verdict, []Asset, error)

RebuildOne runs a rebuild for the given package artifact.

func RebuildRemote 

func RebuildRemote(ctx context.Context, input Input, id string, opts RemoteOptions) error

RebuildRemote executes the given target strategy on a remote builder.

func ScopedLogCapture 

func ScopedLogCapture(l *log.Logger, w io.Writer) func()

func Summarize 

func Summarize(ctx context.Context, t Target, rb, up Asset, assets AssetStore) (csRB, csUP *archive.ContentSummary, err error)

Summarize constructs ContentSummary objects for the upstream and rebuilt artifacts.

Types

type Asset 

type Asset struct {
	Type   AssetType
	Target Target
}

Asset represents one of many side effects of rebuilding a single artifact.

Examples include the upstream artifact, the rebuilt artifact, or the logs.

func Canonicalize 

func Canonicalize(ctx context.Context, t Target, mux RegistryMux, rbPath string, fs billy.Filesystem, assets AssetStore) (rb, up Asset, err error)

Canonicalize canonicalizes the upstream and rebuilt artifacts.

type AssetStore 

type AssetStore interface {
	Reader(ctx context.Context, a Asset) (io.ReadCloser, string, error)
	Writer(ctx context.Context, a Asset) (io.WriteCloser, string, error)
}

AssetStore is a storage mechanism for debug assets.

func DebugStoreFromContext 

func DebugStoreFromContext(ctx context.Context) (AssetStore, error)

DebugStoreFromContext constructs a DebugStorer using values from the given context.

type AssetType 

type AssetType string

AssetType is the type of an asset we're storing for debug later. 

const (
	// DebugRebuildAsset is the artifact that we rebuilt.
	DebugRebuildAsset AssetType = "rebuild"
	// DebugUpstreamAsset is the upstream artifact we compared against.
	DebugUpstreamAsset AssetType = "upstream"
	// DebugLogsAsset is the log we collected.
	DebugLogsAsset AssetType = "logs"

	// RebuildAsset is the artifact associated with the Target.
	RebuildAsset AssetType = "<artifact>"
	// DockerfileAsset is the Dockerfile used to create the builder.
	DockerfileAsset AssetType = "Dockerfile"
	// BuildInfoAsset is the serialized BuildInfo summarizing the remote rebuild.
	BuildInfoAsset AssetType = "info.json"
	// ContainerImageAsset is the container state after executing the rebuild.
	ContainerImageAsset AssetType = "image.tgz"

	// AttestationBundleAsset is the signed attestation bundle generated for a rebuild.
	AttestationBundleAsset AssetType = "rebuild.intoto.jsonl"

	// BuildDef is the build definition, including strategy.
	BuildDef AssetType = "build.yaml"
)

type BuildEnv 

type BuildEnv struct {
	TimewarpHost           string
	HasRepo                bool
	PreferPreciseToolchain bool
}

BuildEnv contains resources provided by the build environment that a strategy may use.

func (*BuildEnv) TimewarpURL 

func (b *BuildEnv) TimewarpURL(ecosystem string, registryTime time.Time) (string, error)

TimewarpURL constructs the correct URL for this ecosystem and registryTime.

type BuildInfo 

type BuildInfo struct {
	Target      Target
	ID          string
	Builder     string
	BuildID     string
	BuildStart  time.Time
	BuildEnd    time.Time
	BuildImages map[string]string
	Steps       []*cloudbuild.BuildStep
}

type Ecosystem 

type Ecosystem string

Ecosystem represents a package ecosystem. 

const (
	NPM      Ecosystem = "npm"
	PyPI     Ecosystem = "pypi"
	CratesIO Ecosystem = "cratesio"
	Maven    Ecosystem = "maven"
)

Ecosystem constants. These are used to select an ecosystem, and used as prefixes in storage.

type FilesystemAssetStore 

type FilesystemAssetStore struct {
	// contains filtered or unexported fields
}

FilesystemAssetStore will store assets in a billy.Filesystem

func NewFilesystemAssetStore 

func NewFilesystemAssetStore(fs billy.Filesystem) *FilesystemAssetStore

NewFilesystemAssetStore creates a new FilesystemAssetStore.

func (*FilesystemAssetStore) Reader 

func (s *FilesystemAssetStore) Reader(ctx context.Context, a Asset) (r io.ReadCloser, uri string, err error)

Reader returns a reader for the given asset.

func (*FilesystemAssetStore) Writer 

func (s *FilesystemAssetStore) Writer(ctx context.Context, a Asset) (r io.WriteCloser, uri string, err error)

Writer returns a writer for the given asset.

type GCSStore 

type GCSStore struct {
	// contains filtered or unexported fields
}

GCSStore is a debug asset storage backed by GCS.

func NewGCSStore 

func NewGCSStore(ctx context.Context, uploadPrefix string) (*GCSStore, error)

NewGCSStore creates a new GCSStore.

func (*GCSStore) Reader 

func (s *GCSStore) Reader(ctx context.Context, a Asset) (r io.ReadCloser, uri string, err error)

Reader returns a reader for the given asset.

func (*GCSStore) Writer 

func (s *GCSStore) Writer(ctx context.Context, a Asset) (r io.WriteCloser, uri string, err error)

Writer returns a writer for the given asset.

type Input 

type Input struct {
	Target   Target
	Strategy Strategy
}

Input is a request to rebuild a single target.

type Instructions 

type Instructions struct {
	// The location these instructions should be executed from.
	Location   Location
	SystemDeps []string
	Source     string
	Deps       string
	Build      string
	// Where the generated artifact can be found.
	OutputPath string
}

Instructions represents the source, dependencies, and build steps to execute a rebuild.

type Location 

type Location struct {
	Repo string `json:"repo"`
	Ref  string `json:"ref"`
	Dir  string `json:"dir"`
}

Location is where a set of rebuild instruction should be executed.

type LocationHint 

type LocationHint struct {
	Location
}

LocationHint is a partial strategy used to provide a hint (git repo, git ref) to the inference machinery, but it is not sufficient for execution.

func (*LocationHint) GenerateFor 

func (s *LocationHint) GenerateFor(t Target, be BuildEnv) (Instructions, error)

GenerateFor is unsupported for LocationHint.

type ManualStrategy 

type ManualStrategy struct {
	Location
	Deps       string
	Build      string
	SystemDeps []string
	OutputPath string
}

ManualStrategy allows full control over the build instruction steps, for builds that don't fit any other strategy.

func (*ManualStrategy) GenerateFor 

func (s *ManualStrategy) GenerateFor(t Target, be BuildEnv) (Instructions, error)

GenerateFor generates the instructions for a ManualStrategy.

type Rebuilder 

type Rebuilder interface {
	InferRepo(context.Context, Target, RegistryMux) (string, error)
	CloneRepo(context.Context, Target, string, billy.Filesystem, storage.Storer) (RepoConfig, error)
	InferStrategy(context.Context, Target, RegistryMux, *RepoConfig, Strategy) (Strategy, error)
	Rebuild(context.Context, Target, Instructions, billy.Filesystem) error
	Compare(context.Context, Target, Asset, Asset, AssetStore, Instructions) (error, error)
}

Rebuilder defines the operations used to rebuild an ecosystem's packages.

type RegistryMux 

type RegistryMux struct {
	NPM      npm.Registry
	PyPI     pypi.Registry
	CratesIO cratesio.Registry
}

RegistryMux offers a unified accessor for package registries.

func RegistryMuxWithCache 

func RegistryMuxWithCache(registry RegistryMux, c cacheinternal.Cache) (RegistryMux, error)

RegistryMuxWithCache returns a new RegistryMux with the provided cache wrapping each registry.

type RemoteOptions 

type RemoteOptions struct {
	Project             string
	BuildServiceAccount string
	LogsBucket          string
	MetadataBucket      string
	UtilPrebuildBucket  string
	// TODO: Consider moving this to Strategy.
	UseTimewarp bool
}

RemoteOptions provides the configuration to execute rebuilds on Cloud Build.

type RepoConfig 

type RepoConfig struct {
	*git.Repository
	URI    string
	Dir    string
	RefMap map[string]string
}

RepoConfig describes the repo currently being used.

type Strategy 

type Strategy interface {
	GenerateFor(Target, BuildEnv) (Instructions, error)
}

Strategy generates instructions to execute a rebuild.

type Target 

type Target struct {
	Ecosystem Ecosystem
	Package   string
	Version   string
	Artifact  string
}

Target is a single target we might attempt to rebuild.

func (Target) ArchiveType 

func (t Target) ArchiveType() archive.Format

ArchiveType provide the Target's archive.Format.

type Timings 

type Timings struct {
	CloneEstimate time.Duration
	Source        time.Duration
	Infer         time.Duration
	Build         time.Duration
}

Timings describe how long different sections of the rebuild took.

func (Timings) EstimateCleanBuild 

func (t Timings) EstimateCleanBuild() time.Duration

func (Timings) Total 

func (t Timings) Total() time.Duration

type Verdict 

type Verdict struct {
	Target   Target
	Message  string
	Strategy Strategy
	Timings  Timings
}

Verdict is the result of a single rebuild attempt.

func RebuildMany 

func RebuildMany(ctx context.Context, rebuilder Rebuilder, inputs []Input, registry RegistryMux) ([]Verdict, error)

RebuildMany executes rebuilds for each provided rebuild.Input returning their rebuild.Verdicts.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.