artifactory

README

Artifactory

Artifactory is a host tool which uploads build artifacts from the local build directory to a cloud storage bucket in an organized layout. The artifacts which are uploaded are determined by build API metadata, e.g. images.json, blobs.json, binaries.json, etc.

The infrastructure invokes the tool after each build, so that the artifacts may be accessed by downstream clients.

Cloud storage layout

Artifacts specific to a build e.g. images are uploaded to a unique namespace according to a unique -namespace. Artifacts which may be shared across builds, e.g. blobs, debug binaries, etc. are uploaded to a shared namespace. The precise layout is documented in cmd/up.go.

Deduplication

Artifacts which live in shared namespaces are not uploaded more than once. Thus the number of files uploaded, and the runtime of the tool, go down depending on the amount of deduplication across builds and/or repeat invocations.

Signing

The tool can be instructed to sign artifacts prior to upload by providing a ED25519 private key. See the -pkey flag in cmd/up.go and the implementation in sign.go.

Documentation

Index

• func PrivateKey(keyPath string) (ed25519.PrivateKey, error)
• type Data
• type Image
• type Package
• type ProductBundle
• type Upload
  • func BlobsUpload(mods *tools.Modules, destination string) (Upload, error)
  • func BuildAPIModuleUploads(mods *build.Modules, namespace string) []Upload
  • func DebugBinaryUploads(ctx context.Context, mods *build.Modules, ...) ([]Upload, map[string]string, []string, error)
  • func HostTestUploads(testSpecs []build.TestSpec, buildDir, namespace string) ([]Upload, error)
  • func ImageUploads(mods *build.Modules, namespace string) ([]Upload, error)
  • func JiriSnapshotUpload(mods *build.Modules, namespace string) (*Upload, error)
  • func ProductBundleUploads(mods *build.Modules, ...) (*Upload, error)
  • func PublicKeyUpload(namespace string, publicKey ed25519.PublicKey) (*Upload, error)
  • func SDKArchiveUploads(mods *build.Modules, namespace string) []Upload
  • func Sign(uploads []Upload, privateKey ed25519.PrivateKey) ([]Upload, error)
  • func ToolUploads(mods *build.Modules, namespace string) []Upload

Functions

func PrivateKey

func PrivateKey(keyPath string) (ed25519.PrivateKey, error)

PrivateKey parses an ed25519 private key from a pem-formatted key file. It expects the key to be in a PKCS#8, ASN.1 DER format as generated by running `openssl genpkey -algorithm ed25519 -out <private key>.pem`.

Types

type Data

type Data struct {
	DeviceRefs  json.RawMessage `json:"device_refs"`
	Images      []*Image        `json:"images"`
	Type        json.RawMessage `json:"type"`
	Name        json.RawMessage `json:"name"`
	Packages    []*Package      `json:"packages"`
	Description json.RawMessage `json:"description"`
	Metadata    json.RawMessage `json:"metadata,omitempty"`
	Manifests   json.RawMessage `json:"manifests,omitempty"`
}

Data contained in the product bundle.

type Image

type Image struct {
	BaseURI string `json:"base_uri"`
	Format  string `json:"format"`
}

A set of artifacts necessary to provision a physical or virtual device.

type Package

type Package struct {
	Format  string `json:"format"`
	BlobURI string `json:"blob_uri,omitempty"`
	RepoURI string `json:"repo_uri"`
}

A set of artifacts necessary to run a physical or virtual device.

type ProductBundle

type ProductBundle struct {
	Data     Data   `json:"data"`
	SchemaID string `json:"schema_id"`
}

ProductBundle is a struct that contains a collection of images and packages that are outputted from a product build.

type Upload

type Upload struct {
	// Source is the path to the local file to upload.
	Source string

	// Destination is the path to upload to relative to a gcs bucket and namespace.
	Destination string

	// Compress is a directive to gzip the object before uploading.
	Compress bool

	// Contents contains the contents to upload to Destination. This field
	// will only be used if Source is empty.
	Contents []byte

	// Deduplicate gives a collision strategy. If true, then an upload should
	// not fail in the event of a collision, allowing for deduplication of, for
	// example, content-addressed uploads.
	Deduplicate bool

	// Recursive tells whether to recursively upload all files in Source if
	// Source is a directory.
	Recursive bool

	// Metadata contains the metadata to be uploaded with the file.
	Metadata map[string]string

	// TarHeader tells whether or not to compress with tar and contains the
	// associated header.
	TarHeader *tar.Header
}

Upload is a struct that contains source and destination paths to files to upload to GCS.

func BlobsUpload

func BlobsUpload(mods *tools.Modules, destination string) (Upload, error)

BlobsUpload parses all the package manifests in the build and returns an Upload for all the blobs.

func BuildAPIModuleUploads

func BuildAPIModuleUploads(mods *build.Modules, namespace string) []Upload

BuildAPIModuleUploads returns a set of Uploads corresponding to the collection of build API modules in the build.

func DebugBinaryUploads

func DebugBinaryUploads(ctx context.Context, mods *build.Modules, debugNamespace, buildidNamespace string) ([]Upload, map[string]string, []string, error)

DebugBinaryUploads parses the binary manifest associated to a build and returns a list of Uploads of debug binaries, a mapping of build IDs to binary labels, and a list of associated fuchsia build IDs.

func HostTestUploads

func HostTestUploads(testSpecs []build.TestSpec, buildDir, namespace string) ([]Upload, error)

HostTestUploads returns a set of Uploads corresponding to the host tests in testSpecs.

func ImageUploads

func ImageUploads(mods *build.Modules, namespace string) ([]Upload, error)

ImageUploads parses the image manifest located in the build and returns a list of Uploads for the images used for testing.

func JiriSnapshotUpload

func JiriSnapshotUpload(mods *build.Modules, namespace string) (*Upload, error)

JiriSnapshotUpload returns an upload object representing a jiri snapshot.

func ProductBundleUploads

func ProductBundleUploads(mods *build.Modules, packageNamespaceDir, blobNamespaceDir, imageNamespaceDir string) (*Upload, error)

ProductBundleUploads parses the image manifest located in the build and returns a list of Uploads containing only product bundles with updated relative URI's based on artifactory's structure.

func PublicKeyUpload

func PublicKeyUpload(namespace string, publicKey ed25519.PublicKey) (*Upload, error)

PublicKeyUpload returns an Upload representing the public key used for release builds.

func SDKArchiveUploads

func SDKArchiveUploads(mods *build.Modules, namespace string) []Upload

func Sign

func Sign(uploads []Upload, privateKey ed25519.PrivateKey) ([]Upload, error)

Sign signs the upload files and attaches the signatures as metadata.

func ToolUploads

func ToolUploads(mods *build.Modules, namespace string) []Upload

ToolUploads returns a set of tools to upload.