catalog

package
v0.0.0-...-04422ec Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 21, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

Package catalog builds the published pack-registry artifacts from a loaded pack set: the machine-readable catalog.json, the lean suggest.json index, the JSON schemas, and one immutable gzip tarball per pack. The runner's own loader (internal/packs) is the single source of the content hash — this package never re-hashes a pack differently, so the portal, the runner, and the published catalog agree byte-for-byte.

Immutability falls out of content-addressing: a pack tarball lives at v1/packs/<id>/<version>/<sha256hex>/pack.tar.gz, so identical bytes always resolve to the same object and a byte change without a version bump lands at a different path (and is rejected at build time by the drift check). The mutable pointers (catalog.json, suggest.json) are overwritten each publish; the GCS bucket's object versioning keeps every prior generation fetchable.

Index

Constants

View Source 
const DefaultGCSEndpoint = "https://storage.googleapis.com"

DefaultGCSEndpoint is the Google Cloud Storage JSON API base. Overridable (PublishOptions.Endpoint) so tests can point at an httptest server.

View Source 
const DefaultPreviousKept = 3

DefaultPreviousKept is how many prior versions of each pack the catalog carries in previous_versions — "the last few" the portal trust window auto-trusts alongside the current version. Not an operator knob.

View Source 
const DefaultRepoURL = "https://github.com/andrewdryga/emisar"

DefaultRepoURL is the public source repository the catalog links back to for pack and action source. Overridable via BuildOptions.

View Source 
const MaxActionBytes = 32 * 1024

MaxActionBytes is the largest compact trusted action descriptor the MCP API will expose as one indivisible object.

View Source 
const SchemaArtifactVersion = 4

SchemaArtifactVersion versions the immutable authoring-schema suite. Bump it whenever any embedded schema changes, then update each schema's $id to the matching published object path. Older schema objects remain permanently available under their prior filenames.

View Source 
const SchemaVersion = 1

SchemaVersion is the catalog.json document schema version. It is independent of the pack/action on-disk schema versions.

Variables

This section is empty.

Functions

func Schemas 

func Schemas() map[string][]byte

Schemas returns the object-name → bytes map of published JSON schemas.

func Tarball 

func Tarball(files []packs.PackFile) ([]byte, error)

Tarball builds a deterministic gzip-compressed tar of EXACTLY a pack's hash-input files (from Registry.PackFiles) — pack.yaml + its referenced action YAMLs + scripts — with flat pack-relative entry names (pack.yaml, actions/…), exactly what `emisar pack install` extracts and re-hashes.

Building from the hash-input set instead of a directory walk is the trust invariant: the archived bytes are precisely the bytes the content hash covers, so no unreferenced file (a stray README, a .DS_Store, an editor backup) can ride along inside the content-addressed object outside the hash, and an unchanged pack always reproduces identical archive bytes (entry order, mtime, ownership, and mode are all fixed) — a true no-op republish.

func TarballObject 

func TarballObject(id, version, contentHash string) string

TarballObject is the immutable object path for a pack's tarball, content- addressed by its content hash so identical bytes always resolve to the same object (v1/packs/<id>/<version>/<sha256hex>/pack.tar.gz).

func ValidateCatalogDocument 

func ValidateCatalogDocument(data []byte) error

ValidateCatalogDocument checks raw catalog.json bytes against the embedded catalog schema — the same contract published to machine consumers.

It exists so that packctl's `--previous` acceptance and CD's fallback decision cannot drift apart. Both call this; neither hand-rolls a predicate. The shallow jq check CD used before (`schema_version == 1 and packs non-empty`) accepted a catalog whose `previous_versions` had been truncated, and `--previous` accepted anything that merely unmarshalled — so a subtly corrupt history was carried forward silently, amputating pack version windows on the way through.

This is a structural aid, not a trust boundary: the security root stays pack bytes → content hash, exactly as the schema comment above says.

Types

type Action 

type Action struct {
	actionspec.ModelDescriptor
	Command *Command `json:"command,omitempty"`
}

Action is the complete reviewed model-facing contract for one action. The executable command remains only for the approval UI; MCP trust and runner descriptor matching deliberately ignore it.

type Arg 

type Arg = actionspec.ModelArg

Keep the catalog package's descriptor type names as aliases for callers that construct registry fixtures. The canonical definitions live in actionspec.

type BuildOptions 

type BuildOptions struct {
	// BaseURL is the public HTTPS base the tarball URLs join onto, e.g.
	// https://registry.emisar.dev. Required.
	BaseURL string
	// RepoURL is the source repo for source_url links. Defaults to
	// DefaultRepoURL when empty.
	RepoURL string
	// Previous, when non-nil, is the currently-published catalog. Build
	// fails if any pack changed bytes for an already-published id+version
	// (the "preserve every version/hash" guarantee) — bump the version to
	// publish new bytes. Previous is also the source of each pack's carried-
	// forward previous_versions history, and the published retired_below
	// floor the build enforces monotonicity against.
	Previous *Catalog
}

BuildOptions parameterizes catalog construction.

type Catalog 

type Catalog struct {
	SchemaVersion int    `json:"schema_version"`
	Packs         []Pack `json:"packs"`
}

Catalog is the full published catalog.json document.

func Build 

func Build(reg *packs.Registry, opts BuildOptions) (*Catalog, error)

Build turns a loaded pack registry into a Catalog.

func (*Catalog) Suggest 

func (c *Catalog) Suggest() SuggestIndex

Suggest derives the suggest.json index from the catalog.

type Command 

type Command struct {
	Binary string   `json:"binary"`
	Argv   []string `json:"argv"`
}

Command is an exec action's binary + argv template (placeholders intact), nil for a script-kind action (no single-line invocation to preview).

type Detect 

type Detect struct {
	Binaries  []string `json:"binaries"`
	Processes []string `json:"processes"`
	Ports     []int    `json:"ports"`
}

Detect is the pack-authored service-presence signal used by `emisar pack suggest`. Runtime requirements never become discovery evidence.

type Example 

type Example = actionspec.ModelExample

type Manifest 

type Manifest struct {
	SchemaVersion int      `json:"schema_version"`
	CatalogHash   string   `json:"catalog_hash"`
	Objects       []Object `json:"objects"`
}

Manifest describes a built artifact tree. It is written to the tree root as manifest.json (outside v1/, so it is not itself published) and read by the publish step.

func Write 

func Write(reg *packs.Registry, cat *Catalog, outDir string) (*Manifest, error)

Write lays out the full artifact tree for cat under outDir and returns its manifest. reg supplies the on-disk pack roots the tarballs are built from; it must be the same registry cat was built from.

type Object 

type Object struct {
	// Path is the object path relative to the bucket root, e.g.
	// v1/catalog.json or v1/packs/<id>/<version>/<hex>/pack.tar.gz.
	Path string `json:"path"`
	// Immutable objects use non-colliding paths: content hashes for pack and
	// catalog artifacts, explicit suite versions for schemas. They upload with
	// an if-generation-match:0 precondition (never overwritten). Mutable
	// pointers (the latest catalog.json/suggest.json) are overwritten, relying
	// on bucket versioning to retain prior generations.
	Immutable   bool   `json:"immutable"`
	ContentType string `json:"content_type"`
	// ContentEncoding is the transport encoding the object uploads with
	// ("gzip" for JSON objects). Size and SHA256 always describe the PLAIN
	// bytes — content addressing and post-publish verification compare what
	// consumers actually read, since GCS serves encoded objects decompressed
	// (transcoding for plain clients, transparent decompression for gzip-
	// accepting ones).
	ContentEncoding string `json:"content_encoding,omitempty"`
	Size            int    `json:"size"`
	SHA256          string `json:"sha256"`
}

Object is one published artifact in the built tree, described so the publish step knows how to upload it without re-deriving intent.

type Pack 

type Pack struct {
	ID          string   `json:"id"`
	Name        string   `json:"name"`
	Version     string   `json:"version"`
	Description string   `json:"description"`
	Vendor      string   `json:"vendor"`
	Homepage    string   `json:"homepage"`
	SourceURL   string   `json:"source_url"`
	ContentHash string   `json:"content_hash"`
	TarballURL  string   `json:"tarball_url"`
	Requires    Requires `json:"requires"`
	Detect      Detect   `json:"detect"`
	Actions     []Action `json:"actions"`
	// PreviousVersions carries the last few prior versions of this pack
	// (newest first, excluding the current version, capped at
	// DefaultPreviousKept). Absent on a pack with no shipped history; the
	// portal trust window auto-trusts these alongside the current version.
	PreviousVersions []PreviousVersion `json:"previous_versions,omitempty"`
	// RetiredBelow, when set, retires every version of this pack STRICTLY
	// below it: a runner still advertising such a version is untrusted at
	// dispatch until the operator updates the pack. Absent = nothing retired.
	RetiredBelow string `json:"retired_below,omitempty"`
}

Pack is one catalog entry. It carries everything the portal pack pages, install snippets, command preview, and suggest index need without the pack bytes themselves — those live in the immutable tarball at TarballURL.

type PreviousVersion 

type PreviousVersion struct {
	Version     string   `json:"version"`
	ContentHash string   `json:"content_hash"`
	TarballURL  string   `json:"tarball_url"`
	Actions     []Action `json:"actions,omitempty"`
}

PreviousVersion is one carried-forward prior release of a pack — enough to resolve its immutable tarball and to seed the portal trust window. The tarball it points at was published under a past build and is immutable.

type PublishOptions 

type PublishOptions struct {
	// Bucket is the target GCS bucket (e.g. emisar-pack-registry).
	Bucket string
	// Token is the OAuth2 access token uploads authenticate with. Required
	// unless DryRun. In CI it comes from Workload Identity; locally from
	// `gcloud auth print-access-token`.
	Token string
	// Endpoint defaults to DefaultGCSEndpoint.
	Endpoint string
	// HTTPClient defaults to a 60s-timeout client.
	HTTPClient *http.Client
	// DryRun logs what would be uploaded without contacting GCS.
	DryRun bool
	// Logf receives one progress line per object. Defaults to no-op.
	Logf func(format string, args ...any)
}

PublishOptions parameterizes uploading a built tree to a GCS bucket.

type PublishResult 

type PublishResult struct {
	Uploaded []string
	// Skipped are immutable objects that already existed at their content-
	// addressed path and whose stored bytes were fetched and verified.
	Skipped []string
}

PublishResult summarizes a publish run.

func Publish 

func Publish(ctx context.Context, dir string, opts PublishOptions) (*PublishResult, error)

Publish uploads the artifact tree at dir (as described by its manifest.json) to opts.Bucket. Immutable objects are uploaded with an if-generation-match:0 precondition so an existing object is never overwritten; after a precondition failure the stored bytes must match before the object is skipped. Mutable pointers are overwritten (the bucket's object versioning retains prior generations).

type Requires 

type Requires struct {
	OS       []string `json:"os"`
	Binaries []string `json:"binaries"`
}

Requires mirrors the pack's declared host requirements.

type SuggestIndex 

type SuggestIndex struct {
	Packs []SuggestPack `json:"packs"`
}

SuggestIndex is the lean per-pack index `emisar pack suggest` matches against: id, name, OS, and the derived detect signal. Packs whose detect is all-empty are omitted — with no signal there is nothing to suggest them on. Mirrors EmisarWeb.PacksRegistry.suggest_index.

type SuggestPack 

type SuggestPack struct {
	ID     string   `json:"id"`
	Name   string   `json:"name"`
	OS     []string `json:"os"`
	Detect Detect   `json:"detect"`
}

SuggestPack is one suggest.json entry.

type Validation 

type Validation = actionspec.ModelValidation

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.