provenance

Documentation

Overview

Package provenance builds per-turn provenance bundles for upload.

Index

• Constants
• func DeriveUploadHash(blob []byte, kind string, repoRoot string) (uploadHash string, redactedBlob []byte, err error)
• func IsTerminal(err error) bool
• func IsUnauthorized(err error) bool
• func PackageTurn(ctx context.Context, repoPath string, tc TurnContext)
• func RedactForUpload(blob []byte, kind string, repoRoot string) ([]byte, error)
• func RegisterEnricher(e TranscriptEnricher)
• func RewriteBundleHashes(bundleBytes []byte, hashMap map[string]string) []byte
• type CompanionEvidence
• type EnrichInput
• type ManifestAction
  • func ClassifyOutcome(err error, uploadAttempts int64) ManifestAction
• type SyncResult
  • func SyncPendingTurns(ctx context.Context, repoPath string, watermarkTs int64, limit int) ([]SyncResult, error)
• type TranscriptEnricher
• type TurnContext
• type UploadOptions
• type UploadResult
  • func SyncAndUpload(ctx context.Context, repoRoot, endpoint, token string, watermarkTs int64, ...) ([]UploadResult, error)
  • func UploadTurn(ctx context.Context, endpoint, token string, result SyncResult) UploadResult

Constants

const MaxUploadAttempts = 5

MaxUploadAttempts is the retry cap for transient upload failures.

const UploadTransformVersion = 1

UploadTransformVersion tracks the current redaction/normalization rules. Increment when the transform output changes for the same input.

Functions

func DeriveUploadHash

func DeriveUploadHash(blob []byte, kind string, repoRoot string) (uploadHash string, redactedBlob []byte, err error)

DeriveUploadHash redacts a blob and computes the content hash of the result. Returns both the upload hash (for object naming and dedup) and the redacted bytes.

func IsTerminal

func IsTerminal(err error) bool

IsTerminal returns true if the error should not be retried (e.g. 409 conflict means the turn is registered under a different identity).

func IsUnauthorized

func IsUnauthorized(err error) bool

IsUnauthorized returns true if the error chain contains a 401 from the backend.

func PackageTurn

func PackageTurn(ctx context.Context, repoPath string, tc TurnContext)

PackageTurn builds the per-turn provenance bundle and persists the manifest row.

func RedactForUpload

func RedactForUpload(blob []byte, kind string, repoRoot string) ([]byte, error)

RedactForUpload transforms a raw CAS blob into an upload-safe artifact. Applies path normalization and secret redaction based on the object kind. The output is deterministic for a given transform version.

Invalid JSON, unknown kinds, and redactor failures return an error without upload bytes.

func RegisterEnricher

func RegisterEnricher(e TranscriptEnricher)

RegisterEnricher adds an enricher to the global registry.

func RewriteBundleHashes

func RewriteBundleHashes(bundleBytes []byte, hashMap map[string]string) []byte

RewriteBundleHashes replaces local CAS hashes embedded in a bundle blob with their corresponding upload hashes. The bundle stores prompt.blob_hash and steps[].provenance_hash as local CAS hashes at packaging time, but uploaded objects use their redacted upload hashes. This rewrite happens before bundle redaction so the uploaded bundle references the same hashes as the uploaded objects.

Uses generic map surgery (not a typed struct) so new step fields added later are preserved automatically without updating this function.

Types

type CompanionEvidence

type CompanionEvidence struct {
	PayloadHash string
	Summary     string
	Role        string // "user" (Claude) or "tool" (Copilot)
	Kind        string // "tool_result"
	Ts          int64
}

CompanionEvidence represents a nearby tool_result event that may complete a step.

type EnrichInput

type EnrichInput struct {
	Provider    string
	ToolName    string
	ToolUseID   string
	TurnID      string
	SessionID   string
	PayloadHash string              // CAS hash of the step event's payload
	Companions  []CompanionEvidence // matching tool_result events, ordered by ts
	BlobStore   *blobs.Store
}

EnrichInput holds the data available to an enricher for a single step.

type ManifestAction

type ManifestAction int

ManifestAction describes what the caller should do with a manifest after upload.

const (
	// ActionUploaded means the upload succeeded.
	ActionUploaded ManifestAction = iota
	// ActionRetry means a transient error occurred below the retry cap.
	ActionRetry
	// ActionFail means a terminal error or retry cap exceeded.
	ActionFail
)

func ClassifyOutcome

func ClassifyOutcome(err error, uploadAttempts int64) ManifestAction

ClassifyOutcome decides the manifest state transition based on the upload result and the manifest's current attempt count.

type SyncResult

type SyncResult struct {
	TurnID         string
	ManifestID     string
	ObjectCount    int
	UploadAttempts int64             // Current attempt count from the manifest row.
	Envelope       []byte            // JSON manifest envelope for registration.
	RedactedBlobs  map[string][]byte // Upload-hash to redacted blob bytes.
	Skipped        bool              // True if a required blob was missing.
}

SyncResult holds the prepared upload artifacts for a single turn.

func SyncPendingTurns

func SyncPendingTurns(ctx context.Context, repoPath string, watermarkTs int64, limit int) ([]SyncResult, error)

SyncPendingTurns prepares upload artifacts for packaged manifests. The local schema includes upload_transform_version and remote_verified_at on provenance_manifests, but the current upload path does not write them yet. Wire these when adding GC or version-aware re-upload logic. The watermarkTs bounds the sync to manifests created at or before this timestamp. Pass 0 to drain all packaged manifests.

type TranscriptEnricher

type TranscriptEnricher interface {
	// CanEnrich returns true if this enricher can handle the given provider
	// and tool name combination.
	CanEnrich(provider, toolName string) bool

	// Enrich synthesizes a provenance blob from available transcript evidence.
	// Returns nil bytes if enrichment is not possible for this step.
	Enrich(ctx context.Context, input EnrichInput) ([]byte, error)
}

TranscriptEnricher synthesizes a provenance blob from transcript evidence when a step event has tool_name but no provenance_hash.

type TurnContext

type TurnContext struct {
	TurnID        string
	SessionID     string // provider session ID
	Provider      string
	TranscriptRef string
	StartedAt     int64
	CompletedAt   int64
	CWD           string
}

TurnContext holds the metadata needed to package a single turn.

type UploadOptions

type UploadOptions struct {
	// OnProgress is called after each manifest is processed. The arguments
	// are (current index starting at 1, total count, result for this item).
	// If nil, no progress is reported.
	OnProgress func(current, total int, result UploadResult)
}

UploadOptions configures optional SyncAndUpload behavior.

type UploadResult

type UploadResult struct {
	TurnID     string
	ManifestID string
	Uploaded   bool
	Action     ManifestAction // What happened: ActionUploaded, ActionRetry, or ActionFail.
	Err        error
}

UploadResult captures the outcome of uploading a single turn.

func SyncAndUpload

func SyncAndUpload(ctx context.Context, repoRoot, endpoint, token string, watermarkTs int64, limit int, opts *UploadOptions) ([]UploadResult, error)

SyncAndUpload prepares packaged manifests and uploads them to the backend. It handles the full cycle: prepare blobs, claim the manifest, upload, and persist the resulting manifest state. It returns one UploadResult per manifest it processed, including local preparation failures.

func UploadTurn

func UploadTurn(ctx context.Context, endpoint, token string, result SyncResult) UploadResult

UploadTurn performs the full upload cycle for a prepared sync result: prepare -> S3 PUT -> complete.