samples

Documentation

Overview

Package samples implements the local half of the public-sample pipeline (goal.md §9, plan C13): canonical content-addressed artifacts, leakage scanning, sanitized clean-room specs, and the create workflow. Nothing in this package ever transmits data; publishing is a separate, explicit step.

Index

• Constants
• Variables
• func BuildArtifact(dir string) (tgz []byte, sampleID string, err error)
• func NewProposalWorkspace(home string, spec SanitizedSpec, fp domain.EnvironmentFingerprint) (string, error)
• func ProposalManifest(spec SanitizedSpec, fp domain.EnvironmentFingerprint) domain.SampleManifest
• func Unpack(tgz []byte, destDir string) error
• func VerifyProposalWorkspace(dir string) error
• func WorkspaceBase(home string) string
• type Created
  • func CreateFromDir(ctx context.Context, dir string, manifest domain.SampleManifest) (*Created, error)
• type Finding
  • func Scan(dir string, opts ScanOptions) ([]Finding, error)
• type SanitizedSpec
  • func BuildSpec(res ScanInputs) SanitizedSpec
  • func (s SanitizedSpec) PromptText() string
• type ScanInputs
• type ScanOptions
  • func ProvenanceOptions(dir string) ScanOptions
• type TextFile
  • func ReadTextFiles(tgz []byte) ([]TextFile, error)

Constants

const (
	// MaxCompressedBytes caps the canonical tar.gz size.
	MaxCompressedBytes = 256 * 1024
	// MaxFiles caps the number of regular files in a sample.
	MaxFiles = 200
)

Artifact limits (goal.md §16.4, plan C13).

const (
	KindAWSKey        = "AWS_ACCESS_KEY"
	KindGitHubToken   = "GITHUB_TOKEN"
	KindAPIKey        = "API_KEY"
	KindGoogleKey     = "GOOGLE_API_KEY"
	KindPrivateKey    = "PRIVATE_KEY"
	KindEmail         = "EMAIL"
	KindAbsolutePath  = "ABSOLUTE_PATH"
	KindURL           = "URL"
	KindEnvAssignment = "ENV_ASSIGNMENT"
	KindProjectName   = "PROJECT_NAME"
	// KindUnscanned marks a file the scanner could not read or was too
	// large to read. It is a finding rather than a silence on purpose: the
	// publish gate refuses on findings, and "we did not look" must not be
	// reported as "there is nothing there".
	KindUnscanned = "UNSCANNED"
)

Finding kinds.

const DefaultLicense = "MIT-0"

DefaultLicense is the default public-sample license (goal.md §7.5).

const MaxViewBytes = 64 * 1024

MaxViewBytes bounds how much of one file a reader is shown in a browser.

It is a display bound, not a safety one: the safety caps are the artifact's own (MaxCompressedBytes, MaxFiles, maxUnpackedBytes) and they already hold before a byte of this is reached. 64 KiB is more source than anyone reads on a page and small enough that a sample full of large fixtures cannot turn one page render into megabytes of HTML.

const WorkspaceIdentityFile = "csx-workspace.json"

WorkspaceIdentityFile is what `csx sample create` writes into the directory to record which sample it became (CodeSampleX-Farm#12). It is named here, beside the rule that keeps it out of artifacts, so the writer and the packer cannot drift apart.

const WorkspaceTempPrefix = ".csx-workspace-"

WorkspaceTempPrefix is the partial file that write goes through before it is renamed into place.

It exists only between two syscalls, but a create killed in that window leaves one behind, and it lands in the directory that is about to be packed -- so the NEXT create would fold it into the artifact and move the sample id of a tree nobody edited. That is the defect this whole list exists to prevent, so the temporary name is covered too rather than trusted to be short-lived.

Variables

var ErrScaffold = errors.New("clean-room scaffold")

ErrScaffold marks every failure to build a clean-room workspace. It exists so a caller can tell "your packages were not purls" apart from "this machine could not create the workspace" — two failures an agent must react to differently, and which used to arrive as the same opaque string.

Functions

func BuildArtifact

func BuildArtifact(dir string) (tgz []byte, sampleID string, err error)

BuildArtifact renders dir as the canonical sample artifact: files sorted by slash-path, tar mode 0644, uid/gid 0, mtime Unix epoch 0, USTAR only (no PAX or user headers), gzip with zero MTime and no name. The returned sampleID is domain.SHA256Hex over the tar.gz bytes, so identical trees always produce identical IDs regardless of file timestamps.

func NewProposalWorkspace

func NewProposalWorkspace(home string, spec SanitizedSpec, fp domain.EnvironmentFingerprint) (string, error)

NewProposalWorkspace creates the clean-room workspace for one proposal and fills it with the scaffold the generation instructions promise: spec.json, PROMPT.md and the csx.json manifest with an empty contract for the agent to complete.

It returns only workspaces that exist and are complete. The files are written under a staging name and the directory is renamed into place, so the returned path is either a fully scaffolded workspace or the call failed — there is no third outcome, and in particular there is no successful call that returns an empty directory.

Retry policy: an identical proposal whose workspace is still untouched — the three scaffold files and nothing else — is handed back rather than duplicated, so an agent retrying the same call does not leave a new directory behind each time.

func ProposalManifest

func ProposalManifest(spec SanitizedSpec, fp domain.EnvironmentFingerprint) domain.SampleManifest

ProposalManifest builds the csx.json scaffold for a proposal: the case facts the agent must preserve, and the environment, commands and adapter it is asked to correct for the files it actually generates. Case.Contract is deliberately empty — the assertions are the agent's work.

func Unpack

func Unpack(tgz []byte, destDir string) error

Unpack extracts a sample artifact into destDir with the same safety posture as BuildArtifact: clean relative paths only, no traversal or absolute names, forbidden entries rejected, compressed and decompressed size caps, and no write can land outside destDir.

func VerifyProposalWorkspace

func VerifyProposalWorkspace(dir string) error

VerifyProposalWorkspace reports whether dir really holds a usable clean-room scaffold. It is the check standing between a scaffold and any message telling an agent that a scaffold is there.

func WorkspaceBase

func WorkspaceBase(home string) string

WorkspaceBase is the directory holding every clean-room workspace.

Types

type Created

type Created struct {
	SampleID string
	Artifact []byte
	Findings []Finding
	MaxLevel domain.VerificationLevel
	Manifest domain.SampleManifest
}

Created is the result of turning a clean-room directory into a local sample. Findings are advisory here: they block PUBLISH, never creation (the user still gets to preview and fix). MaxLevel records the honest verification ceiling — a contract-less sample can never claim more than L2_COMPILED (goal.md §7.5).

func CreateFromDir

func CreateFromDir(ctx context.Context, dir string, manifest domain.SampleManifest) (*Created, error)

CreateFromDir validates the manifest, writes it as csx.json into dir, runs the leakage scan, and builds the canonical artifact. Nothing is transmitted; the caller decides about publishing after preview.

type Finding

type Finding struct {
	File    string `json:"file"` // slash-relative path inside the sample
	Line    int    `json:"line"` // 1-based
	Kind    string `json:"kind"`
	Excerpt string `json:"excerpt"`
}

Finding is one potential leak in a sample tree. Findings never block sample creation, but they block publish until reviewed (goal.md §9.3).

func Scan

func Scan(dir string, opts ScanOptions) ([]Finding, error)

Scan walks the sample tree at dir and returns every potential leak. It is advisory at creation time and blocking at publish time.

type SanitizedSpec

type SanitizedSpec struct {
	SchemaVersion     int               `json:"schemaVersion"`
	Goal              string            `json:"goal"`
	Kind              string            `json:"kind,omitempty"`
	Packages          []string          `json:"packages"`
	Symbols           []string          `json:"symbols,omitempty"`
	Constraints       map[string]string `json:"constraints,omitempty"`
	RuntimeConditions map[string]string `json:"runtimeConditions,omitempty"`
	PassedStages      []string          `json:"passedStages,omitempty"`
	UserDescription   string            `json:"userDescription,omitempty"`
}

SanitizedSpec is the clean-room brief given to the user's local LLM. By construction it can only carry the goal.md §9.2 whitelist: public packages, public symbols, intent, constraints (incl. executionContext), required runtime conditions, passed stages, and the user's description. Nothing project-identifying is representable.

func BuildSpec

func BuildSpec(res ScanInputs) SanitizedSpec

BuildSpec derives a SanitizedSpec from extractor results. Packages and symbols are deduplicated and sorted; runtime conditions come from the environment fingerprint (already free of paths and identifiers); the execution context becomes an explicit constraint.

func (SanitizedSpec) PromptText

func (s SanitizedSpec) PromptText() string

PromptText renders the clean-room generation instructions handed to the user's local LLM. The model starts from an empty directory and works only from this spec.

type ScanInputs

type ScanInputs struct {
	Packages        []string // purls
	Symbols         []string
	Goal            string // intent
	Kind            string // HOW | FIX | MIGRATION | CONFIG
	Constraints     map[string]string
	Environment     domain.EnvironmentFingerprint
	PassedStages    []string // stages that succeeded in the origin project
	UserDescription string
}

ScanInputs is what the local extractor is allowed to hand the spec builder: public facts about the problem, never project source, paths, or names (goal.md §9.2).

type ScanOptions

type ScanOptions struct {
	ProjectDirName    string
	GitRemoteName     string
	ExtraAllowedHosts []string // appended to the URL allowlist
}

ScanOptions parameterizes project-identifying checks. Both names come from the contributing project (its directory name and the repository name of its git remote); a sample mentioning either leaks provenance.

func ProvenanceOptions

func ProvenanceOptions(dir string) ScanOptions

ProvenanceOptions derives the project-identifying names to scan for, from the directory the contributor is working in and its git remote.

These two fields are what the whole KindProjectName check is built on — "a sample mentioning either leaks provenance" — and they were never set. Every call site passed ScanOptions{}, so the check compiled no patterns and matched nothing, at create time and at the publish gate alike. A sample carrying `// part of the acme-billing-core monorepo` published clean, and an employer's internal project name went to a public network.

Names too short or too generic are dropped: someone working in ~/src or ~/app would otherwise have every occurrence of that word flagged, which rejects honest samples to catch nothing.

type TextFile

type TextFile struct {
	Name string
	Body string
	// Truncated says the body is the first MaxViewBytes of a longer file.
	Truncated bool
}

TextFile is one readable file out of a sample artifact.

func ReadTextFiles

func ReadTextFiles(tgz []byte) ([]TextFile, error)

ReadTextFiles returns the readable files of a sample artifact, in the order the archive holds them.

Unpack's sibling, sharing its safety posture — the same compressed cap, the same entry and file limits, the same safeEntryName check that refuses traversal and absolute names — but returning contents instead of writing them, because a page that shows source must not need a directory to put it in first.

A file that is not valid UTF-8, or that carries a NUL, is left out entirely rather than rendered as replacement characters: a sample's fixtures can be binary, and a page of U+FFFD is not source anybody can read or copy. Its name still appears in the file list, which is where a reader learns it exists.