harness

package module
v0.1.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 24, 2026 License: MIT Imports: 17 Imported by: 0

README

harness

harness is a Go library for driving AI coding CLIs in headless mode. It supports Claude Code, Codex, GitHub Copilot CLI, and OpenCode through one interface while leaving process placement to the caller. A command can run on the host, inside a container, or through a remote runner with the same arguments and event parser.

Supported backends

Name Binary Credential environment Staged skill path Project instructions Model API hosts
claude claude ANTHROPIC_API_KEY, CLAUDE_CODE_OAUTH_TOKEN .claude/skills/<name> CLAUDE.md *.anthropic.com
codex codex CODEX_API_KEY skills/<name> AGENTS.md api.openai.com, auth0.openai.com, chatgpt.com
copilot copilot COPILOT_GITHUB_TOKEN, GH_TOKEN, GITHUB_TOKEN .github/skills/<name> .github/copilot-instructions.md github.com, api.github.com, api.mcp.github.com, *.githubcopilot.com
opencode opencode OPENAI_API_KEY, ANTHROPIC_API_KEY, OPENCODE_CONFIG_CONTENT, OPENCODE_AUTH_CONTENT .opencode/skill/<name> AGENTS.md models.dev, api.openai.com, *.anthropic.com

The Copilot parser targets CLI 1.0.75 or later, where --output-format json emits JSONL in prompt mode.

The library owns the details that differ between CLIs: binary names, arguments, credential and state environment variables, project instruction files, skill directories, model API hosts, JSONL parsing, account-limit errors, default models, and token prices.

Install

go get github.com/alpha-omega-security/harness

Go 1.26 or later is required.

Core API

A Job contains resolved values for one invocation. Callers apply their own configuration defaults before constructing it.

type Job struct {
    Workspace string
    SrcDir    string
    SkillName string

    Prompt       string
    SystemPrompt string

    Model    string
    Effort   string
    MaxTurns int

    OutputFile  string
    AllowedTools string
    BaseURL      string

    ResumeSessionID string
    ResumePrompt    string
}

Workspace is the command's working directory. SrcDir is the repository directory relative to it and defaults to src; set it to . when Workspace is already the repository root. SkillName selects a staged SKILL.md; when Prompt is empty, the backend builds a short activation prompt. SystemPrompt uses --system-prompt with Claude and the backend's project instruction file for the other CLIs.

MaxTurns uses the backend default when set to zero. Effort and AllowedTools currently apply only to Claude. ResumeSessionID and ResumePrompt continue an existing conversation.

The Harness interface exposes the parts needed by local, container, and remote runners:

type Harness interface {
    Binary() string
    Args(Job) []string
    Prompt(Job) string
    ParseStream(io.Reader, func(Event))
    SkillDir(workspace, name string) string
    GuideFilename() string
    SystemPromptViaArgs() bool
    EgressHosts() []string
    Env(baseURL string) []string
    StateEnv(dir string) []string
    AccountErrorText(string) string
    DefaultModels() []ModelDefault
}

Use ByName to select a backend. An empty name selects Claude.

h, err := harness.ByName("codex")
name := harness.Name(h)
available := harness.Names() // "claude, codex, copilot, opencode"

Each parser produces the same event type:

type Event struct {
    Kind      string
    Tool      string
    Text      string
    CostUSD   float64
    Turns     int
    Usage     Usage
    SessionID string
    RateLimit *RateLimitInfo
}

Kinds are thinking, text, tool, result, error, session, and rate_limit. FormatEvent renders an event for a plain-text log. CostFromUsage calculates a list-price estimate when the CLI reports tokens without a dollar amount.

Run a local subprocess

Run starts the selected binary in the workspace, applies its environment, writes a project instruction file when needed, and parses combined output as it arrives.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/alpha-omega-security/harness"
    "github.com/alpha-omega-security/harness/egress"
    "github.com/alpha-omega-security/harness/skills"
)

func main() {
    ctx := context.Background()
    workspace := "/work/project"

    h, err := harness.ByName("claude")
    if err != nil {
        log.Fatal(err)
    }
    instructions, err := skills.Parse("/work/instructions/review.md")
    if err != nil {
        log.Fatal(err)
    }

    job := harness.Job{
        Workspace:    workspace,
        SrcDir:       ".",
        Prompt:       "Review this project for security defects.",
        SystemPrompt: skills.Concat(instructions),
        Model:        "claude-sonnet-4-6",
        MaxTurns:     20,
    }

    if err := egress.WriteSandboxSettings(workspace, h.EgressHosts()); err != nil {
        log.Fatal(err)
    }
    err = harness.Run(ctx, h, job, func(event harness.Event) {
        fmt.Println(harness.FormatEvent(event))
    })
    if err != nil {
        log.Fatal(err)
    }
}

When a non-zero exit contains a provider account-limit message, Run returns an *harness.AccountError. Its optional reset time can be used to schedule a later retry.

Use another process runner

Callers that own process creation can use the same API without Run. This is useful for containers, job queues, and remote execution.

h, err := harness.ByName("codex")
if err != nil {
    return err
}
skill, err := skills.Parse("/skills/security-review/SKILL.md")
if err != nil {
    return err
}
job := harness.Job{
    Workspace:  "/work",
    SrcDir:     ".",
    SkillName:  skill.Name,
    Model:      "gpt-5.3-codex",
    OutputFile: "report.json",
}
if err := skills.Stage(h, job, skill); err != nil {
    return err
}
if err := harness.WriteSystemPrompt(h, job); err != nil {
    return err
}
argv := append([]string{h.Binary()}, h.Args(job)...)
env := append(h.Env(job.BaseURL), h.StateEnv("/state")...)

// Pass argv and env to the process runner. Entries such as "CODEX_API_KEY"
// use the `docker run -e KEY` passthrough form.
stdout, err := startContainer(ctx, argv, env)
if err != nil {
    return err
}
h.ParseStream(stdout, func(event harness.Event) {
    fmt.Println(harness.FormatEvent(event))
})

WriteSystemPrompt is needed only when SystemPrompt is non-empty. Backends whose SystemPromptViaArgs method returns true receive that value in their arguments, so the helper does not write a guide file.

Process isolation

The generated arguments allow unattended tool use. Claude uses bypassPermissions unless AllowedTools is set. Codex uses danger-full-access, OpenCode uses --auto, and Copilot uses --allow-all. Run these commands only in a workspace and execution environment where those permissions are acceptable. Container and remote callers should apply their own filesystem, process, secret, and network limits.

The egress package can restrict outbound HTTP and HTTPS by hostname. Its proxy checks the resolved destination immediately before connecting and rejects loopback, private, link-local, carrier-grade NAT, unspecified, and multicast addresses. This closes the usual DNS rebinding path after a hostname has passed the allowlist.

Packages

skills parses SKILL.md files following the Agent Skills specification. YAML frontmatter is optional, so plain markdown instruction files parse too. Parse returns a Skill with the spec fields (name, description, license, compatibility, allowed-tools, metadata), the body, a sibling schema.json when present, and a content hash covering both. Walk finds skills under a directory; Stage writes one into the selected backend's discovery directory; Concat joins bodies for a system prompt; Render produces SKILL.md bytes from a Skill built in memory. ValidateNamespace and the Match/PathIncluded glob helpers support callers that add their own metadata keys and path filters.

egress contains the authenticated allowlist proxy and WriteSandboxSettings, which writes Claude's .claude/settings.json domain allowlist.

llm sends a single schema-constrained request to the Anthropic Messages API. It accepts a caller-owned HTTP client and permits plain HTTP only for local development endpoints.

License

MIT

Documentation

Overview

Package harness drives AI coding command-line tools through a common API.

Index

Constants

View Source 
const (
	KindThinking  = "thinking"
	KindText      = "text"
	KindTool      = "tool"
	KindResult    = "result"
	KindError     = "error"
	KindSession   = "session"
	KindRateLimit = "rate_limit"
)
View Source 
const DefaultMaxTurns = 30

Variables

This section is empty.

Functions

func AccountErrorResumable 

func AccountErrorResumable(s string) bool

AccountErrorResumable reports whether an account error describes a transient limit. Revoked access wins when both permanent and transient phrases appear.

func CostFromUsage 

func CostFromUsage(model string, usage Usage) float64

CostFromUsage calculates a result event's list-price cost. It returns zero for an unknown model rather than presenting an incorrect estimate.

InputTokens includes all prompt tokens. CacheReadTokens is a discounted subset. CacheWriteTokens is separate only for models with a dedicated write rate; it remains ordinary input when CacheWrite is zero.

func DirectivePaths added in v0.1.5 

func DirectivePaths() (dirs, files []string)

DirectivePaths returns directory and file basename patterns that agent CLIs automatically load as project instructions or configuration. Matching is case-insensitive and follows path.Match semantics. The returned slices are copies and may be modified by the caller.

func FormatEvent 

func FormatEvent(e Event) string

FormatEvent renders an event as one plain-text log line.

func Name 

func Name(h Harness) string

Name returns the registered name of h. Comparing concrete types avoids an interface equality panic if a future implementation contains a slice or map. An unregistered implementation falls back to its binary name.

func Names 

func Names() string

Names returns the registered backend names in lexical order, excluding the empty default alias.

func PreferAccountErrorText 

func PreferAccountErrorText(current, candidate string) string

PreferAccountErrorText keeps the first account error unless a later message identifies revoked access. This prevents a permanent failure from being scheduled for retry because an earlier line happened to describe a limit.

func ResumableReset 

func ResumableReset(errText string, limit *RateLimitInfo) *time.Time

ResumableReset returns a rejected limit's reset time only when the associated account error is transient. Revoked access always requires manual action.

func Run 

func Run(ctx context.Context, h Harness, j Job, emit func(Event)) error

Run starts h as a local subprocess in j.Workspace and streams parsed events to emit.

func StripDirectives added in v0.1.5 

func StripDirectives(root string) (int, error)

StripDirectives removes files and directories below root whose basenames match DirectivePaths. A removed directory counts as one item regardless of its contents. The .git subtree is skipped. A missing root is a no-op.

func WriteSystemPrompt 

func WriteSystemPrompt(h Harness, j Job) error

WriteSystemPrompt writes j.SystemPrompt to the guide file used by h. A backend that passes the system prompt in Args has no file to write.

Types

type AccountError 

type AccountError struct {
	Detail  string
	ResetAt *time.Time
}

AccountError reports a provider-level account problem for which immediately retrying the command is unlikely to help.

func (*AccountError) Error 

func (e *AccountError) Error() string

type ClaudeHarness 

type ClaudeHarness struct{}

ClaudeHarness drives Claude Code in print mode. It is the default backend because the original caller used Claude before the shared interface existed.

func (ClaudeHarness) AccountErrorText 

func (ClaudeHarness) AccountErrorText(s string) string

func (ClaudeHarness) Args 

func (ClaudeHarness) Args(j Job) []string

Args builds the claude -p invocation. An allowed-tools list uses acceptEdits only when the job has a legitimate output file; otherwise the default permission mode keeps the requested read-only boundary intact.

func (ClaudeHarness) Binary 

func (ClaudeHarness) Binary() string

func (ClaudeHarness) DefaultModels 

func (ClaudeHarness) DefaultModels() []ModelDefault

func (ClaudeHarness) EgressHosts 

func (ClaudeHarness) EgressHosts() []string

func (ClaudeHarness) Env 

func (ClaudeHarness) Env(baseURL string) []string

func (ClaudeHarness) GuideFilename 

func (ClaudeHarness) GuideFilename() string

func (ClaudeHarness) ParseStream 

func (ClaudeHarness) ParseStream(r io.Reader, emit func(Event))

ParseStream reads Claude's stream-json output. scanJSONL uses a buffered reader rather than Scanner so an oversized thinking or tool-result line cannot discard the later result event that carries usage and turn counts.

func (ClaudeHarness) Prompt 

func (ClaudeHarness) Prompt(j Job) string

func (ClaudeHarness) SkillDir 

func (ClaudeHarness) SkillDir(workspace, name string) string

func (ClaudeHarness) StateEnv 

func (ClaudeHarness) StateEnv(dir string) []string

func (ClaudeHarness) SystemPromptViaArgs 

func (ClaudeHarness) SystemPromptViaArgs() bool

type CodexHarness 

type CodexHarness struct{}

CodexHarness drives Codex in headless exec mode. Codex reads AGENTS.md as project guidance and stores resumable threads under CODEX_HOME.

func (CodexHarness) AccountErrorText 

func (CodexHarness) AccountErrorText(s string) string

func (CodexHarness) Args 

func (CodexHarness) Args(j Job) []string

Args builds codex exec argv. Headless Codex has no slash-style skill invocation, so Prompt names the staged SKILL.md. Resume inserts the thread id after "exec resume". Codex has no per-turn cap, so Job.MaxTurns is intentionally ignored.

func (CodexHarness) Binary 

func (CodexHarness) Binary() string

func (CodexHarness) DefaultModels 

func (CodexHarness) DefaultModels() []ModelDefault

func (CodexHarness) EgressHosts 

func (CodexHarness) EgressHosts() []string

func (CodexHarness) Env 

func (CodexHarness) Env(_ string) []string

func (CodexHarness) GuideFilename 

func (CodexHarness) GuideFilename() string

func (CodexHarness) ParseStream 

func (CodexHarness) ParseStream(r io.Reader, emit func(Event))

ParseStream maps codex exec --json output onto backend-neutral events. Session announcements enable resume, item completions carry text and tools, and unknown or non-JSON lines pass through as text. Codex has no max-turns event because its exec command has no turn cap.

func (CodexHarness) Prompt 

func (CodexHarness) Prompt(j Job) string

func (CodexHarness) SkillDir 

func (CodexHarness) SkillDir(workspace, name string) string

func (CodexHarness) StateEnv 

func (CodexHarness) StateEnv(dir string) []string

func (CodexHarness) SystemPromptViaArgs 

func (CodexHarness) SystemPromptViaArgs() bool

type CopilotHarness 

type CopilotHarness struct{}

CopilotHarness drives GitHub Copilot CLI in non-interactive prompt mode. Its arguments and JSONL mapping are based on Copilot CLI 1.0.75.

func (CopilotHarness) AccountErrorText 

func (CopilotHarness) AccountErrorText(s string) string

func (CopilotHarness) Args 

func (CopilotHarness) Args(j Job) []string

Args enables autopilot and tool use without interactive confirmation. The caller must isolate the process and workspace because --allow-all grants the CLI every tool it exposes.

func (CopilotHarness) Binary 

func (CopilotHarness) Binary() string

func (CopilotHarness) DefaultModels 

func (CopilotHarness) DefaultModels() []ModelDefault

func (CopilotHarness) EgressHosts 

func (CopilotHarness) EgressHosts() []string

func (CopilotHarness) Env 

func (CopilotHarness) Env(baseURL string) []string

func (CopilotHarness) GuideFilename 

func (CopilotHarness) GuideFilename() string

func (CopilotHarness) ParseStream 

func (CopilotHarness) ParseStream(r io.Reader, emit func(Event))

ParseStream combines per-call assistant.usage and per-turn assistant.turn_end records into one result emitted after the stream ends. This gives callers the same single-run totals exposed by the other backends and prevents consumers that retain only the last result from losing usage.

func (CopilotHarness) Prompt 

func (CopilotHarness) Prompt(j Job) string

func (CopilotHarness) SkillDir 

func (CopilotHarness) SkillDir(workspace, name string) string

func (CopilotHarness) StateEnv 

func (CopilotHarness) StateEnv(dir string) []string

func (CopilotHarness) SystemPromptViaArgs 

func (CopilotHarness) SystemPromptViaArgs() bool

type Event 

type Event struct {
	Kind      string
	Tool      string
	Text      string
	CostUSD   float64
	Turns     int
	Usage     Usage
	SessionID string
	RateLimit *RateLimitInfo
}

Event is one backend-neutral item from an agent's output stream. Tool, CostUSD, Turns, Usage, SessionID, and RateLimit are populated only for their corresponding Kind.

type Harness 

type Harness interface {
	// Binary is the executable name expected on PATH.
	Binary() string
	// Args returns argv without the binary for one job.
	Args(Job) []string
	// Prompt returns the final user prompt passed by Args.
	Prompt(Job) string
	// ParseStream maps the backend's combined output onto Event values.
	ParseStream(io.Reader, func(Event))
	// SkillDir returns the directory where the backend discovers a staged
	// SKILL.md and its sibling files.
	SkillDir(workspace, name string) string
	// GuideFilename is the workspace-relative project instruction file loaded
	// by the backend.
	GuideFilename() string
	// SystemPromptViaArgs reports whether Args passes Job.SystemPrompt itself.
	// When false, WriteSystemPrompt writes it to GuideFilename instead.
	SystemPromptViaArgs() bool
	// EgressHosts lists the model and authentication hosts needed by the
	// backend, using "*.example.com" for wildcard suffixes.
	EgressHosts() []string
	// Env returns backend environment entries. A bare key asks a process
	// runner to pass through the caller's value.
	Env(baseURL string) []string
	// StateEnv points the backend at a caller-owned persistent state directory
	// so a later process can resume the same session.
	StateEnv(dir string) []string
	// AccountErrorText returns the matching provider account error, or an empty
	// string. Callers should consult it only after a non-zero process exit so
	// ordinary model text cannot pause retries.
	AccountErrorText(string) string
	// DefaultModels returns the built-in model picker entries. The first entry
	// is the backend default.
	DefaultModels() []ModelDefault
}

Harness describes the CLI-specific parts of an agent invocation. It owns the binary, arguments, stream format, project guide, skill discovery, provider environment, and default models. Process placement and isolation remain the caller's responsibility.

func ByName 

func ByName(name string) (Harness, error)

ByName resolves a backend name. The empty name selects Claude.

type Job 

type Job struct {
	// Workspace is the command's working directory. Paths passed to a CLI are
	// relative to it.
	Workspace string

	// SrcDir is the workspace-relative repository directory used in generated
	// prompts. It defaults to "src"; use "." when Workspace is the repository
	// root.
	SrcDir string

	// SkillName selects a staged SKILL.md directory. An empty value means no
	// staged skill.
	SkillName string

	// Prompt is the user turn for a fresh run. When it is empty and SkillName
	// is set, the harness builds an activation prompt.
	Prompt string

	// SystemPrompt supplies additional instructions. Claude receives it via
	// --system-prompt. Run writes it to the project guide file used by the
	// other backends.
	SystemPrompt string

	Model    string
	Effort   string
	MaxTurns int

	// OutputFile is a workspace-relative path the skill should write. It is
	// empty for free-form runs.
	OutputFile string

	// ValidationHint is appended to generated prompts after the OutputFile
	// clause when OutputFile ends in .json. It lets a caller tell the agent
	// how to validate its output (an API endpoint, a staged validator) in
	// terms of the caller's own context. When empty, a generic instruction
	// to check against ./schema.json is used.
	ValidationHint string

	// AllowedTools is Claude's comma-separated tool allowlist. Other backends
	// leave tool restrictions to their caller's sandbox.
	AllowedTools string

	// BaseURL overrides the model API endpoint where the backend supports it.
	BaseURL string

	// ResumeSessionID continues a prior conversation. ResumePrompt is the
	// corrective turn used for the resumed invocation.
	ResumeSessionID string
	ResumePrompt    string
}

Job contains the resolved inputs for one CLI invocation.

type ModelDefault 

type ModelDefault struct {
	Name string
	ID   string
	Tier string
}

ModelDefault is one model offered by a backend. Tier is "mid", "high", "max", or empty when the model is selectable but not a tier default.

type OpencodeHarness 

type OpencodeHarness struct{}

OpencodeHarness drives OpenCode in headless run mode. OpenCode is provider-neutral, so its environment and egress list cover Anthropic, OpenAI, and OpenCode's model registry by default.

func (OpencodeHarness) AccountErrorText 

func (OpencodeHarness) AccountErrorText(s string) string

func (OpencodeHarness) Args 

func (OpencodeHarness) Args(j Job) []string

Args builds opencode run argv. OpenCode discovers SKILL.md but does not invoke a named skill itself, so Prompt points at the staged file. --auto suppresses interactive permission prompts and --format json selects JSONL. The caller remains responsible for process isolation.

func (OpencodeHarness) Binary 

func (OpencodeHarness) Binary() string

func (OpencodeHarness) DefaultModels 

func (OpencodeHarness) DefaultModels() []ModelDefault

func (OpencodeHarness) EgressHosts 

func (OpencodeHarness) EgressHosts() []string

func (OpencodeHarness) Env 

func (OpencodeHarness) Env(_ string) []string

Env ignores baseURL because OpenCode has no single provider endpoint. A caller can configure each provider through OPENCODE_CONFIG_CONTENT.

func (OpencodeHarness) GuideFilename 

func (OpencodeHarness) GuideFilename() string

func (OpencodeHarness) ParseStream 

func (OpencodeHarness) ParseStream(r io.Reader, emit func(Event))

func (OpencodeHarness) Prompt 

func (OpencodeHarness) Prompt(j Job) string

func (OpencodeHarness) SkillDir 

func (OpencodeHarness) SkillDir(workspace, name string) string

func (OpencodeHarness) StateEnv 

func (OpencodeHarness) StateEnv(dir string) []string

func (OpencodeHarness) SystemPromptViaArgs 

func (OpencodeHarness) SystemPromptViaArgs() bool

type RateLimitInfo 

type RateLimitInfo struct {
	Status         string `json:"status"`
	OverageStatus  string `json:"overageStatus"`
	IsUsingOverage bool   `json:"isUsingOverage"`
	ResetsAt       int64  `json:"resetsAt"`
	Type           string `json:"rateLimitType"`
}

RateLimitInfo contains subscription limit status reported by a backend.

func PreferRateLimitReset 

func PreferRateLimitReset(current, candidate *RateLimitInfo) *RateLimitInfo

PreferRateLimitReset returns the rejected rate limit with the later reset so a retry is not scheduled while another reported window still blocks use.

func (*RateLimitInfo) Rejected 

func (r *RateLimitInfo) Rejected() bool

Rejected reports whether the limit currently blocks requests.

func (*RateLimitInfo) ResetTime 

func (r *RateLimitInfo) ResetTime() *time.Time

ResetTime converts ResetsAt to UTC. It returns nil for an absent or invalid reset time.

type Usage 

type Usage struct {
	InputTokens      int `json:"input_tokens"`
	OutputTokens     int `json:"output_tokens"`
	CacheReadTokens  int `json:"cache_read_input_tokens"`
	CacheWriteTokens int `json:"cache_creation_input_tokens"`
}

Usage is a result event's token accounting.

Source Files

View all Source files

Directories

Path Synopsis
Package egress provides an authenticated outbound proxy with a hostname allowlist and a helper for Claude workspace sandbox settings.
 Package egress provides an authenticated outbound proxy with a hostname allowlist and a helper for Claude workspace sandbox settings.
Package llm provides small, structured one-shot model calls for work that does not need a full agent workspace or tool loop.
 Package llm provides small, structured one-shot model calls for work that does not need a full agent workspace or tool loop.
Package skills parses, filters, and stages agent skill directories.
 Package skills parses, filters, and stages agent skill directories.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.