llmproxy

package
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 7, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package llmproxy implements the AgentGuard LLM API Proxy: an HTTP server that speaks OpenAI Chat Completions and Anthropic Messages wire formats, forwards requests to the real upstreams, and gates any tool calls the model emits through the central AgentGuard policy engine.

Phase 4C is split across four workers:

  • A21 (this file's worker) — server skeleton, non-streaming forwarding, protocol type definitions, integration hooks for the rest.
  • A22 — streaming pause/resume/rewrite (the technically deepest piece; SSE parsing for both providers, byte-identity invariant on ALLOW, synthetic refusal on DENY/REQUIRE_APPROVAL).
  • A23 — tool-call → policy-scope mapping (built-in defaults + YAML override).
  • A24 — wires the PolicyCheck hook against /v1/check, builds synthetic refusal payloads, fail-mode handling.

See docs/LLM_API_PROXY.md for the wire format design and docs/PROXY_ARCHITECTURE.md for cross-cutting decisions (audit transport tag, two-binary topology, fail-mode flag parity).

Index

Constants

View Source 
const (
	// DefaultListen binds to loopback by design — non-loopback binds
	// without --proxy-api-key are refused at validation time.
	DefaultListen = "127.0.0.1:8081"

	// Defaults aligned with docs/LLM_API_PROXY.md § 2 CLI surface.
	DefaultUpstreamOpenAI    = "https://api.openai.com"
	DefaultUpstreamAnthropic = "https://api.anthropic.com"
	DefaultGuardURL          = "http://127.0.0.1:8080"
	DefaultTenantID          = "local"
	DefaultFailMode          = "deny"
	DefaultLogLevel          = "info"

	// DefaultMaxBufferBytes (1 MiB) is the per-stream tool-call
	// accumulation cap — Phase 4A Q2 user-approved value. Mirrors
	// the central server's MaxRequestBodySize so /v1/check side-channel
	// payloads always fit. A22 wires the actual buffering against
	// this; A21 only stores the configured value.
	DefaultMaxBufferBytes = 1024 * 1024 // 1 MiB

	// MaxConfigurableBufferBytes refuses pathological values up front.
	// 64 MiB is the upper bound — beyond that, gating an individual
	// tool call is no longer the right tool (operators should split
	// the workload).
	MaxConfigurableBufferBytes = 64 * 1024 * 1024 // 64 MiB

	// DefaultMaxConcurrentStreams caps simultaneously-active streaming
	// requests on the proxy. Each in-flight stream owns a per-request
	// accumulator + read buffer (up to ~2x --max-buffer-bytes), so a
	// global cap is the only thing keeping memory bounded under fan-out
	// load. 100 was picked as a default that balances normal SDK
	// concurrency (typically <10) against worst-case memory (100 × 1
	// MiB ≈ 200 MiB read+accumulator territory). Operators with heavy
	// fan-out should raise this and lower --max-buffer-bytes
	// proportionally; operators on tiny boxes should lower it.
	//
	// 0 disables the cap entirely (legacy behaviour). Closes R-Sec H3.
	DefaultMaxConcurrentStreams = 100
)

Default flag values. Centralised here so tests can refer to them without re-running the flag parser.

View Source 
const (
	FailModeRuleClosed      = "deny:llm_api_proxy:fail_closed"
	FailModeRuleClosedAudit = "deny:llm_api_proxy:fail_closed_audit"
	FailModeRuleOpen        = "allow:llm_api_proxy:fail_open"
	InvalidResponseRule     = "deny:llm_api_proxy:invalid_response"
)

FailModeRuleClosed and friends are the synthetic Rule strings the gate stamps on fail-mode decisions so operators can alert on them without confusing them with real policy verdicts. Stable string contracts — referenced from tests + dashboard.

View Source 
const DefaultGuardHTTPTimeout = 5 * time.Second

DefaultGuardHTTPTimeout is the per-/v1/check-call timeout the gate applies when the operator does not pass a custom http.Client. Five seconds matches the SDKs and the value documented in docs/PROXY_ARCHITECTURE.md § 6.1.

View Source 
const ProxyAuthHeader = "X-AgentGuard-Proxy-Auth"

ProxyAuthHeader is the inbound-auth header the proxy enforces when --proxy-api-key is set. Distinct from the Authorization header so the upstream's bearer token (which the proxy forwards verbatim) is not aliased with the proxy's own credential.

docs/LLM_API_PROXY.md § 8.1 originally proposed re-keying the Authorization header; A21 chose a separate header to avoid having the SDK send two `Authorization` values (which is not legal in HTTP/1.1 anyway). SDK callers wishing to authenticate with the proxy set this header in addition to (not instead of) the upstream's Authorization.

View Source 
const UnmappedScope = "unmapped"

UnmappedScope is the sentinel scope returned when a tool name does not match any default or operator-supplied entry. The policy engine has no built-in handling for this scope — by design — so the gate fails closed (default DENY) unless the operator opts into a `scope: unmapped` rule. Stable string contract: documented in docs/POLICY_REFERENCE.md and referenced from A24's gate code.

Variables

View Source 
var BuildVersion = "dev"

BuildVersion is overridden via -ldflags by the binary entry point. Defaults to "dev" for `go test` / `go run`.

View Source 
var DefaultLLMToolScopeMap = []policy.ToolScopeMapping{

	{Pattern: "bash", Scope: "shell"},
	{Pattern: "sh", Scope: "shell"},
	{Pattern: "shell", Scope: "shell"},
	{Pattern: "run_command", Scope: "shell"},
	{Pattern: "execute_command", Scope: "shell"},
	{Pattern: "cmd", Scope: "shell"},
	{Pattern: "system", Scope: "shell"},
	{Pattern: "exec", Scope: "shell"},

	{Pattern: "read_file", Scope: "filesystem"},
	{Pattern: "write_file", Scope: "filesystem"},
	{Pattern: "list_directory", Scope: "filesystem"},
	{Pattern: "list_files", Scope: "filesystem"},
	{Pattern: "file_read", Scope: "filesystem"},
	{Pattern: "file_write", Scope: "filesystem"},
	{Pattern: "edit_file", Scope: "filesystem"},
	{Pattern: "delete_file", Scope: "filesystem"},
	{Pattern: "create_directory", Scope: "filesystem"},
	{Pattern: "ls", Scope: "filesystem"},
	{Pattern: "cat", Scope: "filesystem"},
	{Pattern: "find", Scope: "filesystem"},
	{Pattern: "glob", Scope: "filesystem"},

	{Pattern: "web_search", Scope: "network"},
	{Pattern: "fetch_url", Scope: "network"},
	{Pattern: "http_request", Scope: "network"},
	{Pattern: "http_get", Scope: "network"},
	{Pattern: "http_post", Scope: "network"},
	{Pattern: "search", Scope: "network"},
	{Pattern: "fetch", Scope: "network"},
	{Pattern: "url_request", Scope: "network"},

	{Pattern: "playwright_*", Scope: "browser"},
	{Pattern: "browser_*", Scope: "browser"},
	{Pattern: "chrome_*", Scope: "browser"},
	{Pattern: "firefox_*", Scope: "browser"},
	{Pattern: "selenium_*", Scope: "browser"},
	{Pattern: "navigate", Scope: "browser"},
	{Pattern: "click", Scope: "browser"},
	{Pattern: "screenshot", Scope: "browser"},
}

DefaultLLMToolScopeMap is the baked-in mapping from common tool names to existing AgentGuard policy scopes. Per Phase 4A, the LLM API Proxy maps to existing scopes only — no new policy primitives.

Patterns are glob-matched (same matcher as policy rule patterns: `*` matches any chars including `/`, `?` matches a single char, `**` matches zero or more `/`-separated segments). First-match-wins.

Operators override or extend via the `tool_scope_map:` section of policy YAML (see docs/POLICY_REFERENCE.md § "LLM API Proxy tool scope mapping" for examples).

Categories per docs/LLM_API_PROXY.md § "Tool call → scope mapping":

  • shell: bash, run_command, execute_command, sh, cmd, ...
  • filesystem: read_file, write_file, list_directory, ls, cat, ...
  • network: web_search, fetch_url, http_request, ...
  • browser: playwright_*, browser_*, chrome_*, firefox_*, ...
  • data: (no defaults — operators map fill_form / submit_form here for PII gating against `data` scope rules)
  • cost: (no defaults — model-cost gating is its own scope with different field semantics; operators wire it via SDK est_cost rather than via tool-name mapping)
View Source 
var ErrPolicyNotLoaded = errors.New("llmproxy: policy snapshot not loaded")

ErrPolicyNotLoaded is returned by gate-side helpers when no policy snapshot has been wired into the gate yet. Currently unused inside gate.go (the nil-policy path falls through to default mappings) but exposed so downstream callers (cmd/agentguard-llm-proxy/main.go) can sentinel-check should they need to.

Functions

func BuildRefusalRich 

func BuildRefusalRich(provider string, decision Decision, ctx *RefusalContext) []byte

BuildRefusalRich is the function A24 wires into Server.BuildRefusal. Called by streaming.go's gateAndFlush* on DENY/REQUIRE_APPROVAL or by the overflow path with a synthetic Decision (see streaming.go's runOpenAIStreamLoop / runAnthropicStreamLoop). Also called by F9 (B2) for non-streaming refusals via the same hook — the ctx.NonStreaming flag picks the shape.

Always returns a non-nil byte slice: an empty refusal would leave the SSE stream open (or yield a zero-length response on the non-streaming path) and SDK clients would hang.

func MapLLMToolScope 

func MapLLMToolScope(toolName string, mappings []policy.ToolScopeMapping) string

MapLLMToolScope is the function A24 wires into Server.ScopeMap (via a closure capturing the merged mapping list). Returns the scope of the first matching entry, or UnmappedScope if nothing matches.

"Unmapped" tools are routed to scope "unmapped" at gate time. The policy engine has no built-in rules for this scope, so the default behaviour is fail-closed (DENY: "No matching allow rule (default deny)"). Operators who want unknown LLM tools to pass through must write an explicit `scope: unmapped` rule — typically `require_approval: [{pattern: "*"}]` so a human sees the tool name before it runs.

Implementation note: dispatches through a synthetic *policy.Policy so the canonical glob matcher (`pkg/policy/engine.go:globMatch`, unexported) is used without re-implementing it. The synthetic policy is allocated on every call — cheap (one struct + one slice header) and avoids the alternative of exporting globMatch from pkg/policy.

func NewLLMToolScopeMap 

func NewLLMToolScopeMap(pol *policy.Policy) []policy.ToolScopeMapping

NewLLMToolScopeMap merges operator policy entries with the bundled defaults. Operator entries appear FIRST in the returned slice so first-match-wins iteration honours operator overrides.

pol may be nil (no policy loaded) — returns DefaultLLMToolScopeMap directly (without copying — callers must not mutate the slice).

The returned slice is freshly allocated when pol is non-nil, so callers may pass it across goroutines or stash it in atomic.Pointer without aliasing concerns vs the operator's live policy snapshot.

func Run 

func Run(ctx context.Context, cfg *Config) error

Run is the package-level convenience entry point invoked by cmd/agentguard-llm-proxy/main.go. Mirrors mcpgw.Run.

Types

type AnthropicAccumulator 

type AnthropicAccumulator struct {
	// contains filtered or unexported fields
}

AnthropicAccumulator stitches Anthropic streaming events into gating-ready ToolCallCheck records and holds the raw SSE bytes for byte-identical ALLOW-path replay. Mirrors OpenAIToolCallAccumulator's contract; the orchestrator branches on FeedResult identically.

Note Anthropic's stream is not strictly serial: in principle there can be a text content block at index 0, a tool_use at index 1, another text at index 2, and they may interleave during streaming. We buffer ALL events from the first tool_use content_block_start onward — even text deltas to other indices — until the tool_use's content_block_stop closes it. Reasoning: emitting text deltas while holding back a tool_use would deliver an out-of-order stream that some clients don't tolerate. (In practice Anthropic emits content blocks serially: each block fully closes before the next begins.)

func NewAnthropicAccumulator 

func NewAnthropicAccumulator(maxBufferBytes int) *AnthropicAccumulator

NewAnthropicAccumulator constructs a fresh accumulator with the given per-stream buffer cap. maxBufferBytes <= 0 means "no cap".

func (*AnthropicAccumulator) ActiveToolUseIndex 

func (a *AnthropicAccumulator) ActiveToolUseIndex() int

ActiveToolUseIndex returns the content-block index of the currently-buffering tool_use, or -1 if none. The orchestrator uses this to populate RefusalContext.AnthropicToolUseIndex so a refusal rewrites the right block.

func (*AnthropicAccumulator) BufferedEvents 

func (a *AnthropicAccumulator) BufferedEvents() [][]byte

BufferedEvents returns raw event byte slices held back since the active tool_use's start. Caller treats them as read-only.

func (*AnthropicAccumulator) FeedEvent 

func (a *AnthropicAccumulator) FeedEvent(rawEvent []byte) (FeedResult, error)

FeedEvent ingests one complete Anthropic SSE event. Returns FeedResult per the same contract as OpenAIToolCallAccumulator.

Errors are returned for malformed JSON in the data line; the orchestrator's policy is log + drop without injecting bytes.

func (*AnthropicAccumulator) Reset 

func (a *AnthropicAccumulator) Reset()

Reset clears accumulator state. Called after a successful flush so subsequent tool_use blocks in the same message get a fresh state.

type AnthropicContentBlock 

type AnthropicContentBlock struct {
	Type string `json:"type"`

	// type=="text"
	Text string `json:"text,omitempty"`

	// type=="tool_use"
	ID    string          `json:"id,omitempty"`
	Name  string          `json:"name,omitempty"`
	Input json.RawMessage `json:"input,omitempty"`
}

AnthropicContentBlock is one element of the response content array. Text blocks carry `text`; tool_use blocks carry `id`, `name`, `input`. Other types (e.g. image) round-trip untouched because the gating logic only acts on tool_use.

type AnthropicMessagesRequest 

type AnthropicMessagesRequest struct {
	Model    string            `json:"model"`
	Stream   bool              `json:"stream,omitempty"`
	Messages []json.RawMessage `json:"messages,omitempty"`
	Tools    []AnthropicTool   `json:"tools,omitempty"`
}

AnthropicMessagesRequest is the /v1/messages request body. Like the OpenAI struct, only fields the proxy reads are typed; the rest passes through in the original bytes.

type AnthropicMessagesResponse 

type AnthropicMessagesResponse struct {
	ID         string                  `json:"id"`
	Type       string                  `json:"type"`
	Role       string                  `json:"role"`
	Model      string                  `json:"model"`
	Content    []AnthropicContentBlock `json:"content"`
	StopReason string                  `json:"stop_reason"`
	Usage      *AnthropicUsage         `json:"usage,omitempty"`
}

AnthropicMessagesResponse is the non-streaming response. The content array is heterogeneous (text blocks, tool_use blocks); the proxy parses the type discriminator on each block and walks tool_use blocks for gating.

type AnthropicTool 

type AnthropicTool struct {
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	InputSchema json.RawMessage `json:"input_schema,omitempty"`
}

AnthropicTool describes one tool the model can call.

type AnthropicUsage 

type AnthropicUsage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
}

AnthropicUsage mirrors the Anthropic billing report.

type ChatCompletionChoice 

type ChatCompletionChoice struct {
	Index        int                   `json:"index"`
	Message      ChatCompletionMessage `json:"message"`
	FinishReason string                `json:"finish_reason"`
}

ChatCompletionChoice is one candidate in the response.

type ChatCompletionMessage 

type ChatCompletionMessage struct {
	Role      string                       `json:"role"`
	Content   *string                      `json:"content,omitempty"`
	ToolCalls []ChatCompletionToolCallEcho `json:"tool_calls,omitempty"`
}

ChatCompletionMessage carries the assistant turn — content (string or null when there are tool calls) and zero-or-more tool_calls.

type ChatCompletionRequest 

type ChatCompletionRequest struct {
	Model    string               `json:"model"`
	Stream   bool                 `json:"stream,omitempty"`
	Messages []json.RawMessage    `json:"messages,omitempty"`
	Tools    []ChatCompletionTool `json:"tools,omitempty"`
}

ChatCompletionRequest is the OpenAI /v1/chat/completions request body, parsed only to the depth the proxy needs. The Messages slice is kept as raw JSON because the proxy never inspects message content; tools however are parsed so A23 can name-match them against the scope map.

type ChatCompletionResponse 

type ChatCompletionResponse struct {
	ID      string                 `json:"id"`
	Object  string                 `json:"object"`
	Created int64                  `json:"created"`
	Model   string                 `json:"model"`
	Choices []ChatCompletionChoice `json:"choices"`
	Usage   *ChatCompletionUsage   `json:"usage,omitempty"`
}

ChatCompletionResponse is the non-streaming response. Modelled to the depth A24 needs (choices[i].message.tool_calls) plus passthrough-style fields the proxy might surface in audit meta (id, model, usage).

type ChatCompletionTool 

type ChatCompletionTool struct {
	Type     string                 `json:"type"`
	Function ChatCompletionToolFunc `json:"function"`
}

ChatCompletionTool models one element of the request's `tools` array. Only "function" type is defined in the OpenAI spec.

type ChatCompletionToolCallEcho 

type ChatCompletionToolCallEcho struct {
	ID       string                 `json:"id"`
	Type     string                 `json:"type"`
	Function ChatCompletionToolEcho `json:"function"`
}

ChatCompletionToolCallEcho is the response-side shape of a tool call: the model picked a function and synthesised arguments. The `arguments` field on the wire is a JSON-encoded STRING (yes — a string holding JSON) per the OpenAI spec. The proxy preserves it as-is so re-parsing happens at the policy-gate boundary.

type ChatCompletionToolEcho 

type ChatCompletionToolEcho struct {
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
}

ChatCompletionToolEcho is the (name, JSON-string-arguments) pair.

type ChatCompletionToolFunc 

type ChatCompletionToolFunc struct {
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	Parameters  json.RawMessage `json:"parameters,omitempty"`
}

ChatCompletionToolFunc carries the function-tool definition. The JSON-Schema parameters object is kept raw — the proxy does not validate arguments, only forwards them.

type ChatCompletionUsage 

type ChatCompletionUsage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

ChatCompletionUsage is the token-usage report. Optional in streaming and absent from intermediate chunks.

type Config 

type Config struct {
	// Listen is the proxy's HTTP bind address. Default DefaultListen
	// (loopback). Non-loopback is rejected unless --proxy-api-key
	// is set (mirrors central server's localhost-only fallback).
	Listen string

	// UpstreamOpenAI is the base URL for OpenAI-shape requests
	// (/v1/chat/completions, /v1/completions, /v1/embeddings, /v1/models).
	UpstreamOpenAI string

	// UpstreamAnthropic is the base URL for Anthropic-shape requests
	// (/v1/messages).
	UpstreamAnthropic string

	// GuardURL is the central AgentGuard server's base URL — A24's
	// PolicyCheck hook calls <GuardURL>/v1/check.
	GuardURL string

	// APIKey is the bearer token sent on the /v1/check side channel.
	// Falls back to AGENTGUARD_API_KEY env var when the flag is empty.
	// Distinct from the user's upstream Authorization header (which
	// is forwarded verbatim, never read).
	APIKey string

	// ProxyAPIKey is the OPTIONAL bearer the proxy enforces on inbound
	// requests via the X-AgentGuard-Proxy-Auth header. Empty disables
	// proxy-level auth (safe on loopback). Sent in a separate header
	// from Authorization so the upstream's own bearer token can be
	// forwarded unambiguously.
	ProxyAPIKey string

	// TenantID is plumbed through to /v1/check and audit entries.
	TenantID string

	// FailMode mirrors the SDK / MCP gateway contract from
	// docs/PROXY_ARCHITECTURE.md § 6.1: "deny" | "allow" |
	// "fail-closed-with-audit". A24 honours this on /v1/check failures.
	FailMode string

	// MaxBufferBytes caps the per-stream tool-call accumulation
	// buffer. Phase 4A Q2 default = 1 MiB. A22 wires actual buffering
	// against this; A21 only validates and stores it so the flag
	// surface is stable from the first build.
	MaxBufferBytes int

	// MaxConcurrentStreams caps simultaneously-active streaming
	// requests across the whole proxy. When the cap is reached, new
	// streaming requests are refused with 503 + Retry-After: 5 instead
	// of being processed (which would otherwise allocate another
	// per-stream accumulator + read buffer pair). 0 disables the cap.
	// Default: DefaultMaxConcurrentStreams.
	//
	// Closes R-Sec H3 (audit B6). Memory ceiling for streaming was
	// previously unbounded in the limit; the per-stream cap
	// (--max-buffer-bytes) only constrains a single in-flight call.
	MaxConcurrentStreams int

	// LogLevel controls stderr verbosity. "info" or "debug".
	LogLevel string

	// PolicyPath points at the policy YAML the proxy reads to resolve
	// the LLM tool→scope mapping locally (operators run the same YAML
	// the central server loads — typically a shared file path).
	//
	// Optional: when unset, A24's gate falls back to the bundled
	// DefaultLLMToolScopeMap and skips hot-reload. main.go logs a
	// WARNING in that case because operator overrides won't apply.
	// Cross-host deployments must mount the file on a shared volume
	// (or replicate the YAML out-of-band) so the proxy and the
	// central server stay in lockstep.
	PolicyPath string
}

Config is the parsed CLI/env configuration for one proxy invocation. Populated by ParseConfig. The server reads it once at startup and treats it as immutable thereafter; hot-reload of any flag is out of scope for v0.5.

func ParseConfig 

func ParseConfig(args []string) (*Config, error)

ParseConfig parses CLI args (without the leading binary name) and returns a Config. Errors are returned for the caller to surface. API-key resolution: explicit --api-key wins over AGENTGUARD_API_KEY.

func ParseConfigWithOutput 

func ParseConfigWithOutput(args []string, errOut io.Writer) (*Config, error)

ParseConfigWithOutput is ParseConfig with the usage stream pluggable for tests. Mirrors mcpgw.ParseConfigWithOutput.

func (*Config) Validate 

func (c *Config) Validate() error

Validate enforces invariants on a parsed Config. Exposed for tests that build a Config struct directly without going through the flag parser.

type Decision 

type Decision struct {
	Allow            bool
	RequiresApproval bool
	Reason           string
	Rule             string
	ApprovalID       string
	ApprovalURL      string
}

Decision is the verdict returned by PolicyCheck. Mirrors mcpgw.Decision so refusal-rewriting code (A24) can share helpers.

type FeedResult 

type FeedResult struct {
	// PassThrough: forward the raw event bytes to the client now,
	// byte-identical. (No tool_call in flight; or this is a [DONE]
	// terminator after the whole stream completes without tool_calls.)
	PassThrough bool

	// Accumulating: buffer the raw event bytes; do NOT flush to client
	// yet. A tool_call is in flight.
	Accumulating bool

	// Completed: a tool_call finish boundary was observed. The
	// orchestrator should now gate each tool call in CompletedToolCalls
	// through PolicyCheck. On ALLOW it flushes the buffered events
	// (BufferedEvents()); on DENY it discards them and emits a
	// synthetic refusal.
	Completed bool

	// CompletedToolCalls is the list of fully-assembled tool calls,
	// in tool_calls[i].index order. Populated only when Completed=true.
	CompletedToolCalls []ToolCallCheck

	// OverflowBufferBytes signals that the cumulative buffered byte
	// count exceeded the cap. The orchestrator emits the canonical
	// "tool call arguments exceed gating buffer" refusal.
	OverflowBufferBytes bool
}

FeedResult is what FeedEvent returns to the streaming orchestrator. Exactly one of {PassThrough, Accumulating, Completed, OverflowBufferBytes} should be true; the orchestrator branches on it.

type HTTPPolicyClient 

type HTTPPolicyClient struct {
	GuardURL string // e.g. "http://127.0.0.1:8080"
	APIKey   string // bearer token; empty if --api-key not set
	TenantID string // "local" for v0.5
	FailMode string // "deny" | "allow" | "fail-closed-with-audit"

	// HTTPClient is reused across calls. Set to a custom client in
	// tests; otherwise the constructor defaults to a 5s-timeout client.
	HTTPClient *http.Client
	// contains filtered or unexported fields
}

HTTPPolicyClient is the LLM API Proxy's wire-level connection to the central AgentGuard server's /v1/check endpoint. One instance per proxy process; the underlying http.Client (and its connection pool) is reused.

Mirrors pkg/mcpgw/gate.go:HTTPPolicyClient with these differences:

  • No dual-check (the mapped scope IS the gate scope).
  • Argument projection per mapped scope (path for filesystem, url+domain for network/browser, command for shell, ...).
  • meta.transport stamped as "llm_api_proxy" on every call so the central server's audit log + SSE chip + transport-tag tests (Phase 4B A19) attribute the entry to this proxy.

func NewHTTPPolicyClient 

func NewHTTPPolicyClient(cfg *Config, pol *policy.Policy) *HTTPPolicyClient

NewHTTPPolicyClient constructs a gate against cfg + an initial policy snapshot. The caller is expected to subscribe to the policy provider's Watch and call SetPolicy on every reload (mirrors mcpgw).

func (*HTTPPolicyClient) Check 

func (c *HTTPPolicyClient) Check(ctx context.Context, req *ToolCallCheck) (Decision, error)

Check is the function wired into Server.PolicyCheck. Builds an ActionRequest from the ToolCallCheck and POSTs to /v1/check.

Closes the streaming gate: A22 calls this per assembled tool_call in either provider's stream; this returns an llmproxy.Decision the streaming orchestrator branches on (ALLOW → flush buffered events; DENY/REQUIRE_APPROVAL → synthesize refusal).

Errors from /v1/check (network, malformed response, non-2xx status) are translated into the configured fail-mode decision; the err is returned alongside so callers that need to log/instrument the underlying cause can do so. The streaming orchestrator (A22) already inspects err for fail-mode-allow short-circuiting.

func (*HTTPPolicyClient) MapScope 

func (c *HTTPPolicyClient) MapScope(toolName string) string

MapScope returns the existing-scope assignment for a tool name. Wired into Server.ScopeMap. Tools not covered by the merged map return UnmappedScope; the policy engine fails closed on this scope unless an explicit `scope: unmapped` rule is configured.

func (*HTTPPolicyClient) SetPolicy 

func (c *HTTPPolicyClient) SetPolicy(pol *policy.Policy)

SetPolicy atomically updates the cached policy snapshot and the derived mapping list. Called from main.go's provider.Watch callback for hot-reload. nil is accepted (resets to default mappings only).

type OpenAIToolCallAccumulator 

type OpenAIToolCallAccumulator struct {
	// contains filtered or unexported fields
}

OpenAIToolCallAccumulator stitches streaming tool_call fragments back into complete ToolCallCheck records and holds the raw SSE event bytes for byte-identical ALLOW-path replay.

One accumulator per stream — never share between requests. The streaming orchestrator constructs a fresh one in each request goroutine (per-request goroutine isolation is the Phase 4A rule).

func NewOpenAIToolCallAccumulator 

func NewOpenAIToolCallAccumulator(maxBufferBytes int) *OpenAIToolCallAccumulator

NewOpenAIToolCallAccumulator constructs a fresh accumulator with the given per-stream buffer cap. maxBufferBytes <= 0 means "no cap" (used in unit tests; production wires --max-buffer-bytes from Config).

func (*OpenAIToolCallAccumulator) BufferedEvents 

func (a *OpenAIToolCallAccumulator) BufferedEvents() [][]byte

BufferedEvents returns the raw SSE event byte slices held back since the first tool_call delta. Caller MUST treat the returned slices as read-only; the accumulator retains them until Reset().

func (*OpenAIToolCallAccumulator) FeedEvent 

func (a *OpenAIToolCallAccumulator) FeedEvent(rawEvent []byte) (FeedResult, error)

FeedEvent ingests one complete SSE event (raw bytes including the final blank-line terminator). Returns a FeedResult describing what the orchestrator should do with the event.

Errors are returned for malformed JSON in the data line; the orchestrator's policy is to log + drop (do NOT inject our own bytes into the byte-identity stream) and continue.

func (*OpenAIToolCallAccumulator) Reset 

func (a *OpenAIToolCallAccumulator) Reset()

Reset clears accumulator state for a new gating cycle. Called by the orchestrator after a successful flush so subsequent tool_calls in the same stream get a fresh accumulator.

type RefusalContext 

type RefusalContext struct {
	// Provider is "openai" or "anthropic". A24 uses this to pick
	// the right SSE event shape.
	Provider string

	// OriginalToolCall is the parsed tool call that was denied. May
	// be a zero-value when the refusal fires for a non-tool-call
	// cause (buffer overflow before any complete tool_call landed).
	OriginalToolCall ToolCallCheck

	// AnthropicToolUseIndex is the content_block index of the
	// in-flight tool_use the refusal is replacing. -1 when not
	// applicable (overflow before any content_block_start arrived,
	// or OpenAI provider).
	AnthropicToolUseIndex int

	// NonStreaming flips the refusal builder from "SSE event bytes"
	// (the streaming path's shape — assistant-text content delta +
	// [DONE] for OpenAI; content_block_* events for Anthropic) to a
	// single non-streaming JSON object the SDK decodes as a normal
	// chat.completion / message response. F9 (B2) wires this for the
	// non-streaming /v1/chat/completions and /v1/messages forwarders;
	// the streaming orchestrator never sets it (zero-value preserves
	// the SSE shape A22 wired).
	//
	// Model is the original request's model string, surfaced into the
	// synthetic non-streaming response so SDKs that index by model
	// don't see "" / unknown-model errors. Optional; empty falls back
	// to "agentguard-refusal" in the builder.
	NonStreaming bool
	Model        string
}

RefusalContext is the input to Server.BuildRefusal (A24's hook). Carries everything A24 needs to construct a provider-specific synthetic refusal payload.

type Server 

type Server struct {

	// PolicyCheck is the hook A24 wires. The default (nil) returns
	// ALLOW — useful for early bring-up before the policy engine
	// integration. A24 sets this to a function that:
	//   1. Builds a policy.ActionRequest from the ToolCallCheck
	//      (scope from ScopeMap, command = tool name + redacted args,
	//      agent_id from auth/header).
	//   2. POSTs to <guard-url>/v1/check with
	//      meta["transport"] = "llm_api_proxy".
	//   3. Returns a Decision struct.
	//
	// Audit + SSE flow: PolicyCheck POSTs to /v1/check, which already
	// writes the audit entry with transport="llm_api_proxy" and
	// broadcasts the SSE event with the llm_api_proxy chip (per
	// Phase 4B A19's transport-tag plumbing). The proxy itself does
	// NOT emit audit entries directly — single source of truth.
	PolicyCheck func(ctx context.Context, req *ToolCallCheck) (Decision, error)

	// ScopeMap is the hook A23 wires. The default (nil) returns
	// "unmapped" — which the policy engine fails closed unless an
	// "unmapped" scope rule is configured. A23 ships a default
	// mapping (bash → shell, read_file → filesystem, etc.) plus
	// optional policy-YAML overrides.
	ScopeMap func(toolName string) string

	// BuildRefusal is the hook A24 wires. Builds the synthetic-refusal
	// SSE bytes for a denied tool_call. Default (nil) returns a basic
	// generic refusal — useful for early bring-up. A24 sets this to a
	// function that constructs the provider-specific refusal shape
	// (OpenAI assistant-text + [DONE]; Anthropic text-block at the
	// buffered tool_use's index + stop_reason rewrite).
	//
	// See pkg/llmproxy/streaming.go (defaultRefusalBytes) for the
	// fallback implementation. Per Phase 4A the OpenAI shape is
	// assistant-text + [DONE]; the rejected `role: "tool"` shape
	// caused SDK hangs.
	BuildRefusal func(provider string, decision Decision, ctx *RefusalContext) []byte
	// contains filtered or unexported fields
}

Server is the HTTP server. Constructed via NewServer, run via Run. Hooks for A22 (streaming), A23 (scope mapping), and A24 (policy gate + refusal) default to nil-safe pass-through; downstream workers wire concrete implementations.

Concurrency: Server itself is read-only after construction. Per- request goroutines isolate state (no shared mutable map between requests). The shared http.Client pools connections per host.

func NewServer 

func NewServer(cfg *Config) (*Server, error)

NewServer constructs a Server from a parsed Config. URL validation already happened in Config.Validate; we only re-parse here to stash *url.URL pointers.

func (*Server) Run 

func (s *Server) Run(ctx context.Context) error

Run starts the HTTP server. Blocks until ctx.Done() (graceful shutdown) or ListenAndServe returns an error. http.ErrServerClosed is mapped to nil because that's the expected shutdown path.

type ToolCallCheck 

type ToolCallCheck struct {
	// Provider is "openai" or "anthropic". A24 uses this to pick
	// upstream-specific synthetic-refusal shapes.
	Provider string

	// ToolName is the function/tool name as the model emitted it
	// (no namespace prefix for the LLM proxy — that's MCP's pattern).
	ToolName string

	// ToolCallID is the upstream-assigned id (OpenAI: call_xxx,
	// Anthropic: toolu_xxx). Surfaced in audit meta so operators can
	// correlate with provider-side logs.
	ToolCallID string

	// Arguments is the parsed tool-call arguments. For OpenAI, this
	// is the result of json.Unmarshal on the
	// tool_calls[*].function.arguments STRING (which holds JSON);
	// for Anthropic, it's the parsed input object. May be nil if the
	// model emitted invalid JSON — A24 decides how to handle that.
	Arguments map[string]interface{}

	// RawArguments is the unparsed arguments byte slice, in case
	// A23/A24 want to inspect or redact before re-marshalling.
	RawArguments json.RawMessage

	// AgentID, SessionID, TenantID, ApprovalID are the
	// /v1/check-side metadata. AgentID is synthesised from inbound
	// headers (X-Agent-ID) or falls back to "llm-proxy". TenantID
	// comes from cfg.TenantID. ApprovalID is set when the LLM SDK
	// echoed a previously-issued approval id back via a meta
	// channel (v0.6 wires the actual round-trip; A21 leaves the
	// field plumbed-through so A24 doesn't have to add it later).
	AgentID    string
	SessionID  string
	TenantID   string
	ApprovalID string

	// Model is the model name from the request body — surfaced in
	// audit meta.
	Model string

	// Stream indicates whether the request was a streaming one.
	// Surfaced in audit meta so operators can distinguish gating
	// patterns.
	Stream bool

	// UpstreamStatus is the HTTP status code the upstream returned.
	// 0 means "no upstream call yet" (shouldn't happen in the
	// gating path; reserved for future failure-mode plumbing).
	UpstreamStatus int
}

ToolCallCheck is the bridge-internal shape passed to PolicyCheck. A24 reads this, calls /v1/check, returns Decision. The shape mirrors mcpgw.ToolsCallRequest deliberately — operators get a uniform mental model across the two proxies.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.