optictrace

package module
v0.15.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 20, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

README

👁️ OpticTrace

Declarative API telemetry & governance — like OpenAPI, but for observability.

One optic.yaml controls what your API traffic reveals: which routes are monitored, which payloads are captured, what gets redacted, and which request attributes become Prometheus dimensions.

Go License Status

optictrace product page →


Real output from a running agent — the client gets the original bytes, the telemetry never sees the card.

Contents

Why OpticTrace?

API observability usually forces a bad trade: log everything (and leak credit cards into your log pipeline) or log nothing useful. OpticTrace makes the trade-off declarative and reviewable:

rules:
  - name: redact-payment-secrets
    match: { path: "/api/v1/payments/**" }
    redact:
      headers: [Authorization]
      json_fields: ["$.**.credit_card.number"]   # any nesting depth
    labels:
      tenant: "header:X-Tenant-ID"               # a real Prometheus dimension
  • 🔍 Capture-by-default, restrict-by-rule — everything is observable unless a rule says otherwise, and the rules live in your repo where they get code-reviewed.
  • 🛡️ Traffic is never mutated — governance applies to what gets recorded; clients and upstreams always see original bytes.
  • 📊 Prometheus-native — request counts, error rates, P50/P95/P99 per route, plus your own label dimensions extracted from headers or query params.
  • 🖥️ Built-in developer dashboard — live charts, a searchable request inspector, a config linter, served by the same single binary.
  • ⚡ Built for the hot path — rules compile once at startup; restricted routes skip capture entirely; body capture is size-bounded; storage is async and drops rather than blocks.

And because OpticTrace owns your real traffic history, it does things static tools can't:

  • 🤖 Reviews your pull requests — a GitHub Action that comments on every PR with what the change does to governance, measured by replaying real traffic under both the old and new rules. It catches the change that looks harmless in a diff but stops redacting a card number.
  • 🕵️ Leak detector — optictrace scan finds sensitive values your rules didn't cover. Redaction masks what you name; this catches the field you forgot, and prints the rule that would have stopped it.
  • 🧪 Testable governance — optictrace test asserts your rules behave as intended, with no server and no network, so CI proves a refactor didn't stop redacting.
  • 🧬 Traffic → OpenAPI → SDK — optictrace spec infers a spec from what clients actually send; optictrace sdk emits typed TypeScript, Python or Go clients.
  • 🚨 Breaking-change linter — optictrace check answers "is any live client actually using the field I'm about to remove?" with usage counts and last-seen times. Exits non-zero in CI.
  • 🎭 Stateful mock server — optictrace mock gives you a mock where POST /cart then GET /cart really returns the added item; optional AI-generated responses via Claude.
  • 💰 FinOps metering — extract usage figures (e.g. LLM token counts) from responses, attribute cost per tenant, export billing CSVs.

How it works

The idea

Everything is captured by default; rules subtract from that baseline. The governing invariant makes this safe to adopt:

Live traffic is never modified. Restriction and redaction apply only to the telemetry OpticTrace records. A rule that masks $.credit_card.number does not strip the card number from the payment request — the payment still works. It strips it from what gets logged, stored, and exported.

Request flow: two lanes, one tee point

A request travels one path while its telemetry travels another. The traffic lane is a plain reverse proxy. The telemetry lane branches off a bounded copy and is where all governance happens.

flowchart LR
    C(["Client"])
    P["OpticTrace :8080"]
    U(["Your service"])

    subgraph tel ["TELEMETRY LANE - governed before anything is written"]
        direction LR
        G["1 - Evaluate<br/>match rules, merge policy"]
        A["2 - Attach<br/>buffers, or skip entirely"]
        O["3 - Observe<br/>status, latency, bytes"]
        V["4 - Govern<br/>restrict, redact, meter"]
        F["5 - Fan out<br/>one canonical record"]
        G --> A --> O --> V --> F
    end

    S1["Console<br/>structured JSON"]
    S2["Prometheus<br/>metrics endpoint"]
    S3["SQLite<br/>async, drops before blocking"]
    S4["Exporters<br/>file, webhook, plugin"]

    C -->|"request"| P
    P -->|"forwarded verbatim"| U
    U -.->|"response"| P
    P -.->|"returned byte-for-byte"| C
    P ==>|"bounded copy"| G
    F --> S1
    F --> S2
    F --> S3
    F --> S4

    style P fill:#0a6b89,stroke:#0a6b89,color:#ffffff
    style tel fill:#f6ebd6,stroke:#8e5c0d,color:#8e5c0d
Stage What happens Why it matters
1 · Evaluate Method + path matched against compiled rules; every match merges into one policy Rules compile once at startup — the hot path is a linear scan of cheap comparisons
2 · Attach Capture buffers wired up — or skipped entirely The policy resolves before capture attaches, so a restricted route allocates no buffers at all
3 · Observe Upstream runs; status, latency and byte counts recorded Metadata is always recorded, even when payload capture is fully restricted
4 · Govern Restricted fields dropped, redacted fields masked, labels and meters extracted Nothing downstream ever sees raw sensitive data
5 · Fan out One canonical record handed to every sink Console, Prometheus, store and exporters all receive the same governed record

Why a bounded copy: capture is capped at capture_limit_bytes (64 KB default) and flagged as truncated on overflow. A 2 GB upload streams through to the upstream at full speed while telemetry stays small — the tap never becomes a bottleneck.

Architecture: components, and who talks to whom

Traffic and control plane listen on separate ports by design, so the dashboard and metrics can be firewalled independently of the API being proxied.

flowchart TB
    subgraph ing ["INGRESS"]
        direction LR
        I1["Sidecar :8080"]
        I2["Go middleware<br/>embedded in your app"]
        I3["SDK ingest<br/>Express, FastAPI, Gin"]
    end

    subgraph core ["CORE"]
        direction LR
        E["Rule engine<br/>compiled globs, hot-swappable"]
        X["Interceptor<br/>tees, applies policy, builds record"]
    end

    subgraph sink ["SINKS"]
        direction LR
        M["Collector<br/>private Prometheus registry"]
        W["Async writer<br/>bounded queue to SQLite"]
        D["Dispatcher<br/>per-exporter queues"]
        L["Logger<br/>slog JSON"]
    end

    subgraph ctrl ["CONTROL PLANE :9095"]
        direction LR
        API["Admin API<br/>logs, stats, usage, reload"]
        MET["Metrics endpoint"]
        UI["Dashboard<br/>Next.js static export"]
    end

    subgraph tools ["OFFLINE TOOLS - read the captured history"]
        direction LR
        T1["spec<br/>to OpenAPI"]
        T2["check<br/>spec vs. usage"]
        T3["sdk<br/>to TypeScript"]
        T4["mock<br/>to stateful server"]
    end

    ing --> core
    core --> sink
    M --> ctrl
    W --> ctrl
    D --> ctrl
    W -.-> tools

    style core fill:#0a6b89,stroke:#0a6b89,color:#ffffff

Everything above ships in a single binary — the dashboard is compiled in as static files. The offline tools are commands you run against captured history, not long-running services.

A step-by-step integration guide covers each route in full — sidecar on a host, in Docker, in Compose or as a Kubernetes sidecar; middleware for Express, FastAPI, Java, Go and Gin — plus a rollout order for a service that is already in production.

Two deployment modes share one code path:

  • Sidecar / gateway — optictrace run reverse-proxies to your service.
  • Embedded — native middleware inside your app (Go, Express, FastAPI). SDKs apply governance in-process, so sensitive data never leaves your app raw, then ship governed records to the agent.

Quickstart

Docker Compose (agent + demo API + Prometheus)
git clone https://github.com/dwarka-prasad/optictrace && cd optictrace
docker compose up --build
URL What
http://localhost:8080 your API, proxied through OpticTrace
http://localhost:9095 dashboard · /metrics · query APIs
http://localhost:9090 Prometheus, pre-configured to scrape OpticTrace
http://localhost:3000 Grafana, dashboard provisioned

A one-shot seeder service drives multi-tenant traffic through the demo API as the stack comes up, so the dashboard has real records, labels and traces on it immediately — an empty dashboard tells you nothing about whether the rules work. Re-run it any time with docker compose run --rm seeder.

Install
brew install dwarka-prasad/tap/optictrace     # macOS and Linux
go install github.com/dwarka-prasad/optictrace/cmd/optictrace@latest

Or grab a signed binary from releases. The Homebrew formula ships an annotated optic.yaml and its rule tests, so optictrace validate works the moment it's installed.

On Linux, Homebrew pulls in its own glibc/gcc for any formula — OpticTrace itself is a static, CGO-free binary and needs neither, so the tarball or go install is lighter if you'd rather skip that.

From source
go build -o bin/ ./cmd/...
(cd ui && npm install && npm run build)   # optional: the embedded dashboard
./bin/optictrace validate -config optic.yaml
./bin/optictrace run -config optic.yaml

Send traffic through service.listen, then open http://localhost:9095.

optic.yaml reference

Parsing is strict: unknown keys are rejected rather than silently ignored, so a typo like restirct: fails at load instead of quietly disabling your governance.

# ── identity ──────────────────────────────────────────────
version: 1

service:
  name: payments-api
  listen: ":8080"                    # proxied traffic
  upstream: "http://localhost:9000"

# ── where telemetry goes ──────────────────────────────────
telemetry:
  admin_listen: "127.0.0.1:9095"     # dashboard + /metrics + APIs (loopback by default)
  cors_origins: []                   # browser origins allowed cross-origin; none by default
  console_log: true                  # structured JSON on stdout
  metrics:
    enabled: true
    buckets: [0.005, 0.05, 0.5, 5]   # latency histogram bounds (seconds)
  auth:                              # control-plane authentication (off by default)
    token_env: OPTICTRACE_ADMIN_TOKEN  # preferred: keeps the secret out of git
    # token: "literal-token"           # alternative, discouraged
    allow_health: true                 # keep /healthz open for probes
  tls:                               # optional HTTPS for the control plane
    cert_file: /etc/optictrace/tls.crt
    key_file:  /etc/optictrace/tls.key
  store:
    driver: sqlite                   # sqlite | postgres | clickhouse | none
    dsn: optictrace.db
    queue_size: 4096                 # async queue; overflow drops, never blocks
    retention_max_rows: 100000       # oldest rows pruned
    retention_max_age: 720h
    analysis_max_rows: 20000         # cap for scan/spec/suggest/review reads          # ...and anything older than 30 days
  exporters:                         # fan out governed records
    - { name: audit, type: file,    path: ./export/audit.jsonl }
    - { name: siem,  type: webhook, url: "https://siem.internal/ingest",
        headers: { Authorization: "Bearer ..." }, batch_size: 100, flush_interval: 5s }
    - { name: mine,  type: command, command: ["python3", "my_exporter.py"] }
  billing:                           # cost attribution (FinOps)
    consumer_label: tenant
    currency: USD
    prices:
      per_request: 0.0001
      per_gb: 0.05
      per_meter_unit: { tokens: 0.000002 }   # $2 per 1M tokens

# ── the opt-out baseline ──────────────────────────────────
defaults:
  capture: { request_body: true, response_body: true, headers: true }
  capture_limit_bytes: 65536         # per-body telemetry cap (traffic unaffected)

# ── rules: evaluated top-to-bottom, actions merge ─────────
rules:
  - name: no-capture-on-auth
    match:
      path: "/api/v1/auth/**"        # * = one segment, ** = zero or more
      methods: [POST]                # optional; omitted = all methods
    restrict: [request_body, response_body, headers, query]

  - name: redact-payment-secrets
    match: { path: "/api/v1/payments/**" }
    redact:
      headers: [Authorization, X-Api-Key]
      query_params: [api_key, token] # ?api_key=… masked in captured queries
      json_fields:
        - "$.credit_card.number"     # exact dotted path
        - "$.*.ssn"                  # * = any single key
        - "$.**.card_token"          # ** = any nesting depth
    labels:
      tenant: "header:X-Tenant-ID"   # Prometheus dimension + log field
      plan:   "query:plan"
    sample: 0.25                     # capture bodies for 25% of matches
                                     # (metrics & metadata stay complete)

  - name: meter-ai-tokens
    match: { path: "/api/v1/ai/**" }
    restrict: [request_body, response_body]   # prompts stay private...
    meter:
      tokens: "$.usage.total_tokens"          # ...but tokens are still counted

How rules merge. Rules are not first-match-wins. Every matching rule contributes: restrictions only ever narrow capture, while redactions, labels and meters accumulate. Later rules win on conflicting scalars like sample. That lets a broad redaction rule and a narrow restriction rule compose instead of fighting.

Arrays and depth. JSON paths traverse arrays implicitly, so $.items.price covers every element of an items list. $.** descends to any depth — which matters when an upstream echoes a payload back inside a wrapper and would otherwise leak the field you just masked.

Hot reload. kill -HUP <pid> or POST /api/reload swaps the rule engine atomically; in-flight requests finish under their old policy. An invalid config is rejected and the old rules stay live.

What's supported today

✅ Shipped · 🟡 Partial or deliberate limit · ⬜ Not yet

Governance engine
Capability Status Notes
Path globbing ✅ * = one segment, ** = zero or more; shell patterns inside a segment; optional method filters
Restriction ✅ Disable request_body, response_body or headers capture per rule
Redaction ✅ Mask headers by name and JSON fields by path, incl. wildcard and recursive descent
Custom labels ✅ Extracted from headers or query params; become real Prometheus dimensions
Body sampling ✅ Capture payloads for a fraction of matches; metrics and metadata stay complete
Tail-based sampling ✅ keep_errors and keep_slower_than rescue 5xx and slow requests that a uniform draw would have discarded
Metering ✅ Pull numbers out of responses by JSON path — works even on fully restricted routes
Hot reload ✅ SIGHUP or API; invalid configs rejected, old rules stay live
Strict validation ✅ Unknown keys rejected at load; optictrace validate for CI
Rule unit tests ✅ optictrace test asserts matched rules, capture flags, redacted output, labels, meters and leak absence
Leak detection ✅ optictrace scan finds sensitive values outside your rules and suggests the fix; masked output only
Observability & storage
Capability Status Notes
Prometheus exporter ✅ Ten metric families on a private registry, so embedding never collides with an app's own
Bounded cardinality ✅ route is always a rule glob or normalized pattern — /users/42 → /users/:id
SQLite payload store ✅ Pure-Go driver (no CGO), WAL mode, async writer that drops under backpressure
Retention & erasure ✅ Row-count and age-based pruning; optictrace purge deletes everything held for one consumer
Label cardinality guard ✅ Caps distinct values per custom label (default 500); overflow collapses to __over_limit__ and is counted
Postgres driver ✅ Multi-node store with JSONB aggregation and percentile_cont; shares a conformance suite with SQLite
ClickHouse driver ✅ Column store for high-volume retention; quantileExact and argMin aggregation, and it runs the same conformance suite as SQLite and Postgres
Inner spans (db · cache · outbound) ✅ Operations inside a request, with governed attributes — a statement is redacted before storage — and a per-request multiplier that makes an N+1 visible
OpenTelemetry export ✅ type: otlp exporter emits spans over OTLP/HTTP JSON; no SDK dependency
Storage at scale

SQLite is right for a sidecar with a single writer. When several agents — or several replicas of one agent — need shared history, point them at Postgres:

telemetry:
  store:
    driver: postgres
    dsn: "postgres://optic:secret@db:5432/optictrace?sslmode=require"

Both drivers implement the same LogStore interface and are held to it by a shared conformance suite, so behaviour cannot quietly diverge. Postgres pushes percentiles (percentile_cont), usage grouping and label matching into the database via JSONB, where SQLite scans and aggregates in Go.

Run the Postgres half of the suite locally with:

docker run -d -e POSTGRES_PASSWORD=optic -e POSTGRES_DB=optictrace -p 5432:5432 postgres:16-alpine
OPTICTRACE_TEST_POSTGRES='postgres://postgres:optic@localhost:5432/optictrace?sslmode=disable' \
  go test ./internal/store

Export plugins

Capability Status Notes
file ✅ Appends JSON Lines, rotates at a size threshold
webhook ✅ POSTs batched JSON arrays with custom headers; one retry, then the batch counts as failed
otlp ✅ Emits OpenTelemetry spans to a collector; bodies never attached
command — custom plugin hook ✅ Spawns any executable, streams one JSON record per line to stdin; stderr folded into the agent log; crashed plugins restart with backoff
Delivery guarantee 🟡 At-most-once, deliberately. Each exporter has its own bounded queue and worker, so a dead plugin drops only its own records and never stalls the request path
Pull-request reviews

Every other command is one you have to remember to run. This one runs itself, on every pull request, and answers the question a reviewer actually has: does this change make governance weaker?

# .github/workflows/governance-review.yml
- uses: dwarka-prasad/optictrace@v0
  with:
    agent-url: ${{ vars.OPTICTRACE_AGENT_URL }}   # an agent watching staging
    token: ${{ secrets.OPTICTRACE_TOKEN }}
    window: 24h

It posts one comment that updates in place:

✗ This change weakens governance on 4 point(s)
Route Change Requests affected
✗ POST /api/v1/payments/** stops redacting $.**.credit_card.cvv 34
✗ POST /api/v1/payments/** stops redacting query param api_key 34
⚠ POST /api/v1/auth/** now captures request bodies (was restricted) 34
⚠ POST /api/v1/payments/** drops label region (breaks its Prometheus dimension) 34

How it knows. It evaluates the same captured traffic under the base branch's optic.yaml and the PR's, then reports where the two disagree. A rule reordering that silently stops masking a field is invisible in a text diff and obvious here — and every row carries the number of real requests it affects, so the finding is arguable rather than theoretical.

Why it won't get muted. By default a PR fails only for what it changed. Pre-existing leaks are reported for context but don't block, because failing every pull request for a problem someone else introduced is how a bot gets ignored — and an ignored bot protects nothing. Once your backlog is clear, fail-on: critical stops new ones creeping in.

The comment also carries a coverage score (share of traffic governed by a rule, routes with rules, sensitive-looking fields handled), any leaks found, and — with spec: set — changes that would break clients seen in traffic. 404s are excluded from coverage, since you can't write a rule for a route that doesn't exist.

No staging environment? Point it at a JSONL export instead:

optictrace review -config optic.yaml -base-config /tmp/base.yaml \
  -from-file examples/traffic-sample.jsonl

See examples/workflows/governance-review.yml for a complete workflow. This repo dogfoods it in .github/workflows/governance.yml.

Traffic-powered tooling

Capability Status Notes
Infer OpenAPI from traffic ✅ required = present in every request; integer+float widen to number; ID segments collapse to path templates; redacted fields still contribute name and type
Breaking-change linter ✅ Reports usage counts and last-seen times; exits non-zero on breaking findings
TypeScript SDK generation ✅ Dependency-free typed fetch client; passes tsc --strict
Python / Go SDK generation ✅ -lang python emits TypedDict models + urllib client; -lang go emits structs + net/http client
Stateful mock server ✅ Real CRUD state on collection/item routes; schema-conforming data elsewhere with field-name heuristics
AI-generated mock responses 🟡 Implemented behind -ai + ANTHROPIC_API_KEY with deterministic fallback, but not yet exercised against the live API
Query-parameter capture ✅ Captured and governed via redact.query_params / restrict: [query]; feeds spec inference and the leak scanner
Integration & deployment
Capability Status Notes
Sidecar + embedded modes ✅ Reverse proxy and Go http.Handler middleware sharing one interception path
Express / FastAPI / Gin SDKs ✅ Express and FastAPI carry semantically identical engine ports, so redaction happens in-process
Docker / Compose / Helm ✅ Multi-stage non-root image; Compose stack with Prometheus; chart with probes, optional PVC and ServiceMonitor
Admin-port authentication ✅ Optional bearer token (constant-time, token_env) + TLS. Off by default — enable it whenever the port could be reachable
WebSockets ✅ Upgrades pass through; the exchange is recorded as a 101, and the connection itself is not inspected
HTTP/2 (h2c) ✅ Opt-in via service.http2: true
gRPC ⬜ Needs service.http2, and even then bodies are length-prefixed protobuf — without descriptors there is nothing to redact or meter. Use the SDK middleware
GraphQL ✅ Set service.graphql_paths; the operation name then becomes part of the route and is matchable with match.graphql_operation

Surfaces

CLI
Command Does
optictrace run Start proxy + control plane
optictrace validate Lint optic.yaml (CI-friendly)
optictrace test Assert rules behave as intended; exit 1 on failure
optictrace scan Find sensitive values your rules missed; exit 1 on findings
optictrace review PR report: policy diff, coverage, leaks, breaking changes
optictrace purge Erase all stored records for one consumer (erasure requests)
optictrace suggest Propose rules for sensitive-looking field names
optictrace replay Re-issue captured traffic against a target and diff statuses
optictrace spec Infer OpenAPI from captured traffic
optictrace check Spec vs. live usage; exit 1 on breaking findings
optictrace sdk Generate a typed client (-lang typescript|python|go)
optictrace mock Serve a stateful mock from a spec
optictrace version Print the build version
Control-plane API :9095
Endpoint Returns
GET /metrics Prometheus exposition
GET /healthz Liveness + uptime
GET /api/logs Filtered captured exchanges
GET /api/logs/{id} One exchange in full
GET /api/stats Aggregates, time series, percentiles
GET /api/routes Per-route latency breakdown
GET /api/rules/stats Rules joined with live match counts
GET /api/usage Per-consumer usage and cost (&format=csv)
GET /api/scan Sensitive values found outside your rules, across payloads, log lines and span attributes (masked)
GET /api/services Per-service aggregates (fleet view)
GET /api/traces Recent traces, one row each (?errors=1 / ?service= / ?q= / ?label.<k>=)
GET /api/spec OpenAPI inferred from traffic
GET /api/export CSV or JSONL download of captured records
GET /api/config Current config + validity
POST /api/config/validate Lint a candidate config
POST /api/reload Re-read config, hot-swap engine
POST /api/ingest Accept governed records from SDKs
POST /api/applogs/ingest Accept application log lines, correlated by span id
GET /api/applogs Lines a request logged (?span= / ?trace= / ?level=)
GET /api/spans Operations inside a request (?trace= / ?parent= / ?kind= / ?errors=1 / ?min_ms=)
GET /api/spans/breakdown Where a window's time went, by operation (?route=)
GET /api/spans/stats Span counts by kind and service
GET /api/system Agent health, store size, exporter stats
Metrics exposed
Metric Type Labels
optictrace_requests_total counter method route status status_class + yours
optictrace_request_duration_seconds histogram method route + yours
optictrace_request_size_bytes histogram method route
optictrace_response_size_bytes histogram method route
optictrace_inflight_requests gauge —
optictrace_store_dropped_total counter —
optictrace_sdk_ingested_total counter —
optictrace_app_logs_stored_total counter —
optictrace_app_logs_dropped_total counter reason (orphan, level, span_cap)
optictrace_spans_stored_total counter —
optictrace_spans_dropped_total counter reason (orphan, too_fast, request_cap)
optictrace_span_duration_seconds histogram name, kind, service
optictrace_exported_total counter exporter
optictrace_export_failed_total counter exporter
optictrace_export_dropped_total counter exporter
optictrace_label_capped_total counter label
optictrace_label_distinct_values gauge label

P99 per route:

histogram_quantile(0.99,
  sum by (le, route) (rate(optictrace_request_duration_seconds_bucket[5m])))

Pull-request reviews

Every other command is one you have to remember to run. This one runs itself, on every pull request, and answers the question a reviewer actually has: does this change make governance weaker?

# .github/workflows/governance-review.yml
- uses: dwarka-prasad/optictrace@v0
  with:
    agent-url: ${{ vars.OPTICTRACE_AGENT_URL }}   # an agent watching staging
    token: ${{ secrets.OPTICTRACE_TOKEN }}
    window: 24h

It posts one comment that updates in place:

✗ This change weakens governance on 4 point(s)
Route Change Requests affected
✗ POST /api/v1/payments/** stops redacting $.**.credit_card.cvv 34
✗ POST /api/v1/payments/** stops redacting query param api_key 34
⚠ POST /api/v1/auth/** now captures request bodies (was restricted) 34
⚠ POST /api/v1/payments/** drops label region (breaks its Prometheus dimension) 34

How it knows. It evaluates the same captured traffic under the base branch's optic.yaml and the PR's, then reports where the two disagree. A rule reordering that silently stops masking a field is invisible in a text diff and obvious here — and every row carries the number of real requests it affects, so the finding is arguable rather than theoretical.

Why it won't get muted. By default a PR fails only for what it changed. Pre-existing leaks are reported for context but don't block, because failing every pull request for a problem someone else introduced is how a bot gets ignored — and an ignored bot protects nothing. Once your backlog is clear, fail-on: critical stops new ones creeping in.

The comment also carries a coverage score (share of traffic governed by a rule, routes with rules, sensitive-looking fields handled), any leaks found, and — with spec: set — changes that would break clients seen in traffic. 404s are excluded from coverage, since you can't write a rule for a route that doesn't exist.

No staging environment? Point it at a JSONL export instead:

optictrace review -config optic.yaml -base-config /tmp/base.yaml \
  -from-file examples/traffic-sample.jsonl

See examples/workflows/governance-review.yml for a complete workflow. This repo dogfoods it in .github/workflows/governance.yml.

Traffic-powered tooling

All of these read the same governed traffic history in the payload store.

Catch what your rules missed

Redaction only masks what you name. The failure that actually bites is the field nobody wrote a rule for — a new endpoint ships and secrets land in the store. scan inverts the model: it reads records that already passed governance and flags values that look sensitive anyway.

optictrace scan -window 24h

✗ [critical] github-token in POST /api/v1/orders → request_body.$.debug_token
    a GitHub personal access / app token · seen 4× (last 8s ago) · sample gh•••••••••••••89
    fix: redact:
           json_fields: ["$.debug_token"]

⚠ [high] credit-card in POST /api/v1/orders → response_body.$.echo.payment.pan
    a Luhn-valid card number — PCI-DSS scope · seen 4× (last 8s ago) · sample 55••••••••••59
    fix: redact:
           json_fields: ["$.echo.payment.pan"]

scanned 8 record(s): 2 critical, 2 high, 3 medium

Detectors are structural — issuer prefixes, Luhn and mod-97 checksums, PEM framing — not "looks random", so order IDs and timestamps don't trip them. Findings never print the value they found: a scanner that echoes a credential has just leaked it again, into your CI logs. Every sample is masked, and the suggested rule is copy-pasteable into optic.yaml.

Exits non-zero at or above -fail-on (default high), so it gates CI. Also available at GET /api/scan.

Test your governance rules

Rules are security-critical but, without tests, verifiable only by eyeball or by pushing live traffic. optictrace test runs assertions against the real engine — no server, no network:

# optic.test.yaml
- name: auth routes record metadata only
  request:
    method: POST
    path: /api/v1/auth/login
    body: { username: ada, password: hunter2 }
    response: { token: session-token }
  expect:
    matched_rules: [no-capture-on-auth]
    captures_request_body: false
    not_contains: ["hunter2", "session-token"]   # the assertion that matters

- name: tail sampling rescues server errors
  request: { method: POST, path: /api/v1/ai/complete, status: 500 }
  expect: { keeps_body: true }

optictrace test -config optic.yaml -tests optic.test.yaml
# ✓ 6/6 rule test(s) passed against optic.yaml

Every expect field is optional, so a case asserts only what it's about and won't break on unrelated config changes. See optic.test.yaml for the full shipped example.

Infer a spec, lint a proposal, generate an SDK
# Learn an OpenAPI 3 doc from the last 24h of real traffic
optictrace spec -window 24h -out openapi.yaml

# CI gate: does the proposed spec still cover live usage?
optictrace check -spec proposed.yaml -window 24h
#   ✗ [breaking] POST /api/v1/payments/charge: clients send request field
#     "credit_card" (28 time(s), last 22m ago) but the spec omits it
# exit code 1 → the pipeline fails before a real client does

# Typed TypeScript client, from a spec file or straight from traffic
optictrace sdk -lang typescript -out client.ts

Redaction never hides structure — masked fields still contribute their name and type to inference, so governance and documentation don't fight.

Stateful mock server
optictrace mock -spec openapi.yaml -listen :7070

Collection/item routes (/cart + /cart/{id}) get real CRUD state: what you POST is what later GETs return, PATCH merges, DELETE 404s afterwards. Other operations return schema-conforming data with realistic values (emails look like emails, prices like prices). Add -ai with ANTHROPIC_API_KEY set and non-CRUD responses are generated by Claude with full request context — any failure falls back to the deterministic generator, so the mock never needs the network.

Suggest rules, replay traffic
# Propose governance for sensitive-looking FIELD NAMES.
# `scan` reads values; `suggest` reads names — run both.
optictrace suggest -window 24h
#   ✗ [high] json_field "$.payment.card_number" on /api/v1/orders
#       payment card data is PCI-DSS scope (seen 3×)
#   10 suggestion(s); 2 sensitive-looking field(s) already covered by your rules
optictrace suggest -apply proposed-rules.yaml   # review, then merge

# Re-issue captured traffic against staging and diff status codes.
optictrace replay -target https://staging.internal -rate 50
#   replayed 6/7 record(s) in 2ms
#   skipped 1:
#     1 × request body was not captured (restricted or sampled out)
#   status match: 6 · diverged: 0 · failed: 0

Replay is honest about its limits: OpticTrace stores governed records, so a redacted field replays as [REDACTED] and a restricted body was never stored at all. Those requests are skipped with the reason stated rather than sent as something they weren't. That makes replay a tool for exercising routing and regression shape, not for reproducing a payment.

Usage & cost attribution (FinOps)

GET /api/usage and the Usage dashboard page show per-tenant requests, data, compute time, metered units and estimated cost; &format=csv produces a billing export. Metering is independent of capture: a fully restricted route still meters, reading the payload for the number without ever storing it.

Storage at scale

SQLite is right for a sidecar with a single writer. When several agents — or several replicas of one agent — need shared history, point them at Postgres:

telemetry:
  store:
    driver: postgres
    dsn: "postgres://optic:secret@db:5432/optictrace?sslmode=require"

Both drivers implement the same LogStore interface and are held to it by a shared conformance suite, so behaviour cannot quietly diverge. Postgres pushes percentiles (percentile_cont), usage grouping and label matching into the database via JSONB, where SQLite scans and aggregates in Go.

Run the Postgres half of the suite locally with:

docker run -d -e POSTGRES_PASSWORD=optic -e POSTGRES_DB=optictrace -p 5432:5432 postgres:16-alpine
OPTICTRACE_TEST_POSTGRES='postgres://postgres:optic@localhost:5432/optictrace?sslmode=disable' \
  go test ./internal/store

Export plugins

Every governed record — post-restriction, post-redaction, so no export path can ever see raw sensitive data — also fans out to the output plugins declared in optic.yaml.

A command plugin receives one JSON record per stdin line. That's the whole contract — ship to Kafka, S3, BigQuery, a SIEM, anywhere:

#!/usr/bin/env python3
import sys, json
for line in sys.stdin:
    record = json.loads(line)     # already restricted/redacted
    ship_somewhere(record)

See examples/exporters/ for a working CSV plugin.

Writing an extension

Almost all of OpticTrace lives under internal/, which Go forbids other modules from importing — that keeps the implementation free to change. The one exception is ext/, the extension surface: the record types and the two plugin interfaces, and nothing else. It follows semantic versioning; nothing under internal/ does.

A driver becomes configurable purely by being linked into the binary:

package main

import (
    "github.com/dwarka-prasad/optictrace"
    "github.com/dwarka-prasad/optictrace/ext"
)

func init() {
    ext.RegisterStore("s3", func(dsn string, s ext.Settings) (ext.Store, error) {
        return newS3Store(dsn, s.String("bucket", ""), s.Int("shards", 4))
    })
}

telemetry:
  store:
    driver: s3                    # accepted because it's registered
    dsn: "s3://archive"
    settings:                     # your keys; the core doesn't validate these
      bucket: optic-archive
      shards: 8

ext.RegisterExporter works the same way for output plugins (telemetry.exporters[].type).

Verify it against the same suite the built-in drivers run

ext/exttest exports the conformance suite. Two drivers means two chances to drift apart; the suite is what stops them answering the same question differently — including the regression test for the erasure bug where purging a tenant named acme_1 also destroyed acmeX1.

func TestConformance(t *testing.T) {
    exttest.RunStoreSuite(t, func(t *testing.T) ext.Store {
        s, err := NewMyStore(dsn)
        if err != nil { t.Fatal(err) }
        t.Cleanup(func() { s.Close() })
        return s   // must be empty
    })
}

examples/memstore is a complete worked example: an in-memory store in its own module, deliberately outside the github.com/dwarka-prasad/optictrace/... path prefix so it genuinely cannot reach internal/. It passes the full suite using only ext. CI builds it on every PR, so a hole in the extension surface fails the build.

Authentication, authorization and audit

The admin surface has three more hooks: ext.RegisterAuthenticator, ext.RegisterAuthorizer and ext.RegisterAuditor, plus ext.RegisterAdminRoutes for a login callback.

Policy is written against capabilities, never URLs. Every route declares what it exposes, and the core owns that mapping — because only the core knows which handlers return captured payloads:

Capability Routes What it grants
public /healthz, login callbacks reachable without credentials
metrics /metrics Prometheus exposition
read:stats /api/stats, /api/routes, /api/services, /api/usage, … aggregates only, no payloads
read:payload /api/logs, /api/logs/{id} captured request/response bodies
export /api/export bulk egress of the whole store
analyse /api/scan, /api/spec reads payloads, returns derived output
read:config /api/config the governance policy
ingest /api/ingest SDK writes
admin /api/reload changes agent state
ui / dashboard assets

read:payload and export are separate on purpose: "can inspect one request while debugging" and "can download everything" are different grants, and conflating them is how access reviews go badly.

Three properties the core guarantees, each with a test:

  • Fail closed. A panicking authenticator yields 401; a panicking authorizer yields 403. An extension bug cannot open the API.
  • Composition narrows. Every registered authorizer must allow, so adding one can never widen access.
  • Denials don't leak. The reason goes to the audit trail and the log, never to the caller.

An audit event carries what was reached, not just which URL was called — record count, ids, the filter used, the tenant — because "alice listed logs" answers no question an auditor would ask.

examples/adminauth is a complete worked example: SSO with a redirect-and-callback flow, RBAC over the capabilities above, and a JSONL audit log — again in its own module, again unable to reach internal/.

What extensions inherit

Records handed to a Store or an Exporter are already governed — restricted fields absent, redacted fields holding the placeholder. Governance sits upstream of every sink, so no extension can see raw sensitive data and none has to be trusted with it. Two obligations follow: don't reconstruct what governance removed, and make Purge actually delete before it returns — it backs erasure requests.

Following one request across services

Every record carries W3C trace context, so records from several services stop being a flat list and become a request tree.

$ curl '.../api/logs?trace=4bf92f3577b34da6a3ce929d0e0e4736'

  leads    POST /api/v1/leads   200  5.5ms   span=512227e3  <- child of the caller
  scoring  POST /api/score      200  1.8ms   span=2985d314  <- child of leads

Point each service's sidecar at one store and you get this with no extra configuration. OpticTrace adopts an inbound traceparent when the caller sends one and starts a fresh trace when they do not, so it works whether or not the edge is instrumented.

The forwarded request carries this hop's span, which is what makes downstream calls nest under it rather than becoming siblings. That is the one place OpticTrace writes to traffic, and it is narrow — the forwarded copy only, never the response, never what the client sent:

service:
  trace:
    propagate_upstream: true      # default; false keeps the forwarded request byte-identical
    response_header: X-Trace-Id   # off by default — returns the id to the caller

response_header is honoured by the proxy and by every SDK, which is what makes a support conversation start from the customer's screenshot rather than from "roughly what time was that?" — a question that, under concurrent traffic, identifies the wrong request. The SDKs set it before the handler writes its first byte, because a header added after the response is committed is silently discarded.

A malformed traceparent starts a fresh trace rather than failing anything: losing correlation is a nuisance, failing a request over a bad header would be a fault.

What this does and does not give you. Each hop is one span covering the whole exchange OpticTrace saw. Inner spans (below) break that hop down into the operations inside it, which used to be the line where you were sent to an APM. What is still not here is automatic instrumentation: nothing hooks your database driver for you, so an operation appears because someone named it. Point the OTLP exporter at Jaeger or Tempo if you want a full APM alongside — it uses these same ids, so the two views line up. What OpticTrace adds is the governed payload at every level: an APM shows you timings, this shows you the redacted request body — and the redacted statement — you are actually allowed to look at.

Inside one hop: what the request actually did

A hop tells you a request took 300ms. It does not tell you that 280 of them were one query. Inner spans do:

POST /api/v1/orders                                    285ms   self 10ms
  ├── db.query products      SELECT ... WHERE sku = ?    18ms   rows=1
  ├── cache.get product      product:SKU-100             0.2ms  hit=false
  ├── db.query stock         SELECT stock ... WHERE ?     0.3ms  ×4  ← N+1
  ├── db.insert order        INSERT INTO orders ...       9.8ms
  │     └── db.index refresh ANALYZE TABLE orders        2.2ms
  └── http POST acquirer     https://acquirer-eu/charge  41ms   status=200

Two things make this different from every other tracing library:

The attributes are governed. A statement quotes its parameters, a cache key embeds an account id, an outbound URL carries a token in its query string. Span attributes run through redaction, byte caps and a level filter before they are stored — the same treatment as a log line, for the same reason. A driver that interpolates its SQL puts the customer's email in the one field a breakdown most wants to show; here it arrives [REDACTED].

The per-request multiplier is a first-class figure. The breakdown reports count and requests separately, so count/requests is visible. Four thousand calls to one query looks like busy traffic until you know it was a hundred requests:

operation kind count requests ×/req total p95
http POST acquirer http 11 11 1.0 363.7ms 43.2ms
db.insert order db 11 11 1.0 46.5ms 9.8ms
db.query stock db 44 11 4.0 32.0ms 1.2ms
db.query products db 12 12 1.0 23.0ms 18.1ms

Turn it on, then name the operations you care about:

telemetry:
  spans:
    enabled: true
    min_duration: 1ms       # a 20µs cache hit ×1000 is volume, not information
    max_per_request: 200    # the cap being HIT is itself the finding
    max_attr_bytes: 4096
    retention_max_age: 72h
    redact:
      patterns:
        - '\b\d{13,19}\b'                 # card-shaped digit runs
        - '[\w.+-]+@[\w-]+\.[\w.]+'       # emails in a WHERE clause
      fields: [cache.key]

ctx, sp := spans.Start(ctx, "db.query", "db")
sp.Set("db.statement", "SELECT * FROM orders WHERE id = $1")   // the TEMPLATE
defer sp.End()

try (InnerSpan sp = spans.start("db.query", "db")) {
    sp.set("db.statement", SQL).setInt("db.rows", rows.size());
}

with spans.start("db.query", "db") as sp:
    sp.set("db.statement", SQL).set_int("db.rows", len(rows))

await spans.observe('db.query', 'db', (sp) => {
  sp.set('db.statement', SQL);
  return pool.query(SQL, [id]);
});

A failed operation is kept however fast it was — "it returned in 200µs" and "it returned in 200µs with an error" are not the same event. Work outside a request is dropped by default and counted, because attaching it to whichever request happened to be in flight would cross-attribute tenants. optictrace purge deletes a tenant's spans with their records in one transaction: erasing the request while keeping the query it ran is not erasure.

Storage is optional in the same way app logs are — ext.SpanStore is a separate interface from ext.Store, so a third-party driver without it is still a complete driver. All three bundled drivers implement it.

Go's spans.Transport(nil) wraps an http.RoundTripper, so outbound calls are timed and propagated without a call site knowing: a downstream call that is timed but not propagated is a gap someone has to guess about, and one that is propagated but not timed leaves the caller's own view of it missing.

What the request logged

A span answers "what did this call do". The lines your application wrote while serving it answer "why did it do that" — and those two live in different systems, so answering both usually means copying a trace id into a log search and hoping the clocks agree.

OpticTrace already hands your app the span: the traceparent on the forwarded request carries the span id it recorded. Ship your log lines with that id and they are filed under the exact request that produced them.

You can also collect what the application already writes, with no code change at all — which matters, because the services whose logs you most want are usually the ones nobody wants to modify:

optictrace run -config optic.yaml -exec "python app.py"     # its stdout/stderr

telemetry:
  spans:                             # operations inside a request (off by default)
    enabled: true
    min_duration: 1ms                # drop trivially fast operations
    max_per_request: 200             # -1 for no cap; the cap being hit is a finding
    max_attr_bytes: 4096
    drop_orphans: true               # work outside any request
    retention_max_age: 72h
    redact:                          # attributes are free text — a statement quotes its parameters
      patterns: ['\b\d{13,19}\b']
      fields: [cache.key]

  app_logs:
    sources:
      - { type: file, path: /var/log/app.jsonl }   # rotation followed

Collected output is echoed through untouched: collecting a service's logs must not stop them reaching wherever its operators already read them. Tailing starts at the end of an existing file, because replaying a large log on every restart floods the store with history nobody asked for.

Or post them explicitly:

curl -X POST localhost:9095/api/applogs/ingest -d '[
  {"trace_id":"4bf92f35…","span_id":"512227e3…","level":"info","message":"charge received"},
  {"trace_id":"4bf92f35…","span_id":"512227e3…","level":"error","message":"gateway declined"}
]'

curl 'localhost:9095/api/applogs?span=512227e3…'

and the inspector shows them under the request itself:

POST /api/v1/payments/charge -> 201   tenant=acme-corp   span=ee6c471b
  [info ] charge received    plan=platinum tenant=acme-corp
  [error] gateway declined   reason=insufficient_funds

Correlation is a fact here, not a guess. Nothing is matched by timestamp. Under concurrent traffic — which is the normal case — timestamp matching files one tenant's log line inside another tenant's request, and that is precisely the cross-tenant bleed the tagging and purge machinery exists to prevent.

Log lines are the highest-risk surface in this tool, so they run through policy on the way in rather than being stored raw and cleaned up later. A payload is structured and can be redacted by path; a log line is free text written by whoever was debugging that day:

telemetry:
  app_logs:
    enabled: true
    level_min: info            # debug is most of the volume, little of the value
    max_lines_per_span: 200    # one retry loop must not write a million lines
    max_message_bytes: 8192
    retention_max_age: 168h    # logs outgrow records by orders of magnitude
    drop_orphans: true         # lines with no span belong to no request
    redact:
      patterns:                # single-quoted: YAML rejects \s in double quotes
        - 'Bearer\s+\S+'
        - '\b\d{13,19}\b'
      fields: [authorization, password, token, api_key]

An app that logs its own Authorization header while debugging — which is how this actually happens — is stored as calling gateway with [REDACTED].

Three consequences worth knowing before you turn it on:

  • Erasure covers logs too. purge deletes the log lines belonging to the records it deletes, in one transaction. Deleting a tenant's requests while leaving the lines those requests wrote is not erasure, and a log is the likelier place for the personal data to be sitting.
  • Orphans are dropped by default. Startup, cron and background-worker lines carry no span. Every drop is counted in optictrace_app_logs_dropped_total{reason="orphan"} — data discarded silently is data nobody knows they are missing. Set drop_orphans: false to keep them unattributed.
  • Policy can narrow per route. telemetry.app_logs is the floor; a logs: block on a rule tightens it — a stricter level, a lower line cap, extra redaction, or drop: true for a route whose output nothing can pattern-match safely. It can only ever tighten, which is what makes it safe to key on a route the producer reports without verifying it: the worst a silent or lying client can do is land on the global floor.
  • optictrace scan and review read them too. A payload is structured and can be masked by JSON path; a log line is free text. A leak detector that only reads payloads looks where the data is easiest to protect rather than where it escapes, so scan reports on both and the suggested fix differs — a pattern or a field name under app_logs.redact, not a json_fields path that could not work here. The count is reported as "N record(s) and M log line(s)", because no findings over zero lines means something very different from no findings over forty thousand.
  • Storage support is optional. ext.AppLogStore is a separate interface from ext.Store, so a third-party driver that does not implement it is still a complete driver — but a driver that DOES implement it must also erase log lines in Purge, and ext/exttest asserts exactly that. All three built-in drivers (SQLite, Postgres, ClickHouse) pass it.

Multi-tenant tagging

One API, many tenants, the same path for all of them. Tags turn that into something you can segregate, meter and bill.

rules:
  # Baseline: every API call gets a tenant, a region and a default tier.
  - name: tag-baseline
    match: { path: "/api/**" }
    labels:
      tenant: "header:X-Tenant-ID"
      region: "header:X-Region|^([a-z]{2})-"   # eu-west-1 -> eu
      tier:   "static:standard"

  # Criteria: gold and platinum plans are tagged premium instead.
  - name: tag-premium
    match:
      path: "/api/**"
      headers:
        X-Plan: "^(gold|platinum)$"            # regex
    labels:
      tier: "static:premium"

  # Tenant carried in the URL rather than a header.
  - name: tag-tenant-from-path
    match: { path: "/api/v1/tenants/*/**" }
    labels:
      tenant: "path:4"                         # 1-indexed segment

Same endpoint, segregated:

/api/orders                     tenant=acme     tier=premium   region=eu
/api/orders                     tenant=globex   tier=standard  region=us
/api/orders?mode=sandbox        tenant=acme     tier=standard  env=sandbox
/api/v1/tenants/umbrella/orders tenant=umbrella tier=standard  region=eu
When the discriminator is in the payload

The hard case is two callers that are identical from the outside: same endpoint, same tenant, same product, differing only in a field of the body — a lead API called by several partners, for instance.

rules:
  - name: lead-attribution
    match: { path: "/api/v1/leads" }
    redact:
      json_fields: ["$.**.phone", "$.**.email"]
    labels:
      partner: "json:$.**.source"            # flipkart | samsung | direct
      channel: "json:$.**.channel"
      lead_id: "json_response:$.lead_id"     # from the RESPONSE

  - name: marketplace-callers
    match:
      path: "/api/v1/leads"
      body: { "$.**.source": "^(flipkart|amazon)$" }
    labels:
      channel_type: "static:marketplace"

partner=flipkart channel=app    type=marketplace product=personal-loan lead=LD-111
partner=samsung  channel=retail type=oem         product=personal-loan lead=LD-113
partner=direct   channel=web    type=-           product=personal-loan lead=LD-109

Then /api/logs?label.partner=samsung&label.product=credit-card, or /api/usage?label=partner to bill each one.

Body labels are extracted after redaction, never before. Otherwise labels: {email: "json:$.**.email"} would copy a redacted value straight into a Prometheus dimension and the stored labels map, routing around the rule next to it. Config validation refuses the overlap outright rather than leaving you with a label that reads [REDACTED]:

✗ rule leak-attempt: labels.who reads $.**.email, which another rule redacts —
  the label would be "[REDACTED]", and using redacted data as a dimension is
  what redaction exists to prevent

Only routes carrying a body rule buffer a body, so a config without one pays nothing.

Label sources
Source Example Value
header:<Name> header:X-Tenant-ID the request header
query:<name> query:tenant a query parameter
path:<n> path:4 the 4th path segment, 1-indexed
static:<value> static:premium a constant — this is how you tag
json:<path> json:$.**.source a field of the governed request body
json_response:<path> json_response:$.lead_id a field of the governed response body

Any source takes an optional |<regex> suffix with exactly one capture group, and that group becomes the label. A non-match yields an empty label rather than the raw value, so a mistyped pattern produces a missing tag rather than a misleading one.

Criteria

match.headers, match.query and match.body take regular expressions (match.body keys are JSON paths). All listed conditions must hold, so adding one narrows the rule. Patterns are unanchored like Go's regexp — write ^ and $ for a whole-value match, and "." for "present and non-empty".

There is no separate tags: block, deliberately. Rules already merge top to bottom with later rules winning, so a broad default plus a narrow override is all conditional tagging needs — and it reuses machinery that already governs redaction, sampling and metering rather than adding a second thing to learn.

What tags do for you
  • Filter the inspector — ?label.tenant=acme&label.tier=premium on /api/logs and /api/export. Multiple labels are an AND, and values match literally, so a tenant named acme_1 never selects acmeX1.
  • Prometheus dimensions — optictrace_requests_total{tenant="acme",tier="premium"}
  • Cost attribution — /api/usage?label=tier groups by any tag, not just tenant
  • Erasure requests — optictrace purge -label tenant -value acme
  • Rules can then target a tagged class for redaction or sampling

Values are client-controlled, so they pass through the cardinality guard (telemetry.metrics.max_label_values) before becoming Prometheus labels.

Framework SDKs

SDKs evaluate the same optic.yaml in-process and POST governed records to the agent's /api/ingest — one dashboard and one metrics endpoint across your whole stack. Sensitive values never cross a process boundary in the clear: redaction happens inside the service that saw the data.

Point the agent at no service.listen and no service.upstream and it runs in collector mode — the admin API, store and metrics, with no proxy, because the SDK is already in the request path:

service:
  name: shop        # no listen, no upstream
telemetry:
  admin_listen: "127.0.0.1:9095"
  store: { driver: sqlite, dsn: shop.db }
Express FastAPI Java / Servlet Gin / net-http
Restriction + redaction ✅ ✅ ✅ ✅
Query-param redaction ✅ ✅ ✅ ✅
Labels — all six sources, |regex capture ✅ ✅ ✅ ✅
Meters ✅ ✅ ✅ ✅
Trace correlation ✅ ✅ ✅ ✅
Tail-based keep_errors / keep_slower_than ✅ ✅ ✅ ✅
Application logs ✅ ✅ ✅ ✅

Gin and net/http route through the same Go interceptor as the proxy, so they inherit everything it does. Every SDK's suite can assert that a live agent accepts what it produces — set OPTIC_AGENT_URL and run it that way in CI, because offline tests cannot catch a record the agent rejects. The Java SDK also ships a generated javax.servlet variant for Spring Boot 2, compiled and run in CI.

Node.js / Express
const optictrace = require('@optictrace/express');
app.use(optictrace({ configPath: 'optic.yaml', agentUrl: 'http://localhost:9095' }));

// Your logs, filed under the request that wrote them
const logs = new optictrace.LogShipper('http://localhost:9095', 'checkout');
logs.info('order received', { sku: 'SKU-100' });

// Calls this service makes downstream
await fetch(url, { headers: optictrace.outboundHeaders() });
Python / FastAPI
from optictrace_fastapi import OpticTraceMiddleware, OpticTraceLogHandler, outbound_headers

app.add_middleware(OpticTraceMiddleware,
                   config_path="optic.yaml", agent_url="http://localhost:9095")

# Your ordinary logging, filed under the request that wrote it. Nothing at the
# call site needs to know about OpticTrace — the span comes from a ContextVar
# the middleware sets.
logging.getLogger().addHandler(OpticTraceLogHandler("http://localhost:9095"))

# Calls this service makes downstream, carrying THIS hop's span so they nest
# under it instead of becoming siblings.
await client.get(url, headers=outbound_headers())

examples/python-shop is a working three-service FastAPI application built on this — real HTTP calls between services, logs correlated per span, and a 25-assertion verification suite.

Java / Jakarta Servlet

Spring Boot 3, Quarkus, Jetty, Tomcat — anything on Jakarta Servlet 5+.

@Bean
FilterRegistrationBean<OpticTraceFilter> optictrace() throws IOException {
    return new FilterRegistrationBean<>(
        new OpticTraceFilter("optic.yaml", "http://localhost:9095", "checkout"));
}

// Your logs, filed under the request that wrote them
Logger.getLogger("").addHandler(
    new OpticTraceLogHandler("http://localhost:9095", "checkout"));

// Calls this service makes downstream
TraceContext.outboundHeaders().forEach(builder::header);
Go / net-http + Gin
agent, _ := optictrace.New("optic.yaml")
defer agent.Close()
agent.ServeAdmin("ui/out")                            // metrics + dashboard on :9095

http.ListenAndServe(":8080", agent.Middleware(mux))   // net/http
r.Use(optictracegin.Middleware(agent))                // Gin

// slog, correlated to the request being served
logger := slog.New(optictrace.NewLogHandler("http://localhost:9095", "checkout", nil))
logger.InfoContext(ctx, "charge captured", "amount", 129.0)

// and the headers for a downstream call
for k, v := range optictrace.OutboundHeaders(ctx) { req.Header.Set(k, v) }

Use slog's ...Context variants: plain logger.Info() passes context.Background(), which carries no span, so those lines arrive as orphans the agent drops by default.

Developer dashboard

ui/ is a Next.js (App Router) + Tailwind + Recharts app, statically exported and served by the agent itself — no separate frontend deployment.

Request Inspector showing a captured payment with card number, CVV, email and Authorization header all masked, the matched rule named, and non-sensitive fields left intact

The Inspector, mid-investigation: redacted values are highlighted, the rule responsible is named, and everything non-sensitive is still there to debug with.

Page Shows
Overview Golden signals; p95 drawn against the average; succeeded/rejected/failed split apart; traffic by tenant; capture & sampling; top routes; which rules are firing
Routes Every route with sortable P50/P95/P99, error rates, traffic volume
Traces One row per request however many services it touched, then a waterfall on a shared timeline — every hop, the operations inside each hop, and the log lines each wrote — with self time per hop
Inspector Searchable/filterable exchanges; redacted fields highlighted; CSV/JSONL export
Logs Application log lines across requests — filter by level, service or text; every line links back to the request that wrote it
Usage Per-consumer requests, data, compute, meters and estimated cost
Governance Each rule's actions (restrict/redact/labels/sample/meter) with live match counts
Config View optic.yaml, lint edits live against the running agent, trigger hot reload
System Agent health, store size, per-exporter delivery/failure/drop counters

Two panels answer questions no other page can:

  • Latency shows p95 next to the average, and the gap is the reading. A mean over a bucket hides the handful of 3s responses that are the actual problem — an incident can triple the p95 while barely moving the average.
  • Capture & sampling reports how many records kept a body. It is the only honest check that a sampling rule is doing what you think: a rule sampling at 0.05 that matches nothing looks identical to one at 1.0 in every other number on the page.
More screenshots

Traces — a failed checkout as a waterfall, with the log lines each hop wrote

Overview — golden signals, p95 against the average, and what sampling actually did

Logs — application lines across requests, each linking back to its own request

Routes — every route with sortable P50/P95/P99

Usage — per-tenant consumption, meters and estimated cost

Governance — each rule's actions with live match counts

The inspector filters by tag, badges long-lived streams, and reconstructs the full request trace for any record — click a hop to jump to it. Usage groups by any tag, not just the billing consumer.

The Traces page needs a store driver that can group by trace id — ext.TraceStore, an optional companion to ext.Store in the same way ext.AppLogStore is. The bundled sqlite, postgres and clickhouse drivers all implement it; a third-party driver that does not is still a complete driver, and loses the listing rather than correlation (every record still carries its trace and span ids, and the Inspector reassembles a trace from any hop in it).

One thing worth knowing if you build on the API: record.time is when the exchange finished, not when it started. Subtract duration_ms for the start — a waterfall built on time directly draws the parent beginning after the children it called, because the parent is the last hop to finish.

Develop it with cd ui && npm run dev. The dev server runs on a different port, so add its origin to telemetry.cors_origins (e.g. ["http://localhost:3001"]) — the agent sends no CORS headers unless an origin is explicitly allowed.

Deployment

  • Docker — multi-stage Dockerfile (UI build → pure-Go build → non-root Alpine image).
  • Compose — docker-compose.yml runs OpticTrace + demo upstream + Prometheus + Grafana, with the dashboard and alert rules provisioned:
The provisioned Grafana dashboard showing request rate, error rate, latency percentiles and agent health
  • Helm — deploy/helm/optictrace with ConfigMap-managed optic.yaml, health probes, optional PVC and ServiceMonitor.

Measured overhead

The claim "built for the hot path" deserves numbers rather than adjectives. go test ./internal/proxy -bench=. -benchmem -run='^$' compares a bare handler against the same handler wrapped by the interceptor:

Policy ns/op Added vs. baseline allocs/op
Baseline (no OpticTrace) 2,050 — 25
Restricted route (capture off) 2,267 +0.22 µs 29
Full capture + redaction 7,434 +5.4 µs 211
…plus Prometheus with a custom label 7,567 +5.5 µs 215
Rule evaluation alone (no HTTP) 812 — 7

12th Gen Intel i5-1235U, Go 1.25, -benchtime=2s, parallel. Reproduce with make bench.

Read the absolute deltas, not the ratios: the baseline includes httptest request construction, which inflates it and flatters the percentages. What the numbers say:

  • Restricting a route really is near-free (+0.22 µs). The policy resolves before any buffer is attached, so a route you've told OpticTrace to leave alone costs almost nothing — this is the design claim, and it holds.
  • Full capture with depth-recursive redaction costs about 5.4 µs per request. Against a typical API call of 1–100 ms that's 0.005–0.5% of the request, but it is not free, and it's dominated by JSON parse + re-serialize. Use sample with keep_errors on very hot routes.
  • Prometheus observation is noise (+0.13 µs) even with a custom label dimension.

Verified behavior

These aren't design intentions — each was observed end to end with real traffic through the proxy, not asserted from the code:

  • Redaction holds under echo. A payment request carrying a card number came back echoed inside a wrapper by the upstream; both the request and the nested response copy stored [REDACTED], while the client received the real number.
  • Restriction is total. A login request's password and the response token appear nowhere in logs, store, or exports — only method, path, status and latency were recorded.
  • Metering survives restriction. An AI route with both bodies restricted still recorded tokens: 63 attributed to a tenant, with the completion text absent everywhere.
  • Labels reach Prometheus. Series carried tenant="acme" and region="ap-south-1" as real dimensions, with routes normalized to /api/v1/users/:id.
  • The linter catches real breakage. A proposed spec dropping a field was rejected with "clients send request field credit_card (28 times, last 22m ago)" and a non-zero exit.
  • Plugins receive governed data only. A Python CSV plugin and a file exporter both ran live; neither output contained a card number or a restricted payload.
  • A clean clone works. Cloned fresh from GitHub: builds, six test packages pass, CLI runs.

Start from your OpenAPI spec

Writing governance by hand against an API you may not have written means finding out what you missed once traffic flows. A specification already lists the routes and payload shapes, so most of the first draft can be derived:

optictrace init -spec openapi.yaml -out optic.yaml

Reads OpenAPI 3.x and Swagger 2.0, YAML or JSON, and produces rules for what the document actually states — credential headers from its securitySchemes, metadata-only capture on /login-shaped routes, and redact.json_fields for payload fields whose names are unambiguous, each annotated with why:

  - name: redact-api-v1-payments-charge
    match:
      path: "/api/v1/payments/charge"
    redact:
      query_params: ["api_key"]
      json_fields:
        - "$.**.card.number"    # high · payment card data is PCI-DSS scope
        - "$.**.card.cvv"       # high · payment card data is PCI-DSS scope
        - "$.**.customer.email" # medium · email addresses are personal data

It is a starting point and says so in its own header. A spec describes what an API claims; governance has to hold for what it does. A field the document does not model cannot be masked by a rule derived from it, specs drift, and a field called ref can hold a card number — which is what optictrace scan finds on real traffic and no name heuristic ever will. Caveats print to stderr, so init -spec x.yaml > optic.yaml still gives you a clean file.

It refuses to overwrite an existing optic.yaml, and validates what it produced before handing it over.

Examples

Both are full applications with their own traffic drivers and assertion suites, not snippets — they exist so the claims above can be checked rather than taken on trust.
examples/python-shop Three FastAPI services making real HTTP calls to each other, governed in-process by the SDK, reporting into one agent in collector mode. Shows trace correlation across services, application logs filed under the request that wrote them, redaction of a card number a service logs on purpose, and metering for billing. ./run.sh then verify.py — 25 assertions. Optional Prometheus + Grafana.
examples/lead-pipeline Three Go services behind sidecars, with the discriminator in the payload rather than a header — the multi-tenant tagging case. 19 assertions.
examples/memstore A third-party ext.Store driver in a separate module, proving the extension surface works from outside the repo. Runs the same conformance suite as the built-in drivers.
examples/adminauth An authentication/authorization extension against the ext.Authenticator hooks — the template for SSO.

Roadmap

Ordered by value, not by ease. Each item states the problem it solves — if the rationale does not hold for your use case, the item should not be built.

Everything originally on this roadmap has shipped. Tiers 1–3 (the leak detector, tail-based sampling, rule unit tests, query capture, retention and erasure, admin auth, the Postgres driver, OTLP export, replay, suggestions, the fleet view, Grafana and release engineering) landed by v0.7.0; see the changelog for what each release since then added. What follows is what is not done.

Next — ordered

  1. Publish the SDKs to their registries. The agent is on Homebrew, go install and ghcr.io. None of the SDKs are on npm, PyPI or Maven Central, so every integration starts with git clone — which is a real adoption tax and the reason the integration guide has to spell out three different install routes. The Java package namespace (io.github.dwarka-prasad) is already chosen for Central's verification rules. Blocked on publishing credentials, not on code.

  2. Governance metrics and alerts. Prometheus carries traffic and agent health but nothing about the policy: no series for redactions applied, rules matched, or scan findings. That means the one failure mode unique to this tool — governance silently stopping — is the one thing you cannot alert on. A rule that matches nothing is already surfaced on the dashboard; it should be able to page someone.

  3. A scheduled scan that opens an issue. optictrace scan runs in CI against a diff. Production traffic is where new fields actually appear, and nobody remembers to run a scan by hand. A scheduled job against a live agent, filing an issue on a new finding, turns the leak detector from something you invoke into something that tells you.

  4. optictrace test -from-traffic. Rule tests are hand-written fixtures today, so they test the payloads someone thought of. Generating cases from captured traffic — masked, so the fixture is safe to commit — would pin the behaviour against payloads the API really produced.

  5. Subject-access export. purge answers erasure requests; the other half of the same regulation is "give me a copy of what you hold about me", and today that means hand-writing a query. It is the same label filter and the same store scan as purge, in the opposite direction.

  6. Express application-log parity. The Java, FastAPI and Go SDKs correlate log lines with no call-site changes (a handler on the logging framework, span from a thread-local or context). Express requires an explicit LogShipper because Node has no ambient logger to hook — worth closing with an AsyncLocalStorage-based adapter for pino and winston.

  7. OTLP ingest. OpticTrace exports to OTLP; it cannot receive it. Accepting OTLP would let a service already instrumented with an OpenTelemetry SDK send spans here and have governance applied to them, without adopting a second SDK.

Deliberately not planned
  • gRPC interception. Bodies are length-prefixed protobuf; without descriptors there is nothing to redact or meter, so a proxy could only count bytes. Use the SDK middleware inside the service, where the messages are typed.
  • A rules UI. The config is the interface, and it is reviewable in a pull request. A form that generates YAML makes the file the second source of truth.
  • Automatic instrumentation of database drivers. Wrapping a driver per language, per version, is a maintenance surface larger than the rest of this project — and a wrapper that silently stops matching a driver's internals is worse than no wrapper. Inner spans are named explicitly; Go's Transport wrapper exists because an http.RoundTripper is one stable interface, not twenty.
  • Sampling that drops records. Sampling gates the stored body; the record is always written. A tool whose own counts move when you change a sampling rate cannot be used to answer questions about traffic.
Known gaps

Documented so nobody discovers them the hard way:

  • The AI mock path is implemented but has never run against the live Anthropic API.
  • Hijacked connections (WebSockets) pass through; once upgraded, the bytes are between client and upstream, so the record covers the exchange up to the 101. Their duration is a connection lifetime, so they are marked as streams and kept out of latency percentiles.
  • Streaming responses (SSE, chunked) reach the client as they are produced, but telemetry is emitted when the stream closes — a long-lived stream is invisible until then, apart from optictrace_streams_open.
  • Admin authentication is available but off by default for the binary; enable telemetry.auth if the port is reachable. The Helm chart turns it on by default and generates a token, because inside a cluster the admin port is reachable by anything that can resolve the Service.
  • record.time is when an exchange finished, not when it started. Subtract duration_ms for the start — a timeline built on it directly draws a parent beginning after the children it called.
  • Trace listing needs a store driver implementing the optional ext.TraceStore. All three bundled drivers do; a third-party driver without it loses the Traces page, not correlation.

Contributing

Bug reports, rule-engine edge cases, and new SDK targets are all welcome — see CONTRIBUTING.md. Ground rules worth repeating:

  1. Governance invariants are non-negotiable. Proxied traffic is never mutated; telemetry never blocks a request. Any PR that could leak a restricted or redacted value needs a test proving it doesn't.
  2. The rule engine is portable. internal/engine (Go), sdks/express/engine.js, sdks/fastapi/.../engine.py and sdks/java/.../Engine.java must stay semantically identical. A source one engine cannot resolve returns an EMPTY value, never a guessed one.
make setup      # dependencies, dashboard, binaries
make dev        # full local stack + seeded traffic, dashboard on :9095
make test-all   # every module, including the satellite ones
make help       # every target

Project layout

cmd/optictrace/     agent binary (run · init · validate · test · scan · suggest ·
                    review · purge · replay · spec · check · sdk · mock · version)
internal/config/    optic.yaml schema + strict validation
internal/engine/    compiled rule engine: globs, policy merge, redaction, meters
internal/proxy/     interception (reverse proxy + embeddable middleware)
internal/metrics/   Prometheus collector with dynamic label schemas
internal/store/     LogStore interface, SQLite · Postgres · ClickHouse drivers,
                    async writer, usage aggregation, trace rollups
internal/export/    output plugins: file · webhook · command (custom executables)
internal/scan/      leak detector: structural detectors, masked findings, fix suggestions
internal/suggest/   name-based rule proposals (complements scan's value checks)
internal/review/    pull-request review: policy diff, coverage score, Markdown report
internal/replay/    re-issue captured traffic against a target
internal/ruletest/  optic.test.yaml runner (pure engine, no server)
internal/spec/      traffic→OpenAPI inference, spec-vs-traffic linter, TS SDK gen
internal/mock/      stateful mock server (+ optional Claude-generated responses)
internal/applog/    application-log governance: level floor, caps, redaction
internal/spans/     inner-span governance: attribute redaction, caps, duration floor
internal/telcap/    the per-request cap and UTF-8 truncation both of them share
internal/scaffold/  optic.yaml generation from an OpenAPI or Swagger document
internal/admin/     admin API + dashboard hosting
ui/                 Next.js dashboard (static export)
ext/                published extension surface: Store · Exporter · auth hooks,
                    plus the optional AppLogStore and TraceStore companions
sdks/               express · fastapi · java · gin
deploy/             compose bits, Helm chart, Grafana dashboard, Prometheus alerts
examples/           exporter plugins, CI workflows, a traffic fixture, and two
                    runnable apps: python-shop (FastAPI) and springboot-shop (Java)

Changelog

See CHANGELOG.md for release history.

License

Apache 2.0

Documentation

Overview

Package optictrace is the public embedding API: drop config-driven API telemetry and governance into any Go HTTP service. 

agent, err := optictrace.New("optic.yaml")
if err != nil { log.Fatal(err) }
defer agent.Close()
http.ListenAndServe(":8080", agent.Middleware(mux))

The agent optionally serves its own admin endpoint (metrics, dashboard, log APIs) on telemetry.admin_listen when started with ServeAdmin.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func OutboundHeaders added in v0.10.0 

func OutboundHeaders(ctx context.Context) map[string]string

OutboundHeaders returns the headers to attach to a call this service makes downstream, so the next hop nests under this one.

Carries THIS hop's span, not the caller's — forwarding the inbound header unchanged would make every downstream call a sibling of this request rather than a child, and the tree flattens into a list.

func SpanFromContext added in v0.10.0 

func SpanFromContext(ctx context.Context) (traceID, spanID string, ok bool)

SpanFromContext returns the trace and span ids of the request being served, for code that wants to correlate something OpticTrace does not do for it — an outbound call, a queue message, a row written for audit.

Returns false outside a request. Startup and background work belong to no request, and inventing one for them would attribute their output to whichever request happened to be in flight.

Types

type Agent 

type Agent struct {
	// contains filtered or unexported fields
}

Agent bundles the rule engine, telemetry sinks, and optional admin server.

func New 

func New(configPath string, opts ...AgentOption) (*Agent, error)

New loads optic.yaml and assembles the telemetry pipeline (metrics collector and async payload store per the config's telemetry block).

func (*Agent) AdminHandler 

func (a *Agent) AdminHandler(uiDir string) http.Handler

AdminHandler exposes /metrics, the dashboard, and query APIs for mounting on a listener you control.

func (*Agent) Close 

func (a *Agent) Close() error

Close drains the telemetry queue, flushes exporters, and releases resources.

func (*Agent) Middleware 

func (a *Agent) Middleware(next http.Handler) http.Handler

Middleware wraps an http.Handler with interception (embedded mode).

func (*Agent) Reload 

func (a *Agent) Reload() error

Reload re-reads optic.yaml, atomically swaps the rule engine, and re-points the metrics label schema. Settings that cannot be hot-swapped are reported rather than silently ignored — see Config.RestartRequired.

func (*Agent) ServeAdmin 

func (a *Agent) ServeAdmin(uiDir string)

ServeAdmin starts the admin server on telemetry.admin_listen in a background goroutine.

type AgentOption 

type AgentOption func(*Agent)

AgentOption customizes construction.

func WithLogger 

func WithLogger(l *slog.Logger) AgentOption

WithLogger overrides the default JSON stdout logger.

type InnerSpan added in v0.15.0 

type InnerSpan struct {
	// contains filtered or unexported fields
}

InnerSpan is one operation in flight. Not safe for concurrent use by several goroutines — one span is one operation, and an operation shared across goroutines is two operations.

func (*InnerSpan) End added in v0.15.0 

func (s *InnerSpan) End()

End closes the span and queues it. Calling End twice is a no-op rather than a double count: a `defer sp.End()` alongside an explicit one in the happy path is a natural thing to write.

func (*InnerSpan) Fail added in v0.15.0 

func (s *InnerSpan) Fail(err error) *InnerSpan

Fail marks the operation as failed. A nil error is a no-op, so `defer sp.Fail(err)` cannot be written by accident — use End for the normal path and Fail before it when there is an error.

A failed operation survives the min_duration filter: "it returned in 200µs" and "it returned in 200µs with an error" are not the same event, and the second is the one someone is looking for.

func (*InnerSpan) Set added in v0.15.0 

func (s *InnerSpan) Set(key, value string) *InnerSpan

Set attaches an attribute. Conventional keys — db.statement, db.rows, cache.key, cache.hit, http.method, http.url, http.status — are what the dashboard reads.

func (*InnerSpan) SetInt added in v0.15.0 

func (s *InnerSpan) SetInt(key string, value int64) *InnerSpan

SetInt is Set for a numeric attribute, so a row count does not need formatting at every call site.

type LogHandler added in v0.10.0 

type LogHandler struct {
	// contains filtered or unexported fields
}

LogHandler is an slog.Handler that ships log records to OpticTrace, correlated to the span serving them. 

logger := slog.New(optictrace.NewLogHandler("http://localhost:9095", "checkout", nil))

Nothing at the call site changes: the span comes from the context slog already passes, so an ordinary logger.InfoContext(ctx, ...) is filed against the exact request that produced it.

Use InfoContext/ErrorContext (the ...Context variants). slog's plain Info() passes context.Background(), which carries no span — those lines are still shipped, but as orphans the agent will drop by default.

func NewLogHandler added in v0.10.0 

func NewLogHandler(agentURL, service string, opts *LogHandlerOptions) *LogHandler

NewLogHandler builds a handler shipping to the agent's app-log endpoint.

func (*LogHandler) Close added in v0.10.0 

func (h *LogHandler) Close() error

Close drains the queue. The last lines before a shutdown are usually the ones explaining it.

func (*LogHandler) Enabled added in v0.10.0 

func (h *LogHandler) Enabled(_ context.Context, level slog.Level) bool

func (*LogHandler) Handle added in v0.10.0 

func (h *LogHandler) Handle(ctx context.Context, r slog.Record) error

func (*LogHandler) Stats added in v0.10.0 

func (h *LogHandler) Stats() (sent, failed, dropped int64, lastErr error)

Stats reports delivery so "is my telemetry actually arriving?" has an answer.

func (*LogHandler) WithAttrs added in v0.10.0 

func (h *LogHandler) WithAttrs(attrs []slog.Attr) slog.Handler

func (*LogHandler) WithGroup added in v0.10.0 

func (h *LogHandler) WithGroup(name string) slog.Handler

type LogHandlerOptions added in v0.10.0 

type LogHandlerOptions struct {
	Level    slog.Leveler
	MaxQueue int           // bounded, so a logging storm drops visibly instead of growing
	Flush    time.Duration // batching interval
	Timeout  time.Duration
}

LogHandlerOptions configures a LogHandler. A nil value uses the defaults.

type SpanOptions added in v0.15.0 

type SpanOptions struct {
	MaxQueue int           // bounded, so a burst drops visibly instead of growing
	Flush    time.Duration // batching interval
	Timeout  time.Duration
}

SpanOptions configures a SpanRecorder. A nil value uses the defaults.

type SpanRecorder added in v0.15.0 

type SpanRecorder struct {
	// contains filtered or unexported fields
}

SpanRecorder ships inner spans to the agent. 

spans := optictrace.NewSpanRecorder("http://localhost:9095", "checkout", nil)
defer spans.Close()

ctx, sp := spans.Start(ctx, "db.query", "db")
sp.Set("db.statement", "SELECT * FROM orders WHERE id = $1")
defer sp.End()

Shipping is fire-and-forget on a background worker: an application must never be slower, or fail, because its telemetry sink is unhappy.

func NewSpanRecorder added in v0.15.0 

func NewSpanRecorder(agentURL, service string, opts *SpanOptions) *SpanRecorder

NewSpanRecorder builds a recorder shipping to the agent's span endpoint.

An empty agentURL yields a recorder that does nothing — so instrumentation can be left in place in an environment with no agent, rather than guarded by an `if` at every call site.

func (*SpanRecorder) Close added in v0.15.0 

func (r *SpanRecorder) Close() error

Close flushes anything queued. Worth calling on shutdown: the last few spans of the last few requests are usually the interesting ones.

func (*SpanRecorder) Observe added in v0.15.0 

func (r *SpanRecorder) Observe(ctx context.Context, name, kind string, fn func(context.Context) error) error

Observe runs fn as a span, which is the shape most call sites want: no defer, no way to forget End, and the error is recorded automatically. 

err := spans.Observe(ctx, "db.query", "db", func(ctx context.Context) error {
    return db.QueryRowContext(ctx, q).Scan(&n)
})

func (*SpanRecorder) Start added in v0.15.0 

func (r *SpanRecorder) Start(ctx context.Context, name, kind string) (context.Context, *InnerSpan)

Start opens a span for an operation and returns a context that nests anything started inside it.

kind classifies the operation for the waterfall and the breakdown: db, cache, http, queue, rpc, internal. Outside a request the span has no parent and the agent drops it by default, which is deliberate — work that belongs to no request cannot be attributed to one.

func (*SpanRecorder) Stats added in v0.15.0 

func (r *SpanRecorder) Stats() (sent, failed, dropped int64, lastErr error)

Stats reports delivery, so "are my spans actually arriving?" has an answer rather than a guess.

func (*SpanRecorder) Transport added in v0.15.0 

func (r *SpanRecorder) Transport(base http.RoundTripper) http.RoundTripper

Transport wraps an http.RoundTripper so every outbound call is recorded as a span AND carries this hop's traceparent.

Those two belong together: a downstream call that is timed but not propagated shows up as a gap someone has to guess about, and one that is propagated but not timed leaves the caller's own view of it missing. 

client := &http.Client{Transport: spans.Transport(nil)}

Source Files

View all Source files

Directories

Path Synopsis
Package cli is the optictrace command line, exposed as a package so a binary built elsewhere can be optictrace plus something extra rather than a reimplementation of it.
 Package cli is the optictrace command line, exposed as a package so a binary built elsewhere can be optictrace plus something extra rather than a reimplementation of it.
cmd
mocktarget command
mocktarget is a throwaway upstream used to exercise the OpticTrace proxy locally.
 mocktarget is a throwaway upstream used to exercise the OpticTrace proxy locally.
optictrace command
Command optictrace is the OpticTrace agent and toolbox.
 Command optictrace is the OpticTrace agent and toolbox.
examples
lead-pipeline/bureausvc command
Command bureausvc stands in for a credit bureau — the leaf of the pipeline and a third-party in real life, which is why its response deliberately contains data you would not want in your telemetry.
 Command bureausvc stands in for a credit bureau — the leaf of the pipeline and a third-party in real life, which is why its response deliberately contains data you would not want in your telemetry.
lead-pipeline/leadsvc command
Command leadsvc is the entry point of the demo lead pipeline: it accepts a lead, asks the scoring service to grade it, and returns a decision.
 Command leadsvc is the entry point of the demo lead pipeline: it accepts a lead, asks the scoring service to grade it, and returns a decision.
lead-pipeline/scoringsvc command
Command scoringsvc grades a lead, calling the bureau for credit history.
 Command scoringsvc grades a lead, calling the bureau for credit history.
ext
Package ext is OpticTrace's extension surface: the contract another Go module implements to add a payload store or an output exporter.
 Package ext is OpticTrace's extension surface: the contract another Go module implements to add a payload store or an output exporter.
exttest
Package exttest is the acceptance suite for an ext.Store implementation.
 Package exttest is the acceptance suite for an ext.Store implementation.
internal
admin
Package admin serves OpticTrace's control-plane HTTP surface, deliberately on a separate listener from proxied traffic so it can be firewalled independently:
 Package admin serves OpticTrace's control-plane HTTP surface, deliberately on a separate listener from proxied traffic so it can be firewalled independently:
applog
Package applog governs application log lines on the way into the store.
 Package applog governs application log lines on the way into the store.
config
Package config defines the optic.yaml schema, its loader, and validation.
 Package config defines the optic.yaml schema, its loader, and validation.
engine
Package engine compiles an optic.yaml Config into an immutable, allocation- light rule engine evaluated on every request.
 Package engine compiles an optic.yaml Config into an immutable, allocation- light rule engine evaluated on every request.
export
Package export is OpticTrace's pluggable output layer: every governed telemetry record (post-restriction, post-redaction — exporters can never see raw sensitive data) fans out to the exporters declared under telemetry.exporters in optic.yaml.
 Package export is OpticTrace's pluggable output layer: every governed telemetry record (post-restriction, post-redaction — exporters can never see raw sensitive data) fans out to the exporters declared under telemetry.exporters in optic.yaml.
metrics
Package metrics exposes OpticTrace telemetry to Prometheus.
 Package metrics exposes OpticTrace telemetry to Prometheus.
mock
Package mock turns an OpenAPI spec into a running, STATEFUL mock server:
 Package mock turns an OpenAPI spec into a running, STATEFUL mock server:
proxy
Package proxy provides OpticTrace's interception layer in two flavors that share one code path:
 Package proxy provides OpticTrace's interception layer in two flavors that share one code path:
replay
Package replay re-issues captured traffic against a target service.
 Package replay re-issues captured traffic against a target service.
review
Package review turns everything OpticTrace knows into one pull-request comment.
 Package review turns everything OpticTrace knows into one pull-request comment.
ruletest
Package ruletest runs assertions about optic.yaml against the real rule engine, with no server and no network.
 Package ruletest runs assertions about optic.yaml against the real rule engine, with no server and no network.
scaffold
Package scaffold turns an OpenAPI or Swagger document into a starting optic.yaml.
 Package scaffold turns an OpenAPI or Swagger document into a starting optic.yaml.
scan
Package scan is OpticTrace's safety net.
 Package scan is OpticTrace's safety net.
spans
Package spans governs inner spans on the way into the store.
 Package spans governs inner spans on the way into the store.
spec
Package spec bridges static API contracts and runtime reality:
 Package spec bridges static API contracts and runtime reality:
store
Package store persists governed telemetry records (what optic.yaml allowed through) and answers the dashboard's query/analytics needs.
 Package store persists governed telemetry records (what optic.yaml allowed through) and answers the dashboard's query/analytics needs.
suggest
Package suggest proposes governance rules for traffic that has none.
 Package suggest proposes governance rules for traffic that has none.
telcap
Package telcap holds the two mechanics every per-request telemetry stream needs: a memory-bounded per-key counter, and a truncation that respects UTF-8.
 Package telcap holds the two mechanics every per-request telemetry stream needs: a memory-bounded per-key counter, and a truncation that respects UTF-8.
tracectx
Package tracectx handles W3C Trace Context — the header that lets records from several services be recognised as one request.
 Package tracectx handles W3C Trace Context — the header that lets records from several services be recognised as one request.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.