braintrust

README

Braintrust Go SDK

Overview

This library provides tools for evaluating and tracing AI applications in Braintrust. Use it to:

• Evaluate your AI models with custom test cases and scoring functions
• Trace LLM calls and monitor AI application performance with OpenTelemetry
• Integrate seamlessly with OpenAI, Anthropic, Google Gemini, Genkit, ADK, CloudWeGo Eino, LangChainGo, and other LLM providers

This SDK is currently in BETA status and APIs may change.

Installation

go get github.com/braintrustdata/braintrust-sdk-go

export BRAINTRUST_API_KEY="your-api-key"  # Get from https://www.braintrust.dev/app/settings

Each tracing integration is published as its own Go module. Install only the ones you need:

go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai       # OpenAI (openai-go)
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/anthropic     # Anthropic
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/genai         # Google GenAI
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/genkit        # Firebase Genkit
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/adk           # Google ADK
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/cloudwego/eino # CloudWeGo Eino
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/langchaingo   # LangChainGo
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/github.com/sashabaranov/go-openai # sashabaranov/go-openai
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/bedrockruntime # AWS Bedrock Runtime
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/a2a          # A2A protocol

Or install all integrations at once with the meta-module:

go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/all

Instrumentation

Trace LLM calls with automatic or manual instrumentation.

Automatic Instrumentation

Use Orchestrion to automatically inject tracing at compile time—no code changes required.

1. Install orchestrion:

go install github.com/DataDog/orchestrion@v1.12.1

2. Create orchestrion.tool.go in your project root:

//go:build tools

package main

import (
    _ "github.com/DataDog/orchestrion"
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/all" // Dedicated meta-module for all Braintrust LLM integrations
)

Or import only the integrations you need:

import (
    _ "github.com/DataDog/orchestrion"
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai"    // OpenAI (openai-go)
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/anthropic" // Anthropic
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/genai"     // Google GenAI
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/genkit"    // Firebase Genkit
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/adk"       // Google ADK
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/cloudwego/eino" // CloudWeGo Eino
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/langchaingo" // LangChainGo
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/github.com/sashabaranov/go-openai" // sashabaranov/go-openai
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/bedrockruntime" // AWS Bedrock Runtime
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/a2a" // A2A protocol
)

3. Build with orchestrion:

# Build with orchestrion
orchestrion go build ./...

# Or configure GOFLAGS to use orchestrion automatically
export GOFLAGS="-toolexec='orchestrion toolexec'"
go build ./...

4. Initialize OpenTelemetry and Braintrust in your application:

import (
    "context"
    "log"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/sdk/trace"
    "github.com/braintrustdata/braintrust-sdk-go"
)

func main() {
    ctx := context.Background()

    // Set up OpenTelemetry tracer
    tp := trace.NewTracerProvider()
    defer tp.Shutdown(ctx)
    otel.SetTracerProvider(tp)

    // Initialize Braintrust (registers the exporter)
    _, err := braintrust.New(tp, braintrust.WithProject("my-project"))
    if err != nil {
        log.Fatal(err)
    }

    // Your LLM calls are now automatically traced
}

That's it! Your LLM client calls are now automatically traced. No middleware or wrapper code needed in your application.

Manual Instrumentation

If you prefer explicit control, you can add tracing middleware manually to your LLM clients. See the Manual Instrumentation Guide for detailed examples with OpenAI, Anthropic, Google Gemini, and other providers.

Span customizers

Register ordered, synchronous export hooks with braintrust.WithSpanCustomizers. Each config.SpanCustomizer has an optional OnSpanExport hook; an omitted hook is a no-op. Hooks receive completed OpenTelemetry sdktrace.ReadOnlySpan export snapshots, not live spans or provider responses. Embed the snapshot and override the fields you want to export, for example:

package main

import (
    "context"
    "log"

    braintrust "github.com/braintrustdata/braintrust-sdk-go"
    "github.com/braintrustdata/braintrust-sdk-go/config"
    "go.opentelemetry.io/otel/attribute"
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

type redactedSpan struct {
    sdktrace.ReadOnlySpan
    attrs []attribute.KeyValue
}

func (s redactedSpan) Attributes() []attribute.KeyValue { return s.attrs }

func redact(span sdktrace.ReadOnlySpan) (sdktrace.ReadOnlySpan, error) {
    attrs := make([]attribute.KeyValue, 0, len(span.Attributes()))
    for _, attr := range span.Attributes() {
        if attr.Key != "braintrust.input_json" && attr.Key != "braintrust.output_json" {
            attrs = append(attrs, attr)
        }
    }
    return redactedSpan{ReadOnlySpan: span, attrs: attrs}, nil
}

func main() {
    tp := sdktrace.NewTracerProvider()
    defer tp.Shutdown(context.Background())
    client, err := braintrust.New(tp,
        braintrust.WithSpanCustomizers(config.SpanCustomizer{OnSpanExport: redact}),
    )
    if err != nil {
        log.Fatal(err)
    }
    _, span := client.Tracer("example").Start(context.Background(), "redacted")
    span.SetAttributes(attribute.String("braintrust.input_json", `"private"`))
    span.End()
}

• Repeated options append in execution order. Each hook receives its predecessor's result. Registration copies the list and hook functions; exporter construction takes another snapshot. Changes to caller-owned slices do not change existing exporters. Mutable state captured by callbacks is still your responsibility. There is no environment-variable registration.
• Hooks apply to all spans reaching the Braintrust exporter, including manual and instrumented spans, after filtering and span-origin metadata, but before automatic attachment conversion/uploads and OTLP serialization. Customizers see inline attachment data; removing or redacting it prevents its upload. Customization finishes for the entire batch before attachment processing begins. Only attachments remaining on successfully customized spans are processed; stripped hook-failure records upload no attachments. This also applies with lazy authentication or a custom exporter. Other span processors and application-visible values are unchanged. Separately enabled trace-console logging is not customized.
• Return the original snapshot or a valid replacement. Replacements are not merged; retain all fields you want to export. Returning nil (including a typed nil) is an error, not a way to drop spans. Attribute removal removes it from this completed export, not from any data previously exported.
• The trace ID, span ID, parent trace ID, and parent span ID must remain identical, including zero/invalid IDs for root spans. They are checked after each hook. Context flags and trace state are not protected IDs. Routing attributes such as braintrust.parent may be changed.
• Errors, panics, nil results, or changed IDs fail closed for the affected span: later hooks stop for that span and its content is stripped. The stripped record retains the original IDs, timestamps, protocol flags, and braintrust.parent routing; braintrust.context_json is replaced with {"customizer_error":true}. The stripped record and unrelated spans continue through export. Hook failures do not themselves cause an export or ForceFlush error, so a successful flush does not imply every hook succeeded or that no data was exported. Failures are logged at most once per process, without hook error/panic text or payloads. Original content is never exported as a fallback, and the caller's batch is not mutated. External side effects performed by hooks are not rolled back.
• Hooks run synchronously on the export path, often on a background goroutine. Keep them fast, avoid blocking I/O, and make shared state concurrency-safe. Do not retain or asynchronously mutate the supplied or returned span. Submitting the batch to the exporter again runs hooks again; retries inside the OTLP transport reuse the already transformed payload. With no hooks, export skips customization and still performs configured attachment processing.
• With the built-in uploader, a temporarily full upload queue briefly delays export, within part of the exporter's timeout, while waiting for capacity. If capacity does not free up in time, the span is exported with its (customized) attachment data inline rather than dropped, and none of that span's attachments are uploaded. Uploader failure or shutdown also falls back to inline data.

Run the complete redaction example with BRAINTRUST_API_KEY set:

go run ./examples/internal/span-customizers/main.go

The example prints Braintrust links for a redacted span and a stripped failure record. The redacted span's input is "[redacted]", its output is absent, and its redacted attribute is true. The failure record has its content stripped and context.customizer_error set to true; it does not prevent the redacted span from being exported.

Evaluations

Run evals with custom test cases and scoring functions.

Define and run

Define an eval once with its task and scorers, then run it against any dataset:

package main

import (
    "context"
    "log"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/sdk/trace"

    "github.com/braintrustdata/braintrust-sdk-go"
    "github.com/braintrustdata/braintrust-sdk-go/eval"
)

func main() {
    ctx := context.Background()

    // Set up OpenTelemetry tracer
    tp := trace.NewTracerProvider()
    defer tp.Shutdown(ctx)
    otel.SetTracerProvider(tp)

    // Initialize Braintrust
    client, err := braintrust.New(tp)
    if err != nil {
        log.Fatal(err)
    }

    // Create an eval
    e := braintrust.NewEval(client, &eval.Eval[string, string]{
        Name: "greeting-experiment",
        Task: eval.T(func(ctx context.Context, input string) (string, error) {
            return "Hello " + input, nil
        }),
        Scorers: []eval.Scorer[string, string]{
            eval.NewScorer("exact_match", func(ctx context.Context, r eval.TaskResult[string, string]) (eval.Scores, error) {
                score := 0.0
                if r.Expected == r.Output {
                    score = 1.0
                }
                return eval.S(score), nil
            }),
        },
    })

    // Run against a dataset
    _, err = e.Run(ctx, eval.RunOpts[string, string]{
        Dataset: eval.NewDataset([]eval.Case[string, string]{
            {Input: "World", Expected: "Hello World"},
            {Input: "Alice", Expected: "Hello Alice"},
        }),
    })
    if err != nil {
        log.Fatal(err)
    }
}

Examples

Complete working examples are available in examples/:

Getting Started:

• openai - OpenAI tracing
• anthropic - Anthropic tracing
• genai - Google Gemini tracing
• genkit - Firebase Genkit middleware tracing
• api-client - Generated REST client: read-only browsing, with opt-in dataset-row and paused project-automation CRUD

Evaluations:

• evals - Evaluations with custom scorers
• datasets - Run evals against downloaded datasets
• dataset-api - Create datasets, use prompts, run evals
• scorers - Custom scoring with online and code-based scorers

Alternative Providers & Libraries:

• sashabaranov-openai - sashabaranov/go-openai tracing
• openrouter - OpenRouter tracing
• langchaingo - LangChainGo integration
• adk - Google ADK agent tracing
• cloudwego/eino - CloudWeGo Eino integration

Advanced:

• manual-llm-logging - Manually log LLM calls
• attachments - Include images and files in traces
• prompts - Render a prompt locally and call the model yourself, or invoke it server-side
• distributed-tracing - W3C baggage propagation across services
• otel - Add Braintrust to existing OpenTelemetry setup

Features

• Evaluations - Systematic testing with custom scoring functions
• Tracing - Automatic instrumentation for major LLM providers
• Prompts - Load Braintrust prompts, render them locally, and call any LLM client
• Datasets - Manage and version evaluation datasets
• Experiments - Track versions and configurations
• Observability - Monitor AI applications in production

Documentation

• Braintrust Documentation
• API Reference
• HTTP API client guide - Public generated client, configuration, models, and a runnable example

Contributing

See CONTRIBUTING.md for development setup and contribution guidelines.

License

Apache License 2.0. See LICENSE for details.

Documentation

Overview

Package braintrust provides the core Braintrust SDK for Go.

Braintrust is a platform for building reliable AI applications. This SDK provides tools for evaluation, experimentation, and observability of AI systems.

Main Packages

For AI model evaluations, see the eval package.

For distributed tracing and observability, see the trace package.

Configuration

The SDK reads configuration from environment variables. See [GetConfig] for a complete list of supported environment variables.

Learn More

• Examples: https://github.com/braintrustdata/braintrust-sdk-go/tree/main/examples
• Documentation: https://www.braintrust.dev/docs

Index

• func NewEval[I, R any](client *Client, e *eval.Eval[I, R]) *eval.Eval[I, R]
• func NewEvaluator[I, R any](client *Client) *eval.Evaluator[I, R]
• func NewWorkflowEval[I, R any](client *Client, definition *eval.WorkflowEval[I, R]) *eval.WorkflowEval[I, R]
• type Client
  • func New(tp *trace.TracerProvider, opts ...Option) (*Client, error)
  • func (c *Client) API() *api.API
  • func (c *Client) LoadPrompt(ctx context.Context, opts prompt.LoadOpts) (*prompt.Prompt, error)
  • func (c *Client) Permalink(span oteltrace.Span) string
  • func (c *Client) String() string
  • func (c *Client) Tracer(name string, opts ...oteltrace.TracerOption) oteltrace.Tracer
  • func (c *Client) TracerProvider() *trace.TracerProvider
• type Option
  • func EnableBuiltinAdkTraces() Option
  • func WithAPIKey(apiKey string) Option
  • func WithAPIURL(apiURL string) Option
  • func WithAppURL(appURL string) Option
  • func WithAutoConvertAIAttachments(enabled bool) Option
  • func WithBlockingLogin(enabled bool) Option
  • func WithEnableTraceConsoleLog(enabled bool) Option
  • func WithEnvironment(environmentType string, name ...string) Option
  • func WithExporter(exporter trace.SpanExporter) Option
  • func WithFilterAISpans(enabled bool) Option
  • func WithLogger(l logger.Logger) Option
  • func WithOrgName(orgName string) Option
  • func WithProject(projectName string) Option
  • func WithProjectID(projectID string) Option
  • func WithSpanCustomizers(customizers ...config.SpanCustomizer) Option
  • func WithSpanFilterFuncs(filterFuncs ...config.SpanFilterFunc) Option

Functions

func NewEval

func NewEval[I, R any](client *Client, e *eval.Eval[I, R]) *eval.Eval[I, R]

NewEval creates a runnable eval.Eval by combining a client with an eval definition.

Example:

client, _ := braintrust.New(tp)
e := braintrust.NewEval(client, &eval.Eval[string, string]{
    Name:    "classify",
    Task:    task,
    Scorers: scorers,
})
result, _ := e.Run(ctx, eval.RunOpts[string, string]{Dataset: dataset})

func NewEvaluator

func NewEvaluator[I, R any](client *Client) *eval.Evaluator[I, R]

NewEvaluator creates a new evaluator for running evaluations with the same input and output types.

func NewWorkflowEval

func NewWorkflowEval[I, R any](client *Client, definition *eval.WorkflowEval[I, R]) *eval.WorkflowEval[I, R]

NewWorkflowEval attaches the client to a durable workflow evaluation.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is the main Braintrust SDK client

func New

func New(tp *trace.TracerProvider, opts ...Option) (*Client, error)

New creates a new Braintrust client.

It will add a Braintrust exporter to the given tracer provider..

Configuration is loaded from environment variables first, then explicit options are applied (options take precedence).

Login happens asynchronously in the background by default.

Example:

tp := trace.NewTracerProvider()
bt, err := braintrust.New(tp,
    braintrust.WithAPIKey("your-api-key"),
    braintrust.WithProject("my-project"),
)
if err != nil {
    log.Fatal(err)
}
defer tp.Shutdown(context.Background())

func (*Client) API

func (c *Client) API() *api.API

API returns the legacy API facade for source compatibility. SDK evaluation and prompt loading use the generated openapi.Client instead. For new direct REST integrations, use the openapi package.

Deprecated: Use openapi.NewClientWithResponses for direct REST operations. Continue to use Client.LoadPrompt, NewEvaluator, and NewWorkflowEval for SDK-managed prompt loading and evaluations.

func (*Client) LoadPrompt

func (c *Client) LoadPrompt(ctx context.Context, opts prompt.LoadOpts) (*prompt.Prompt, error)

LoadPrompt loads a prompt saved in Braintrust and returns it ready to render.

p, err := client.LoadPrompt(ctx, prompt.LoadOpts{Slug: "summarizer"})
if err != nil {
	return err
}
built, err := p.Build(map[string]any{"input": article})

Opts.Project defaults to the client's project. The prompt is fetched every time; nothing is cached.

func (*Client) Permalink

func (c *Client) Permalink(span oteltrace.Span) string

Permalink returns a URL to the span in the Braintrust UI. If the permalink cannot be generated, it returns an empty string and logs a warning.

Example:

client, _ := braintrust.New(tp, braintrust.WithAPIKey("your-key"))
tracer := client.Tracer("my-app")
ctx, span := tracer.Start(ctx, "my-operation")
defer span.End()

// Get the permalink
link := client.Permalink(span)
fmt.Println("View trace:", link)

func (*Client) String

func (c *Client) String() string

String returns a string representation of the client

func (*Client) Tracer

func (c *Client) Tracer(name string, opts ...oteltrace.TracerOption) oteltrace.Tracer

Tracer returns an OpenTelemetry Tracer with the given name. This is a convenience method equivalent to calling TracerProvider().Tracer(name, opts...).

Example:

tracer := client.Tracer("my-app")
ctx, span := tracer.Start(ctx, "my-operation")
defer span.End()

func (*Client) TracerProvider

func (c *Client) TracerProvider() *trace.TracerProvider

TracerProvider returns the OpenTelemetry TracerProvider used by this client. This can be used to create tracers or access the provider for advanced use cases.

type Option

type Option func(*config.Config)

Option is a functional option for configuring a Braintrust client

func EnableBuiltinAdkTraces

func EnableBuiltinAdkTraces() Option

EnableBuiltinAdkTraces can be used to enable exporting spans from Google ADK's built-in telemetry (gcp.vertex.agent), which are not exported to Braintrust by default.

func WithAPIKey

func WithAPIKey(apiKey string) Option

WithAPIKey sets a nonblank API key (overrides BRAINTRUST_API_KEY). Blank or whitespace-only values are treated as unset and leave environment and .braintrust.json fallback lookup available.

func WithAPIURL

func WithAPIURL(apiURL string) Option

WithAPIURL sets the API URL (overrides BRAINTRUST_API_URL)

func WithAppURL

func WithAppURL(appURL string) Option

WithAppURL sets the app URL (overrides BRAINTRUST_APP_URL)

func WithAutoConvertAIAttachments

func WithAutoConvertAIAttachments(enabled bool) Option

WithAutoConvertAIAttachments controls whether the SDK automatically scans spans for base64 LLM attachments and replaces them with uploaded references. Defaults to true. Environment variable: BRAINTRUST_AUTO_CONVERT_AI_ATTACHMENTS

func WithBlockingLogin

func WithBlockingLogin(enabled bool) Option

WithBlockingLogin blocks setup until login completes. Authentication normally happens asynchronously in the background.

This only enables printing links to spans. This is only useful for scripts, testing and examples. It is not recommended for production use.

func WithEnableTraceConsoleLog

func WithEnableTraceConsoleLog(enabled bool) Option

WithEnableTraceConsoleLog enables logging traces to stdout for debugging. Environment variable: BRAINTRUST_ENABLE_TRACE_CONSOLE_LOG

func WithEnvironment

func WithEnvironment(environmentType string, name ...string) Option

WithEnvironment sets span-origin environment provenance.

func WithExporter

func WithExporter(exporter trace.SpanExporter) Option

WithExporter injects a custom OpenTelemetry SpanExporter. If not provided, an OTLP HTTP exporter will be created automatically. This is solely for testing purposes.

func WithFilterAISpans

func WithFilterAISpans(enabled bool) Option

WithFilterAISpans enables filtering to keep only AI-related spans When enabled, only spans with AI-related names or attributes will be sent

func WithLogger

func WithLogger(l logger.Logger) Option

WithLogger sets a custom logger for the SDK If not provided, a default logger will be used

func WithOrgName

func WithOrgName(orgName string) Option

WithOrgName sets the organization name (overrides BRAINTRUST_ORG_NAME)

func WithProject

func WithProject(projectName string) Option

WithProject sets the default project name (overrides BRAINTRUST_DEFAULT_PROJECT)

func WithProjectID

func WithProjectID(projectID string) Option

WithProjectID sets the default project ID (overrides BRAINTRUST_DEFAULT_PROJECT_ID)

func WithSpanCustomizers

func WithSpanCustomizers(customizers ...config.SpanCustomizer) Option

WithSpanCustomizers appends export customizers in execution order. The option snapshots the supplied list; constructing the exporter snapshots it again. Callback closure state remains the caller's responsibility.

func WithSpanFilterFuncs

func WithSpanFilterFuncs(filterFuncs ...config.SpanFilterFunc) Option

WithSpanFilterFuncs adds custom span filter functions Filters are evaluated in order. Return >0 to keep, <0 to drop, 0 to continue