braintrust

README

Braintrust Go SDK

Overview

This library provides tools for evaluating and tracing AI applications in Braintrust. Use it to:

• Evaluate your AI models with custom test cases and scoring functions
• Trace LLM calls and monitor AI application performance with OpenTelemetry
• Integrate seamlessly with OpenAI, Anthropic, Google Gemini, LangChainGo, and other LLM providers

This SDK is currently in BETA status and APIs may change.

Installation

go get github.com/braintrustdata/braintrust-sdk-go

export BRAINTRUST_API_KEY="your-api-key"  # Get from https://www.braintrust.dev/app/settings

Instrumentation

Trace LLM calls with automatic or manual instrumentation.

Automatic Instrumentation

Use Orchestrion to automatically inject tracing at compile time—no code changes required.

1. Install orchestrion:

go install github.com/DataDog/orchestrion@latest

2. Create orchestrion.tool.go in your project root:

//go:build tools

package main

import (
    _ "github.com/DataDog/orchestrion"
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/all" // All LLM providers
)

Or import only the integrations you need:

import (
    _ "github.com/DataDog/orchestrion"
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai"    // OpenAI (openai-go)
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/anthropic" // Anthropic
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/genai"     // Google GenAI
    _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/github.com/sashabaranov/go-openai" // sashabaranov/go-openai
)

3. Build with orchestrion:

# Build with orchestrion
orchestrion go build ./...

# Or configure GOFLAGS to use orchestrion automatically
export GOFLAGS="-toolexec='orchestrion toolexec'"
go build ./...

That's it! Your LLM client calls are now automatically traced. No middleware or wrapper code needed in your application.

Manual Instrumentation

If you prefer explicit control, you can add tracing middleware manually to your LLM clients. See the Manual Instrumentation Guide for detailed examples with OpenAI, Anthropic, Google Gemini, and other providers.

Evaluations

Run evals with custom test cases and scoring functions:

package main

import (
    "context"
    "log"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/sdk/trace"

    "github.com/braintrustdata/braintrust-sdk-go"
    "github.com/braintrustdata/braintrust-sdk-go/eval"
)

func main() {
    ctx := context.Background()

    // Set up OpenTelemetry tracer
    tp := trace.NewTracerProvider()
    defer tp.Shutdown(ctx)
    otel.SetTracerProvider(tp)

    // Initialize Braintrust
    client, err := braintrust.New(tp)
    if err != nil {
        log.Fatal(err)
    }

    // Create an evaluator with your task's input and output types
    evaluator := braintrust.NewEvaluator[string, string](client)

    // Run an evaluation
    _, err = evaluator.Run(ctx, eval.Opts[string, string]{
        Experiment: "greeting-experiment",
        Dataset: eval.NewDataset([]eval.Case[string, string]{
            {Input: "World", Expected: "Hello World"},
            {Input: "Alice", Expected: "Hello Alice"},
        }),
        Task: eval.T(func(ctx context.Context, input string) (string, error) {
            return "Hello " + input, nil
        }),
        Scorers: []eval.Scorer[string, string]{
            eval.NewScorer("exact_match", func(ctx context.Context, r eval.TaskResult[string, string]) (eval.Scores, error) {
                score := 0.0
                if r.Expected == r.Output {
                    score = 1.0
                }
                return eval.S(score), nil
            }),
        },
    })
    if err != nil {
        log.Fatal(err)
    }
}

API Client

Manage Braintrust resources programmatically:

package main

import (
    "context"
    "log"

    "go.opentelemetry.io/otel/sdk/trace"

    "github.com/braintrustdata/braintrust-sdk-go"
    functionsapi "github.com/braintrustdata/braintrust-sdk-go/api/functions"
)

func main() {
    ctx := context.Background()

    // Create tracer provider
    tp := trace.NewTracerProvider()
    defer tp.Shutdown(ctx)

    // Initialize Braintrust
    client, err := braintrust.New(tp,
        braintrust.WithProject("my-project"),
    )
    if err != nil {
        log.Fatal(err)
    }

    // Get API client
    api := client.API()

    // Create a prompt
    prompt, err := api.Functions().Create(ctx, functionsapi.CreateParams{
        ProjectID: "your-project-id",
        Name:      "My Prompt",
        Slug:      "my-prompt",
        FunctionData: map[string]any{
            "type": "prompt",
        },
        PromptData: map[string]any{
            "prompt": map[string]any{
                "type": "chat",
                "messages": []map[string]any{
                    {
                        "role":    "system",
                        "content": "You are a helpful assistant.",
                    },
                    {
                        "role":    "user",
                        "content": "{{input}}",
                    },
                },
            },
            "options": map[string]any{
                "model": "gpt-4o-mini",
            },
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    _ = prompt // Prompt is ready to use
}

Examples

Complete working examples are available in examples/:

• evals - Evaluations with custom scorers
• openai - OpenAI tracing
• anthropic - Anthropic tracing
• genai - Google Gemini tracing
• langchaingo - LangChainGo integration
• datasets - Using Braintrust datasets

Features

• Evaluations - Systematic testing with custom scoring functions
• Tracing - Automatic instrumentation for major LLM providers
• Datasets - Manage and version evaluation datasets
• Experiments - Track versions and configurations
• Observability - Monitor AI applications in production

Documentation

• Braintrust Documentation
• API Reference

Contributing

See CONTRIBUTING.md for development setup and contribution guidelines.

License

Apache License 2.0. See LICENSE for details.

Documentation

Overview

Package braintrust provides the core Braintrust SDK for Go.

Braintrust is a platform for building reliable AI applications. This SDK provides tools for evaluation, experimentation, and observability of AI systems.

Main Packages

For AI model evaluations, see the eval package.

For distributed tracing and observability, see the trace package.

Configuration

The SDK reads configuration from environment variables. See [GetConfig] for a complete list of supported environment variables.

Learn More

• Examples: https://github.com/braintrustdata/braintrust-sdk-go/tree/main/examples
• Documentation: https://www.braintrust.dev/docs

Index

• func NewEvaluator[I, R any](client *Client) *eval.Evaluator[I, R]
• type Client
  • func New(tp *trace.TracerProvider, opts ...Option) (*Client, error)
  • func (c *Client) API() *api.API
  • func (c *Client) Permalink(span oteltrace.Span) string
  • func (c *Client) String() string
  • func (c *Client) Tracer(name string, opts ...oteltrace.TracerOption) oteltrace.Tracer
  • func (c *Client) TracerProvider() *trace.TracerProvider
• type Option
  • func WithAPIKey(apiKey string) Option
  • func WithAPIURL(apiURL string) Option
  • func WithAppURL(appURL string) Option
  • func WithBlockingLogin(enabled bool) Option
  • func WithExporter(exporter trace.SpanExporter) Option
  • func WithFilterAISpans(enabled bool) Option
  • func WithLogger(l logger.Logger) Option
  • func WithOrgName(orgName string) Option
  • func WithProject(projectName string) Option
  • func WithProjectID(projectID string) Option
  • func WithSpanFilterFuncs(filterFuncs ...config.SpanFilterFunc) Option

Functions

func NewEvaluator

func NewEvaluator[I, R any](client *Client) *eval.Evaluator[I, R]

NewEvaluator creates a new evaluator for running multiple evaluations with the same input and output types.

Example:

client, _ := braintrust.New(tp)

// Create an evaluator for string → string evaluations
evaluator := braintrust.NewEvaluator[string, string](client)

// Run multiple evaluations
result1, _ := evaluator.Run(ctx, eval.Opts[string, string]{
    Experiment: "test-1",
    Dataset:    dataset1,
    Task:       task1,
    Scorers:    scorers,
})

result2, _ := evaluator.Run(ctx, eval.Opts[string, string]{
    Experiment: "test-2",
    Dataset:    dataset2,
    Task:       task2,
    Scorers:    scorers,
})

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is the main Braintrust SDK client

func New

func New(tp *trace.TracerProvider, opts ...Option) (*Client, error)

New creates a new Braintrust client.

It will add a Braintrust exporter to the given tracer provider..

Configuration is loaded from environment variables first, then explicit options are applied (options take precedence).

Login happens asynchronously in the background by default.

Example:

tp := trace.NewTracerProvider()
bt, err := braintrust.New(tp,
    braintrust.WithAPIKey("your-api-key"),
    braintrust.WithProject("my-project"),
)
if err != nil {
    log.Fatal(err)
}
defer tp.Shutdown(context.Background())

func (*Client) API

func (c *Client) API() *api.API

API returns an API client for making direct calls to the Braintrust API. This provides low-level access to projects, datasets, experiments, and other resources.

Example:

client, _ := braintrust.New(tp, braintrust.WithAPIKey("your-key"))

// Create a dataset
apiClient := client.API()
project, _ := apiClient.Projects().Create(ctx, "my-project")
dataset, _ := apiClient.Datasets().Create(ctx, api.DatasetRequest{
    ProjectID:   project.ID,
    Name:        "my-dataset",
    Description: "My test dataset",
})

func (*Client) Permalink

func (c *Client) Permalink(span oteltrace.Span) string

Permalink returns a URL to the span in the Braintrust UI. If the permalink cannot be generated, it returns an empty string and logs a warning.

Example:

client, _ := braintrust.New(tp, braintrust.WithAPIKey("your-key"))
tracer := client.Tracer("my-app")
ctx, span := tracer.Start(ctx, "my-operation")
defer span.End()

// Get the permalink
link := client.Permalink(span)
fmt.Println("View trace:", link)

func (*Client) String

func (c *Client) String() string

String returns a string representation of the client

func (*Client) Tracer

func (c *Client) Tracer(name string, opts ...oteltrace.TracerOption) oteltrace.Tracer

Tracer returns an OpenTelemetry Tracer with the given name. This is a convenience method equivalent to calling TracerProvider().Tracer(name, opts...).

Example:

tracer := client.Tracer("my-app")
ctx, span := tracer.Start(ctx, "my-operation")
defer span.End()

func (*Client) TracerProvider

func (c *Client) TracerProvider() *trace.TracerProvider

TracerProvider returns the OpenTelemetry TracerProvider used by this client. This can be used to create tracers or access the provider for advanced use cases.

type Option

type Option func(*config.Config)

Option is a functional option for configuring a Braintrust client

func WithAPIKey

func WithAPIKey(apiKey string) Option

WithAPIKey sets the API key (overrides BRAINTRUST_API_KEY)

func WithAPIURL

func WithAPIURL(apiURL string) Option

WithAPIURL sets the API URL (overrides BRAINTRUST_API_URL)

func WithAppURL

func WithAppURL(appURL string) Option

WithAppURL sets the app URL (overrides BRAINTRUST_APP_URL)

func WithBlockingLogin

func WithBlockingLogin(enabled bool) Option

WithBlockingLogin blocks setup until login completes. Authentication normally happens asynchronously in the background.

This only enables printing links to spans. This is only useful for scripts, testing and examples. It is not recommended for production use.

func WithExporter

func WithExporter(exporter trace.SpanExporter) Option

WithExporter injects a custom OpenTelemetry SpanExporter. If not provided, an OTLP HTTP exporter will be created automatically. This is solely for testing purposes.

func WithFilterAISpans

func WithFilterAISpans(enabled bool) Option

WithFilterAISpans enables filtering to keep only AI-related spans When enabled, only spans with AI-related names or attributes will be sent

func WithLogger

func WithLogger(l logger.Logger) Option

WithLogger sets a custom logger for the SDK If not provided, a default logger will be used

func WithOrgName

func WithOrgName(orgName string) Option

WithOrgName sets the organization name (overrides BRAINTRUST_ORG_NAME)

func WithProject

func WithProject(projectName string) Option

WithProject sets the default project name (overrides BRAINTRUST_DEFAULT_PROJECT)

func WithProjectID

func WithProjectID(projectID string) Option

WithProjectID sets the default project ID (overrides BRAINTRUST_DEFAULT_PROJECT_ID)

func WithSpanFilterFuncs

func WithSpanFilterFuncs(filterFuncs ...config.SpanFilterFunc) Option

WithSpanFilterFuncs adds custom span filter functions Filters are evaluated in order. Return >0 to keep, <0 to drop, 0 to continue