schemas

package
v1.1.36 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 21, 2025 License: Apache-2.0 Imports: 5 Imported by: 23

Documentation

Overview

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Index

Constants

View Source 
const (
	DefaultMaxRetries              = 0
	DefaultRetryBackoffInitial     = 500 * time.Millisecond
	DefaultRetryBackoffMax         = 5 * time.Second
	DefaultRequestTimeoutInSeconds = 30
	DefaultBufferSize              = 5000
	DefaultConcurrency             = 1000
	DefaultStreamBufferSize        = 5000
)
View Source 
const (
	ErrProviderRequest           = "failed to make HTTP request to provider API"
	ErrProviderResponseUnmarshal = "failed to unmarshal response from provider API"
	ErrProviderJSONMarshaling    = "failed to marshal request body to JSON"
	ErrProviderDecodeStructured  = "failed to decode provider's structured response"
	ErrProviderDecodeRaw         = "failed to decode provider's raw response"
	ErrProviderDecompress        = "failed to decompress provider's response"
)

Pre-defined errors for provider operations

View Source 
const (
	DefaultInitialPoolSize = 5000
)
View Source 
const (
	RequestCancelled = "request_cancelled"
)

Variables

View Source 
var DefaultConcurrencyAndBufferSize = ConcurrencyAndBufferSize{
	Concurrency: DefaultConcurrency,
	BufferSize:  DefaultBufferSize,
}

DefaultConcurrencyAndBufferSize is the default concurrency and buffer size for provider operations.

View Source 
var DefaultNetworkConfig = NetworkConfig{
	DefaultRequestTimeoutInSeconds: DefaultRequestTimeoutInSeconds,
	MaxRetries:                     DefaultMaxRetries,
	RetryBackoffInitial:            DefaultRetryBackoffInitial,
	RetryBackoffMax:                DefaultRetryBackoffMax,
}

DefaultNetworkConfig is the default network configuration for provider connections.

View Source 
var StandardProviders = []ModelProvider{
	Anthropic,
	Azure,
	Bedrock,
	Cerebras,
	Cohere,
	Gemini,
	Groq,
	Mistral,
	Ollama,
	OpenAI,
	Parasail,
	SGL,
	Vertex,
	OpenRouter,
}

StandardProviders is the list of all built-in (non-custom) providers.

View Source 
var SupportedBaseProviders = []ModelProvider{
	Anthropic,
	Bedrock,
	Cohere,
	Gemini,
	OpenAI,
}

SupportedBaseProviders is the list of base providers allowed for custom providers.

Functions

This section is empty.

Types

type Account 

type Account interface {
	// GetConfiguredProviders returns a list of providers that are configured
	// in the account. This is used to determine which providers are available for use.
	GetConfiguredProviders() ([]ModelProvider, error)

	// GetKeysForProvider returns the API keys configured for a specific provider.
	// The keys include their values, supported models, and weights for load balancing.
	// The context can carry data from any source that sets values before the Bifrost request,
	// including but not limited to plugin pre-hooks, application logic, or any in app middleware sharing the context.
	// This enables dynamic key selection based on any context values present during the request.
	GetKeysForProvider(ctx *context.Context, providerKey ModelProvider) ([]Key, error)

	// GetConfigForProvider returns the configuration for a specific provider.
	// This includes network settings, authentication details, and other provider-specific
	// configurations.
	GetConfigForProvider(providerKey ModelProvider) (*ProviderConfig, error)
}

Account defines the interface for managing provider accounts and their configurations. It provides methods to access provider-specific settings, API keys, and configurations.

type AllowedRequests added in v1.1.26 

type AllowedRequests struct {
	TextCompletion       bool `json:"text_completion"`
	ChatCompletion       bool `json:"chat_completion"`
	ChatCompletionStream bool `json:"chat_completion_stream"`
	Embedding            bool `json:"embedding"`
	Speech               bool `json:"speech"`
	SpeechStream         bool `json:"speech_stream"`
	Transcription        bool `json:"transcription"`
	TranscriptionStream  bool `json:"transcription_stream"`
}

AllowedRequests controls which operations are permitted. A nil *AllowedRequests means "all operations allowed." A non-nil value only allows fields set to true; omitted or false fields are disallowed.

func (*AllowedRequests) IsOperationAllowed added in v1.1.26 

func (ar *AllowedRequests) IsOperationAllowed(operation Operation) bool

IsOperationAllowed checks if a specific operation is allowed

type Annotation 

type Annotation struct {
	Type     string   `json:"type"`
	Citation Citation `json:"url_citation"`
}

Annotation represents an annotation in a response.

type AssistantMessage added in v1.0.9 

type AssistantMessage struct {
	Refusal     *string      `json:"refusal,omitempty"`
	Annotations []Annotation `json:"annotations,omitempty"`
	ToolCalls   *[]ToolCall  `json:"tool_calls,omitempty"`
	Thought     *string      `json:"thought,omitempty"`
}

AssistantMessage represents a message from an assistant

type AudioLLMUsage added in v1.1.11 

type AudioLLMUsage struct {
	InputTokens        int                `json:"input_tokens"`
	InputTokensDetails *AudioTokenDetails `json:"input_tokens_details,omitempty"`
	OutputTokens       int                `json:"output_tokens"`
	TotalTokens        int                `json:"total_tokens"`
}

type AudioTokenDetails added in v1.1.11 

type AudioTokenDetails struct {
	TextTokens  int `json:"text_tokens"`
	AudioTokens int `json:"audio_tokens"`
}

type AzureKeyConfig added in v1.1.9 

type AzureKeyConfig struct {
	Endpoint    string            `json:"endpoint"`              // Azure service endpoint URL
	Deployments map[string]string `json:"deployments,omitempty"` // Mapping of model names to deployment names
	APIVersion  *string           `json:"api_version,omitempty"` // Azure API version to use; defaults to "2024-08-01-preview"
}

AzureKeyConfig represents the Azure-specific configuration. It contains Azure-specific settings required for service access and deployment management.

type BedrockKeyConfig added in v1.1.14 

type BedrockKeyConfig struct {
	AccessKey    string            `json:"access_key,omitempty"`    // AWS access key for authentication
	SecretKey    string            `json:"secret_key,omitempty"`    // AWS secret access key for authentication
	SessionToken *string           `json:"session_token,omitempty"` // AWS session token for temporary credentials
	Region       *string           `json:"region,omitempty"`        // AWS region for service access
	ARN          *string           `json:"arn,omitempty"`           // Amazon Resource Name for resource identification
	Deployments  map[string]string `json:"deployments,omitempty"`   // Mapping of model identifiers to inference profiles
}

BedrockKeyConfig represents the AWS Bedrock-specific configuration. It contains AWS-specific settings required for authentication and service access.

type BifrostCacheDebug added in v1.1.26 

type BifrostCacheDebug struct {
	CacheHit bool `json:"cache_hit"`

	CacheID *string `json:"cache_id,omitempty"`
	HitType *string `json:"hit_type,omitempty"`

	// Semantic cache only (provider, model, and input tokens will be present for semantic cache, even if cache is not hit)
	ProviderUsed *string `json:"provider_used,omitempty"`
	ModelUsed    *string `json:"model_used,omitempty"`
	InputTokens  *int    `json:"input_tokens,omitempty"`

	// Semantic cache only (only when cache is hit)
	Threshold  *float64 `json:"threshold,omitempty"`
	Similarity *float64 `json:"similarity,omitempty"`
}

BifrostCacheDebug represents debug information about the cache.

type BifrostConfig 

type BifrostConfig struct {
	Account            Account
	Plugins            []Plugin
	Logger             Logger
	InitialPoolSize    int        // Initial pool size for sync pools in Bifrost. Higher values will reduce memory allocations but will increase memory usage.
	DropExcessRequests bool       // If true, in cases where the queue is full, requests will not wait for the queue to be empty and will be dropped instead.
	MCPConfig          *MCPConfig // MCP (Model Context Protocol) configuration for tool integration
}

BifrostConfig represents the configuration for initializing a Bifrost instance. It contains the necessary components for setting up the system including account details, plugins, logging, and initial pool size.

type BifrostContextKey added in v1.1.17 

type BifrostContextKey string

BifrostContextKey is a type for context keys used in Bifrost. 

const (
	BifrostContextKeyDirectKey          BifrostContextKey = "bifrost-direct-key"
	BifrostContextKeyStreamEndIndicator BifrostContextKey = "bifrost-stream-end-indicator"
	BifrostContextKeyRequestType        BifrostContextKey = "bifrost-request-type"
	BifrostContextKeyRequestProvider    BifrostContextKey = "bifrost-request-provider"
	BifrostContextKeyRequestModel       BifrostContextKey = "bifrost-request-model"
)

BifrostContextKeyRequestType is a context key for the request type.

type BifrostEmbedding added in v1.1.14 

type BifrostEmbedding struct {
	Index     int                      `json:"index"`
	Object    string                   `json:"object"`    // embedding
	Embedding BifrostEmbeddingResponse `json:"embedding"` // can be []float32 or string
}

type BifrostEmbeddingResponse added in v1.1.14 

type BifrostEmbeddingResponse struct {
	EmbeddingStr     *string
	EmbeddingArray   *[]float32
	Embedding2DArray *[][]float32
}

func (BifrostEmbeddingResponse) MarshalJSON added in v1.1.14 

func (be BifrostEmbeddingResponse) MarshalJSON() ([]byte, error)

func (*BifrostEmbeddingResponse) UnmarshalJSON added in v1.1.14 

func (be *BifrostEmbeddingResponse) UnmarshalJSON(data []byte) error

type BifrostError 

type BifrostError struct {
	Provider       ModelProvider  `json:"-"`
	EventID        *string        `json:"event_id,omitempty"`
	Type           *string        `json:"type,omitempty"`
	IsBifrostError bool           `json:"is_bifrost_error"`
	StatusCode     *int           `json:"status_code,omitempty"`
	Error          ErrorField     `json:"error"`
	AllowFallbacks *bool          `json:"-"` // Optional: Controls fallback behavior (nil = true by default)
	StreamControl  *StreamControl `json:"-"` // Optional: Controls stream behavior
}

BifrostError represents an error from the Bifrost system.

PLUGIN DEVELOPERS: When creating BifrostError in PreHook or PostHook, you can set AllowFallbacks: - AllowFallbacks = &true: Bifrost will try fallback providers if available - AllowFallbacks = &false: Bifrost will return this error immediately, no fallbacks - AllowFallbacks = nil: Treated as true by default (fallbacks allowed for resilience)

type BifrostMessage added in v1.0.9 

type BifrostMessage struct {
	Role    ModelChatMessageRole `json:"role"`
	Content MessageContent       `json:"content"`

	// Embedded pointer structs - when non-nil, their exported fields are flattened into the top-level JSON object
	// IMPORTANT: Only one of the following can be non-nil at a time, otherwise the JSON marshalling will override the common fields
	*ToolMessage
	*AssistantMessage
}

BifrostMessage represents a message in a chat conversation.

type BifrostNonStreamResponseChoice added in v1.1.8 

type BifrostNonStreamResponseChoice struct {
	Message    BifrostMessage `json:"message"`
	StopString *string        `json:"stop,omitempty"`
	LogProbs   *LogProbs      `json:"log_probs,omitempty"`
}

BifrostNonStreamResponseChoice represents a choice in the non-stream response

type BifrostRequest 

type BifrostRequest struct {
	Provider ModelProvider    `json:"provider"`
	Model    string           `json:"model"`
	Input    RequestInput     `json:"input"`
	Params   *ModelParameters `json:"params,omitempty"`

	// Fallbacks are tried in order, the first one to succeed is returned
	// Provider config must be available for each fallback's provider in account's GetConfigForProvider,
	// else it will be skipped.
	Fallbacks []Fallback `json:"fallbacks,omitempty"`
}

BifrostRequest represents a request to be processed by Bifrost. It must be provided when calling the Bifrost for text completion, chat completion, or embedding. It contains the model identifier, input data, and parameters for the request.

type BifrostResponse 

type BifrostResponse struct {
	ID                string                     `json:"id,omitempty"`
	Object            string                     `json:"object,omitempty"` // text.completion, chat.completion, embedding, speech, transcribe
	Choices           []BifrostResponseChoice    `json:"choices,omitempty"`
	Data              []BifrostEmbedding         `json:"data,omitempty"`       // Maps to "data" field in provider responses (e.g., OpenAI embedding format)
	Speech            *BifrostSpeech             `json:"speech,omitempty"`     // Maps to "speech" field in provider responses (e.g., OpenAI speech format)
	Transcribe        *BifrostTranscribe         `json:"transcribe,omitempty"` // Maps to "transcribe" field in provider responses (e.g., OpenAI transcription format)
	Model             string                     `json:"model,omitempty"`
	Created           int                        `json:"created,omitempty"` // The Unix timestamp (in seconds).
	ServiceTier       *string                    `json:"service_tier,omitempty"`
	SystemFingerprint *string                    `json:"system_fingerprint,omitempty"`
	Usage             *LLMUsage                  `json:"usage,omitempty"`
	ExtraFields       BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostResponse represents the complete result from any bifrost request.

type BifrostResponseChoice 

type BifrostResponseChoice struct {
	Index        int     `json:"index"`
	FinishReason *string `json:"finish_reason,omitempty"`

	*BifrostNonStreamResponseChoice
	*BifrostStreamResponseChoice
}

BifrostResponseChoice represents a choice in the completion result. This struct can represent either a streaming or non-streaming response choice. IMPORTANT: Only one of BifrostNonStreamResponseChoice or BifrostStreamResponseChoice should be non-nil at a time.

type BifrostResponseExtraFields 

type BifrostResponseExtraFields struct {
	Provider    ModelProvider      `json:"provider"`
	Params      ModelParameters    `json:"model_params"`
	Latency     *float64           `json:"latency,omitempty"`
	ChatHistory *[]BifrostMessage  `json:"chat_history,omitempty"`
	BilledUsage *BilledLLMUsage    `json:"billed_usage,omitempty"`
	ChunkIndex  int                `json:"chunk_index"` // used for streaming responses to identify the chunk index, will be 0 for non-streaming responses
	RawResponse interface{}        `json:"raw_response,omitempty"`
	CacheDebug  *BifrostCacheDebug `json:"cache_debug,omitempty"`
}

BifrostResponseExtraFields contains additional fields in a response.

type BifrostSpeech added in v1.1.11 

type BifrostSpeech struct {
	Usage *AudioLLMUsage `json:"usage,omitempty"`
	Audio []byte         `json:"audio"`

	*BifrostSpeechStreamResponse
}

type BifrostSpeechStreamResponse added in v1.1.11 

type BifrostSpeechStreamResponse struct {
	Type string `json:"type"`
}

type BifrostStream added in v1.1.8 

type BifrostStream struct {
	*BifrostResponse
	*BifrostError
}

BifrostStream represents a stream of responses from the Bifrost system. Either BifrostResponse or BifrostError will be non-nil.

type BifrostStreamDelta added in v1.1.8 

type BifrostStreamDelta struct {
	Role      *string    `json:"role,omitempty"`       // Only in the first chunk
	Content   *string    `json:"content,omitempty"`    // May be empty string or null
	Thought   *string    `json:"thought,omitempty"`    // May be empty string or null
	Refusal   *string    `json:"refusal,omitempty"`    // Refusal content if any
	ToolCalls []ToolCall `json:"tool_calls,omitempty"` // If tool calls used (supports incremental updates)
}

BifrostStreamDelta represents a delta in the stream response

type BifrostStreamResponseChoice added in v1.1.8 

type BifrostStreamResponseChoice struct {
	Delta BifrostStreamDelta `json:"delta"` // Partial message info
}

BifrostStreamResponseChoice represents a choice in the stream response

type BifrostTranscribe added in v1.1.11 

type BifrostTranscribe struct {
	// Common fields for both streaming and non-streaming
	Text     string                 `json:"text"`
	LogProbs []TranscriptionLogProb `json:"logprobs,omitempty"`
	Usage    *TranscriptionUsage    `json:"usage,omitempty"`

	// Embedded structs for specific fields only
	*BifrostTranscribeNonStreamResponse
	*BifrostTranscribeStreamResponse
}

BifrostTranscribe represents transcription response data

type BifrostTranscribeNonStreamResponse added in v1.1.11 

type BifrostTranscribeNonStreamResponse struct {
	Task     *string                `json:"task,omitempty"`     // e.g., "transcribe"
	Language *string                `json:"language,omitempty"` // e.g., "english"
	Duration *float64               `json:"duration,omitempty"` // Duration in seconds
	Words    []TranscriptionWord    `json:"words,omitempty"`
	Segments []TranscriptionSegment `json:"segments,omitempty"`
}

BifrostTranscribeNonStreamResponse represents non-streaming specific fields only

type BifrostTranscribeStreamResponse added in v1.1.11 

type BifrostTranscribeStreamResponse struct {
	Type  *string `json:"type,omitempty"`  // "transcript.text.delta" or "transcript.text.done"
	Delta *string `json:"delta,omitempty"` // For delta events
}

BifrostTranscribeStreamResponse represents streaming specific fields only

type BilledLLMUsage 

type BilledLLMUsage struct {
	PromptTokens     *float64 `json:"prompt_tokens,omitempty"`
	CompletionTokens *float64 `json:"completion_tokens,omitempty"`
	SearchUnits      *float64 `json:"search_units,omitempty"`
	Classifications  *float64 `json:"classifications,omitempty"`
}

BilledLLMUsage represents the billing information for token usage.

type Citation 

type Citation struct {
	StartIndex int          `json:"start_index"`
	EndIndex   int          `json:"end_index"`
	Title      string       `json:"title"`
	URL        *string      `json:"url,omitempty"`
	Sources    *interface{} `json:"sources,omitempty"`
	Type       *string      `json:"type,omitempty"`
}

Citation represents a citation in a response.

type CompletionTokensDetails 

type CompletionTokensDetails struct {
	ReasoningTokens          int `json:"reasoning_tokens,omitempty"`
	AudioTokens              int `json:"audio_tokens,omitempty"`
	AcceptedPredictionTokens int `json:"accepted_prediction_tokens,omitempty"`
	RejectedPredictionTokens int `json:"rejected_prediction_tokens,omitempty"`
}

CompletionTokensDetails provides detailed information about completion token usage. It is not provided by all model providers.

type ConcurrencyAndBufferSize 

type ConcurrencyAndBufferSize struct {
	Concurrency int `json:"concurrency"` // Number of concurrent operations. Also used as the initial pool size for the provider reponses.
	BufferSize  int `json:"buffer_size"` // Size of the buffer
}

ConcurrencyAndBufferSize represents configuration for concurrent operations and buffer sizes.

type ContentBlock added in v1.1.0 

type ContentBlock struct {
	Type       ContentBlockType  `json:"type"`
	Text       *string           `json:"text,omitempty"`
	ImageURL   *ImageURLStruct   `json:"image_url,omitempty"`
	InputAudio *InputAudioStruct `json:"input_audio,omitempty"`
}

type ContentBlockType added in v1.1.0 

type ContentBlockType string
const (
	ContentBlockTypeText       ContentBlockType = "text"
	ContentBlockTypeImage      ContentBlockType = "image_url"
	ContentBlockTypeInputAudio ContentBlockType = "input_audio"
)

type ContentLogProb 

type ContentLogProb struct {
	Bytes       []int     `json:"bytes"`
	LogProb     float64   `json:"logprob"`
	Token       string    `json:"token"`
	TopLogProbs []LogProb `json:"top_logprobs"`
}

ContentLogProb represents log probability information for content.

type CustomProviderConfig added in v1.1.26 

type CustomProviderConfig struct {
	CustomProviderKey string           `json:"-"`                  // Custom provider key, internally set by Bifrost
	BaseProviderType  ModelProvider    `json:"base_provider_type"` // Base provider type
	AllowedRequests   *AllowedRequests `json:"allowed_requests,omitempty"`
}

func (*CustomProviderConfig) IsOperationAllowed added in v1.1.26 

func (cpc *CustomProviderConfig) IsOperationAllowed(operation Operation) bool

IsOperationAllowed checks if a specific operation is allowed for this custom provider

type EmbeddingInput added in v1.1.7 

type EmbeddingInput struct {
	Text       *string
	Texts      []string
	Embedding  []int
	Embeddings [][]int
}

EmbeddingInput represents the input for an embedding request.

func (*EmbeddingInput) MarshalJSON added in v1.1.36 

func (e *EmbeddingInput) MarshalJSON() ([]byte, error)

func (*EmbeddingInput) UnmarshalJSON added in v1.1.36 

func (e *EmbeddingInput) UnmarshalJSON(data []byte) error

type ErrorField 

type ErrorField struct {
	Type    *string     `json:"type,omitempty"`
	Code    *string     `json:"code,omitempty"`
	Message string      `json:"message"`
	Error   error       `json:"error,omitempty"`
	Param   interface{} `json:"param,omitempty"`
	EventID *string     `json:"event_id,omitempty"`
}

ErrorField represents detailed error information.

type Fallback 

type Fallback struct {
	Provider ModelProvider `json:"provider"`
	Model    string        `json:"model"`
}

Fallback represents a fallback model to be used if the primary model is not available.

type Function 

type Function struct {
	Name        string             `json:"name"`        // Name of the function
	Description string             `json:"description"` // Description of the function
	Parameters  FunctionParameters `json:"parameters"`  // Parameters of the function
}

Function represents a function that can be called by the model.

type FunctionCall 

type FunctionCall struct {
	Name      *string `json:"name"`
	Arguments string  `json:"arguments"` // stringified json as retured by OpenAI, might not be a valid JSON always
}

FunctionCall represents a call to a function.

type FunctionParameters 

type FunctionParameters struct {
	Type        string                 `json:"type"`                  // Type of the parameters
	Description *string                `json:"description,omitempty"` // Description of the parameters
	Required    []string               `json:"required,omitempty"`    // Required parameter names
	Properties  map[string]interface{} `json:"properties,omitempty"`  // Parameter properties
	Enum        *[]string              `json:"enum,omitempty"`        // Enum values for the parameters
}

FunctionParameters represents the parameters for a function definition.

type ImageURLStruct added in v1.1.0 

type ImageURLStruct struct {
	URL    string  `json:"url"`
	Detail *string `json:"detail,omitempty"`
}

ImageContent represents image data in a message.

type InputAudioStruct added in v1.1.34 

type InputAudioStruct struct {
	Data   string  `json:"data"`
	Format *string `json:"format,omitempty"`
}

InputAudioStruct represents audio data in a message. Data carries the audio payload as a string (e.g., data URL or provider-accepted encoded content). Format is optional (e.g., "wav", "mp3"); when nil, providers may attempt auto-detection.

type Key 

type Key struct {
	ID               string            `json:"id"`                           // The unique identifier for the key (not used by bifrost, but can be used by users to identify the key)
	Value            string            `json:"value"`                        // The actual API key value
	Models           []string          `json:"models"`                       // List of models this key can access
	Weight           float64           `json:"weight"`                       // Weight for load balancing between multiple keys
	AzureKeyConfig   *AzureKeyConfig   `json:"azure_key_config,omitempty"`   // Azure-specific key configuration
	VertexKeyConfig  *VertexKeyConfig  `json:"vertex_key_config,omitempty"`  // Vertex-specific key configuration
	BedrockKeyConfig *BedrockKeyConfig `json:"bedrock_key_config,omitempty"` // AWS Bedrock-specific key configuration
}

Key represents an API key and its associated configuration for a provider. It contains the key value, supported models, and a weight for load balancing.

type LLMUsage 

type LLMUsage struct {
	PromptTokens            int                      `json:"prompt_tokens"`
	CompletionTokens        int                      `json:"completion_tokens"`
	TotalTokens             int                      `json:"total_tokens"`
	TokenDetails            *TokenDetails            `json:"prompt_tokens_details,omitempty"`
	CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
}

LLMUsage represents token usage information

type LogLevel 

type LogLevel string

LogLevel represents the severity level of a log message. Internally it maps to zerolog.Level for interoperability. 

const (
	LogLevelDebug LogLevel = "debug"
	LogLevelInfo  LogLevel = "info"
	LogLevelWarn  LogLevel = "warn"
	LogLevelError LogLevel = "error"
)

LogLevel constants for different severity levels.

type LogProb 

type LogProb struct {
	Bytes   []int   `json:"bytes,omitempty"`
	LogProb float64 `json:"logprob"`
	Token   string  `json:"token"`
}

LogProb represents the log probability of a token.

type LogProbs 

type LogProbs struct {
	Content []ContentLogProb      `json:"content,omitempty"`
	Refusal []LogProb             `json:"refusal,omitempty"`
	Text    TextCompletionLogProb `json:"text,omitempty"`
}

LogProbs represents the log probabilities for different aspects of a response.

type Logger 

type Logger interface {
	// Debug logs a debug-level message.
	// This is used for detailed debugging information that is typically only needed
	// during development or troubleshooting.
	Debug(msg string, args ...any)

	// Info logs an info-level message.
	// This is used for general informational messages about normal operation.
	Info(msg string, args ...any)

	// Warn logs a warning-level message.
	// This is used for potentially harmful situations that don't prevent normal operation.
	Warn(msg string, args ...any)

	// Error logs an error-level message.
	// This is used for serious problems that need attention and may prevent normal operation.
	Error(msg string, args ...any)

	// Fatal logs a fatal-level message.
	// This is used for critical situations that require immediate attention and will terminate the program.
	Fatal(msg string, args ...any)

	// SetLevel sets the log level for the logger.
	SetLevel(level LogLevel)

	// SetOutputType sets the output type for the logger.
	SetOutputType(outputType LoggerOutputType)
}

Logger defines the interface for logging operations in the Bifrost system. Implementations of this interface should provide methods for logging messages at different severity levels.

type LoggerOutputType added in v1.1.23 

type LoggerOutputType string

LoggerOutputType represents the output type of a logger. 

const (
	LoggerOutputTypeJSON   LoggerOutputType = "json"
	LoggerOutputTypePretty LoggerOutputType = "pretty"
)

LoggerOutputType constants for different output types.

type MCPClient added in v1.1.7 

type MCPClient struct {
	Name   string             `json:"name"`   // Unique name for this client
	Config MCPClientConfig    `json:"config"` // Tool filtering settings
	Tools  []string           `json:"tools"`  // Available tools mapped by name
	State  MCPConnectionState `json:"state"`  // Connection state
}

MCPClient represents a connected MCP client with its configuration and tools, and connection information, after it has been initialized. It is returned by GetMCPClients() method.

type MCPClientConfig added in v1.1.4 

type MCPClientConfig struct {
	Name             string            `json:"name"`                        // Client name
	ConnectionType   MCPConnectionType `json:"connection_type"`             // How to connect (HTTP, STDIO, SSE, or InProcess)
	ConnectionString *string           `json:"connection_string,omitempty"` // HTTP or SSE URL (required for HTTP or SSE connections)
	StdioConfig      *MCPStdioConfig   `json:"stdio_config,omitempty"`      // STDIO configuration (required for STDIO connections)
	InProcessServer  MCPServerInstance `json:"-"`                           // MCP server instance for in-process connections (Go package only)
	ToolsToSkip      []string          `json:"tools_to_skip,omitempty"`     // Tools to exclude from this client
	ToolsToExecute   []string          `json:"tools_to_execute,omitempty"`  // Tools to include from this client (if specified, only these are used)
}

MCPClientConfig defines tool filtering for an MCP client.

type MCPConfig added in v1.1.4 

type MCPConfig struct {
	ClientConfigs []MCPClientConfig `json:"client_configs,omitempty"` // Per-client execution configurations
}

MCPConfig represents the configuration for MCP integration in Bifrost. It enables tool auto-discovery and execution from local and external MCP servers.

type MCPConnectionState added in v1.1.7 

type MCPConnectionState string
const (
	MCPConnectionStateConnected    MCPConnectionState = "connected"    // Client is connected and ready to use
	MCPConnectionStateDisconnected MCPConnectionState = "disconnected" // Client is not connected
	MCPConnectionStateError        MCPConnectionState = "error"        // Client is in an error state, and cannot be used
)

type MCPConnectionType added in v1.1.4 

type MCPConnectionType string

MCPConnectionType defines the communication protocol for MCP connections 

const (
	MCPConnectionTypeHTTP      MCPConnectionType = "http"      // HTTP-based connection
	MCPConnectionTypeSTDIO     MCPConnectionType = "stdio"     // STDIO-based connection
	MCPConnectionTypeSSE       MCPConnectionType = "sse"       // Server-Sent Events connection
	MCPConnectionTypeInProcess MCPConnectionType = "inprocess" // In-process (in-memory) connection
)

type MCPServerInstance added in v1.1.18 

type MCPServerInstance interface{}

MCPServerInstance represents an MCP server instance for InProcess connections. This should be a *github.com/mark3labs/mcp-go/server.MCPServer instance. We use interface{} to avoid creating a dependency on the mcp-go package in schemas.

type MCPStdioConfig added in v1.1.4 

type MCPStdioConfig struct {
	Command string   `json:"command"` // Executable command to run
	Args    []string `json:"args"`    // Command line arguments
	Envs    []string `json:"envs"`    // Environment variables required
}

MCPStdioConfig defines how to launch a STDIO-based MCP server.

type MessageContent added in v1.1.0 

type MessageContent struct {
	ContentStr    *string
	ContentBlocks *[]ContentBlock
}

func (MessageContent) MarshalJSON added in v1.1.0 

func (mc MessageContent) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for MessageContent. It marshals either ContentStr or ContentBlocks directly without wrapping.

func (*MessageContent) UnmarshalJSON added in v1.1.0 

func (mc *MessageContent) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for MessageContent. It determines whether "content" is a string or array and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type ModelChatMessageRole 

type ModelChatMessageRole string

ModelChatMessageRole represents the role of a chat message 

const (
	ModelChatMessageRoleAssistant ModelChatMessageRole = "assistant"
	ModelChatMessageRoleUser      ModelChatMessageRole = "user"
	ModelChatMessageRoleSystem    ModelChatMessageRole = "system"
	ModelChatMessageRoleChatbot   ModelChatMessageRole = "chatbot"
	ModelChatMessageRoleTool      ModelChatMessageRole = "tool"
)

type ModelParameters 

type ModelParameters struct {
	ToolChoice        *ToolChoice `json:"tool_choice,omitempty"`         // Whether to call a tool
	Tools             *[]Tool     `json:"tools,omitempty"`               // Tools to use
	Temperature       *float64    `json:"temperature,omitempty"`         // Controls randomness in the output
	TopP              *float64    `json:"top_p,omitempty"`               // Controls diversity via nucleus sampling
	TopK              *int        `json:"top_k,omitempty"`               // Controls diversity via top-k sampling
	MaxTokens         *int        `json:"max_tokens,omitempty"`          // Maximum number of tokens to generate
	StopSequences     *[]string   `json:"stop_sequences,omitempty"`      // Sequences that stop generation
	PresencePenalty   *float64    `json:"presence_penalty,omitempty"`    // Penalizes repeated tokens
	FrequencyPenalty  *float64    `json:"frequency_penalty,omitempty"`   // Penalizes frequent tokens
	ParallelToolCalls *bool       `json:"parallel_tool_calls,omitempty"` // Enables parallel tool calls
	EncodingFormat    *string     `json:"encoding_format,omitempty"`     // Format for embedding output (e.g., "float", "base64")
	Dimensions        *int        `json:"dimensions,omitempty"`          // Number of dimensions for embedding output
	User              *string     `json:"user,omitempty"`                // User identifier for tracking
	// Dynamic parameters that can be provider-specific, they are directly
	// added to the request as is.
	ExtraParams map[string]interface{} `json:"-"`
}

ModelParameters represents the parameters that can be used to configure your request to the model. Bifrost follows a standard set of parameters which mapped to the provider's parameters.

type ModelProvider 

type ModelProvider string

ModelProvider represents the different AI model providers supported by Bifrost. 

const (
	OpenAI     ModelProvider = "openai"
	Azure      ModelProvider = "azure"
	Anthropic  ModelProvider = "anthropic"
	Bedrock    ModelProvider = "bedrock"
	Cohere     ModelProvider = "cohere"
	Vertex     ModelProvider = "vertex"
	Mistral    ModelProvider = "mistral"
	Ollama     ModelProvider = "ollama"
	Groq       ModelProvider = "groq"
	SGL        ModelProvider = "sgl"
	Parasail   ModelProvider = "parasail"
	Cerebras   ModelProvider = "cerebras"
	Gemini     ModelProvider = "gemini"
	OpenRouter ModelProvider = "openrouter"
)

type NetworkConfig 

type NetworkConfig struct {
	// BaseURL is supported for OpenAI, Anthropic, Cohere, Mistral, and Ollama providers (required for Ollama)
	BaseURL                        string            `json:"base_url,omitempty"`                 // Base URL for the provider (optional)
	ExtraHeaders                   map[string]string `json:"extra_headers,omitempty"`            // Additional headers to include in requests (optional)
	DefaultRequestTimeoutInSeconds int               `json:"default_request_timeout_in_seconds"` // Default timeout for requests
	MaxRetries                     int               `json:"max_retries"`                        // Maximum number of retries
	RetryBackoffInitial            time.Duration     `json:"retry_backoff_initial"`              // Initial backoff duration
	RetryBackoffMax                time.Duration     `json:"retry_backoff_max"`                  // Maximum backoff duration
}

NetworkConfig represents the network configuration for provider connections. ExtraHeaders is automatically copied during provider initialization to prevent data races.

type Operation added in v1.1.26 

type Operation string
const (
	OperationTextCompletion       Operation = "text_completion"
	OperationChatCompletion       Operation = "chat_completion"
	OperationChatCompletionStream Operation = "chat_completion_stream"
	OperationEmbedding            Operation = "embedding"
	OperationSpeech               Operation = "speech"
	OperationSpeechStream         Operation = "speech_stream"
	OperationTranscription        Operation = "transcription"
	OperationTranscriptionStream  Operation = "transcription_stream"
)

type Plugin 

type Plugin interface {
	// GetName returns the name of the plugin.
	GetName() string

	// PreHook is called before a request is processed by a provider.
	// It allows plugins to modify the request before it is sent to the provider.
	// The context parameter can be used to maintain state across plugin calls.
	// Returns the modified request, an optional short-circuit decision, and any error that occurred during processing.
	PreHook(ctx *context.Context, req *BifrostRequest) (*BifrostRequest, *PluginShortCircuit, error)

	// PostHook is called after a response is received from a provider or a PreHook short-circuit.
	// It allows plugins to modify the response and/or error before it is returned to the caller.
	// Plugins can recover from errors (set error to nil and provide a response), or invalidate a response (set response to nil and provide an error).
	// Returns the modified response, bifrost error, and any error that occurred during processing.
	PostHook(ctx *context.Context, result *BifrostResponse, err *BifrostError) (*BifrostResponse, *BifrostError, error)

	// Cleanup is called on bifrost shutdown.
	// It allows plugins to clean up any resources they have allocated.
	// Returns any error that occurred during cleanup, which will be logged as a warning by the Bifrost instance.
	Cleanup() error
}

type PluginConfig added in v1.1.23 

type PluginConfig struct {
	Enabled bool   `json:"enabled"`
	Name    string `json:"name"`
	Config  any    `json:"config,omitempty"`
}

PluginConfig is the configuration for a plugin. It contains the name of the plugin, whether it is enabled, and the configuration for the plugin.

type PluginShortCircuit added in v1.1.5 

type PluginShortCircuit struct {
	Response *BifrostResponse    // If set, short-circuit with this response (skips provider call)
	Stream   chan *BifrostStream // If set, short-circuit with this stream (skips provider call)
	Error    *BifrostError       // If set, short-circuit with this error (can set AllowFallbacks field)
}

PluginShortCircuit represents a plugin's decision to short-circuit the normal flow. It can contain either a response (success short-circuit), a stream (streaming short-circuit), or an error (error short-circuit).

type PostHookRunner added in v1.1.8 

type PostHookRunner func(ctx *context.Context, result *BifrostResponse, err *BifrostError) (*BifrostResponse, *BifrostError)

type Provider 

type Provider interface {
	// GetProviderKey returns the provider's identifier
	GetProviderKey() ModelProvider
	// TextCompletion performs a text completion request
	TextCompletion(ctx context.Context, model string, key Key, text string, params *ModelParameters) (*BifrostResponse, *BifrostError)
	// ChatCompletion performs a chat completion request
	ChatCompletion(ctx context.Context, model string, key Key, messages []BifrostMessage, params *ModelParameters) (*BifrostResponse, *BifrostError)
	// ChatCompletionStream performs a chat completion stream request
	ChatCompletionStream(ctx context.Context, postHookRunner PostHookRunner, model string, key Key, messages []BifrostMessage, params *ModelParameters) (chan *BifrostStream, *BifrostError)
	// Embedding performs an embedding request
	Embedding(ctx context.Context, model string, key Key, input *EmbeddingInput, params *ModelParameters) (*BifrostResponse, *BifrostError)
	// Speech performs a text to speech request
	Speech(ctx context.Context, model string, key Key, input *SpeechInput, params *ModelParameters) (*BifrostResponse, *BifrostError)
	// SpeechStream performs a text to speech stream request
	SpeechStream(ctx context.Context, postHookRunner PostHookRunner, model string, key Key, input *SpeechInput, params *ModelParameters) (chan *BifrostStream, *BifrostError)
	// Transcription performs a transcription request
	Transcription(ctx context.Context, model string, key Key, input *TranscriptionInput, params *ModelParameters) (*BifrostResponse, *BifrostError)
	// TranscriptionStream performs a transcription stream request
	TranscriptionStream(ctx context.Context, postHookRunner PostHookRunner, model string, key Key, input *TranscriptionInput, params *ModelParameters) (chan *BifrostStream, *BifrostError)
}

Provider defines the interface for AI model providers.

type ProviderConfig 

type ProviderConfig struct {
	NetworkConfig            NetworkConfig            `json:"network_config"`              // Network configuration
	ConcurrencyAndBufferSize ConcurrencyAndBufferSize `json:"concurrency_and_buffer_size"` // Concurrency settings
	// Logger instance, can be provided by the user or bifrost default logger is used if not provided
	Logger               Logger                `json:"-"`
	ProxyConfig          *ProxyConfig          `json:"proxy_config,omitempty"` // Proxy configuration
	SendBackRawResponse  bool                  `json:"send_back_raw_response"` // Send raw response back in the bifrost response (default: false)
	CustomProviderConfig *CustomProviderConfig `json:"custom_provider_config,omitempty"`
}

ProviderConfig represents the complete configuration for a provider. An array of ProviderConfig needs to be provided in GetConfigForProvider in your account interface implementation.

func (*ProviderConfig) CheckAndSetDefaults added in v1.0.5 

func (config *ProviderConfig) CheckAndSetDefaults()

type ProxyConfig 

type ProxyConfig struct {
	Type     ProxyType `json:"type"`     // Type of proxy to use
	URL      string    `json:"url"`      // URL of the proxy server
	Username string    `json:"username"` // Username for proxy authentication
	Password string    `json:"password"` // Password for proxy authentication
}

ProxyConfig holds the configuration for proxy settings.

type ProxyType 

type ProxyType string

ProxyType defines the type of proxy to use for connections. 

const (
	// NoProxy indicates no proxy should be used
	NoProxy ProxyType = "none"
	// HttpProxy indicates an HTTP proxy should be used
	HttpProxy ProxyType = "http"
	// Socks5Proxy indicates a SOCKS5 proxy should be used
	Socks5Proxy ProxyType = "socks5"
	// EnvProxy indicates the proxy should be read from environment variables
	EnvProxy ProxyType = "environment"
)

type RequestInput 

type RequestInput struct {
	TextCompletionInput *string             `json:"text_completion_input,omitempty"`
	ChatCompletionInput *[]BifrostMessage   `json:"chat_completion_input,omitempty"`
	EmbeddingInput      *EmbeddingInput     `json:"embedding_input,omitempty"`
	SpeechInput         *SpeechInput        `json:"speech_input,omitempty"`
	TranscriptionInput  *TranscriptionInput `json:"transcription_input,omitempty"`
}

RequestInput represents the input for a model request, which can be either a text completion, a chat completion, an embedding request, a speech request, or a transcription request.

type RequestType added in v1.1.34 

type RequestType string

RequestType represents the type of request being made to a provider. 

const (
	TextCompletionRequest       RequestType = "text_completion"
	ChatCompletionRequest       RequestType = "chat_completion"
	ChatCompletionStreamRequest RequestType = "chat_completion_stream"
	EmbeddingRequest            RequestType = "embedding"
	SpeechRequest               RequestType = "speech"
	SpeechStreamRequest         RequestType = "speech_stream"
	TranscriptionRequest        RequestType = "transcription"
	TranscriptionStreamRequest  RequestType = "transcription_stream"
)

type SpeechInput added in v1.1.11 

type SpeechInput struct {
	Input          string           `json:"input"`
	VoiceConfig    SpeechVoiceInput `json:"voice"`
	Instructions   string           `json:"instructions,omitempty"`
	ResponseFormat string           `json:"response_format,omitempty"` // Default is "mp3"
}

SpeechInput represents the input for a speech request.

type SpeechVoiceInput added in v1.1.11 

type SpeechVoiceInput struct {
	Voice            *string
	MultiVoiceConfig []VoiceConfig
}

func (SpeechVoiceInput) MarshalJSON added in v1.1.11 

func (tc SpeechVoiceInput) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for SpeechVoiceInput. It marshals either Voice or MultiVoiceConfig directly without wrapping.

func (*SpeechVoiceInput) UnmarshalJSON added in v1.1.11 

func (tc *SpeechVoiceInput) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for SpeechVoiceInput. It determines whether "voice" is a string or a VoiceConfig object/array and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type StreamControl added in v1.1.15 

type StreamControl struct {
	LogError   *bool `json:"log_error,omitempty"`   // Optional: Controls logging of error
	SkipStream *bool `json:"skip_stream,omitempty"` // Optional: Controls skipping of stream chunk
}

type TextCompletionLogProb 

type TextCompletionLogProb struct {
	TextOffset    []int                `json:"text_offset"`
	TokenLogProbs []float64            `json:"token_logprobs"`
	Tokens        []string             `json:"tokens"`
	TopLogProbs   []map[string]float64 `json:"top_logprobs"`
}

TextCompletionLogProb represents log probability information for text completion.

type TokenDetails 

type TokenDetails struct {
	CachedTokens int `json:"cached_tokens,omitempty"`
	AudioTokens  int `json:"audio_tokens,omitempty"`
}

TokenDetails provides detailed information about token usage. It is not provided by all model providers.

type Tool 

type Tool struct {
	ID       *string  `json:"id,omitempty"` // Optional tool identifier
	Type     string   `json:"type"`         // Type of the tool
	Function Function `json:"function"`     // Function definition
}

Tool represents a tool that can be used with the model.

type ToolCall 

type ToolCall struct {
	Type     *string      `json:"type,omitempty"`
	ID       *string      `json:"id,omitempty"`
	Function FunctionCall `json:"function"`
}

ToolCall represents a tool call in a message

type ToolChoice 

type ToolChoice struct {
	ToolChoiceStr    *string
	ToolChoiceStruct *ToolChoiceStruct
}

ToolChoice represents how a tool should be chosen for a request. (either a string or a struct)

func (ToolChoice) MarshalJSON added in v1.1.3 

func (tc ToolChoice) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for ToolChoice. It marshals either ToolChoiceStr or ToolChoiceStruct directly without wrapping.

func (*ToolChoice) UnmarshalJSON added in v1.1.3 

func (tc *ToolChoice) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ToolChoice. It determines whether "tool_choice" is a string or struct and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type ToolChoiceFunction 

type ToolChoiceFunction struct {
	Name string `json:"name"` // Name of the function to call
}

ToolChoiceFunction represents a specific function to be called.

type ToolChoiceStruct added in v1.1.3 

type ToolChoiceStruct struct {
	Type     ToolChoiceType     `json:"type"`               // Type of tool choice
	Function ToolChoiceFunction `json:"function,omitempty"` // Function to call if type is ToolChoiceTypeFunction
}

ToolChoiceStruct represents a specific tool choice.

type ToolChoiceType 

type ToolChoiceType string

Combined tool choices for all providers, make sure to check the provider's documentation to see which tool choices are supported. 

const (
	// ToolChoiceTypeNone means no tool will be called
	ToolChoiceTypeNone ToolChoiceType = "none"
	// ToolChoiceTypeAuto means the model can choose whether to call a tool
	ToolChoiceTypeAuto ToolChoiceType = "auto"
	// ToolChoiceTypeAny means any tool can be called
	ToolChoiceTypeAny ToolChoiceType = "any"
	// ToolChoiceTypeFunction means a specific tool must be called (converted to "tool" for Anthropic)
	ToolChoiceTypeFunction ToolChoiceType = "function"
	// ToolChoiceTypeRequired means a tool must be called
	ToolChoiceTypeRequired ToolChoiceType = "required"
)

type ToolMessage added in v1.0.9 

type ToolMessage struct {
	ToolCallID *string `json:"tool_call_id,omitempty"`
}

ToolMessage represents a message from a tool

type TranscriptionInput added in v1.1.11 

type TranscriptionInput struct {
	File           []byte  `json:"file"`
	Language       *string `json:"language,omitempty"`
	Prompt         *string `json:"prompt,omitempty"`
	ResponseFormat *string `json:"response_format,omitempty"` // Default is "json"
	Format         *string `json:"file_format,omitempty"`     // Type of file, not required in openai, but required in gemini
}

type TranscriptionLogProb added in v1.1.11 

type TranscriptionLogProb struct {
	Token   string  `json:"token"`
	LogProb float64 `json:"logprob"`
	Bytes   []int   `json:"bytes"`
}

TranscriptionLogProb represents log probability information for transcription

type TranscriptionSegment added in v1.1.11 

type TranscriptionSegment struct {
	ID               int     `json:"id"`
	Seek             int     `json:"seek"`
	Start            float64 `json:"start"`
	End              float64 `json:"end"`
	Text             string  `json:"text"`
	Tokens           []int   `json:"tokens"`
	Temperature      float64 `json:"temperature"`
	AvgLogProb       float64 `json:"avg_logprob"`
	CompressionRatio float64 `json:"compression_ratio"`
	NoSpeechProb     float64 `json:"no_speech_prob"`
}

TranscriptionSegment represents segment-level transcription information

type TranscriptionUsage added in v1.1.11 

type TranscriptionUsage struct {
	Type              string             `json:"type"` // "tokens" or "duration"
	InputTokens       *int               `json:"input_tokens,omitempty"`
	InputTokenDetails *AudioTokenDetails `json:"input_token_details,omitempty"`
	OutputTokens      *int               `json:"output_tokens,omitempty"`
	TotalTokens       *int               `json:"total_tokens,omitempty"`
	Seconds           *int               `json:"seconds,omitempty"` // For duration-based usage
}

TranscriptionUsage represents usage information for transcription

type TranscriptionWord added in v1.1.11 

type TranscriptionWord struct {
	Word  string  `json:"word"`
	Start float64 `json:"start"`
	End   float64 `json:"end"`
}

TranscriptionWord represents word-level timing information

type VertexKeyConfig added in v1.1.9 

type VertexKeyConfig struct {
	ProjectID       string `json:"project_id,omitempty"`
	Region          string `json:"region,omitempty"`
	AuthCredentials string `json:"auth_credentials,omitempty"`
}

VertexKeyConfig represents the Vertex-specific configuration. It contains Vertex-specific settings required for authentication and service access.

type VoiceConfig added in v1.1.11 

type VoiceConfig struct {
	Speaker string `json:"speaker"`
	Voice   string `json:"voice"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.