cerebras

package
v0.0.0-...-61b5e6f Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 4, 2025 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package cerebras provides a Go client library for the Cerebras Inference API.

The Cerebras Inference API offers high-performance language model inference with industry-leading speed. This SDK provides a simple and idiomatic Go interface to all Cerebras API endpoints.

Basic Usage

Create a client and make a chat completion request: 

client := cerebras.NewClient("your-api-key")
// or
client, err := cerebras.NewClientFromEnv()

resp, err := client.Chat.Create(context.Background(), &cerebras.ChatCompletionRequest{
	Model: cerebras.ModelLlama31_8B,
	Messages: []cerebras.Message{
		cerebras.NewUserMessage("Hello, how are you?"),
	},
})

Streaming

The SDK supports streaming responses for real-time output: 

stream, err := client.Chat.CreateStream(context.Background(), &cerebras.ChatCompletionRequest{
	Model: cerebras.ModelLlama31_8B,
	Messages: []cerebras.Message{
		cerebras.NewUserMessage("Tell me a story"),
	},
})
defer stream.Close()

for {
	chunk, err := stream.Next()
	if err == io.EOF {
		break
	}
	// Process chunk
}

Error Handling

The SDK provides typed errors for different failure scenarios: 

resp, err := client.Chat.Create(ctx, request)
if err != nil {
	switch e := err.(type) {
	case *cerebras.RateLimitError:
		// Handle rate limit
	case *cerebras.AuthenticationError:
		// Handle auth error
	default:
		// Handle other errors
	}
}

Tool Use

Define and use tools for function calling: 

tools := []cerebras.Tool{
	cerebras.CreateWeatherTool(),
}

request := &cerebras.ChatCompletionRequest{
	Model: cerebras.ModelLlama31_8B,
	Messages: messages,
	Tools: tools,
	ToolChoice: cerebras.ToolChoiceAuto,
}

Structured Outputs

Use JSON schemas for structured responses: 

schema := cerebras.CreatePersonSchema()
request.WithJSONSchema("person", schema)

Index

Constants

View Source 
const (
	ModelLlama4Scout17B = "llama-4-scout-17b-16e-instruct"
	ModelLlama31_8B     = "llama3.1-8b"
	ModelLlama33_70B    = "llama-3.3-70b"
	ModelQwen3_32B      = "qwen-3-32b"
	ModelDeepSeekR1     = "deepseek-r1-distill-llama-70b"
)

Helper constants for available models

Variables

View Source 
var (
	ToolChoiceNone     = ToolChoice{Type: "none"}
	ToolChoiceAuto     = ToolChoice{Type: "auto"}
	ToolChoiceRequired = ToolChoice{Type: "required"}
)

Tool choice constants

Functions

func Bool 

func Bool(v bool) *bool

func CreatePersonSchema 

func CreatePersonSchema() map[string]interface{}

CreatePersonSchema creates a schema for a person object

func CreateResponseSchema 

func CreateResponseSchema() map[string]interface{}

CreateResponseSchema creates a schema for API responses

func Float32 

func Float32(v float32) *float32

Helper functions

func GenerateSchema 

func GenerateSchema(v interface{}) (map[string]interface{}, error)

GenerateSchema attempts to generate a JSON schema from a Go type

func Int 

func Int(v int) *int

func ParseToolCall 

func ParseToolCall(toolCall ToolCall, v interface{}) error

ParseToolCall parses the arguments of a tool call into the provided interface

func String 

func String(v string) *string

func ValidateJSONSchema 

func ValidateJSONSchema(schema map[string]interface{}) error

ValidateJSONSchema validates that a JSON schema is well-formed

Types

type APIConnectionError 

type APIConnectionError struct{ APIError }

Error types matching SDK error hierarchy

type APIError 

type APIError struct {
	StatusCode int
	Message    string
	Type       string
	Code       string
	Raw        json.RawMessage
}

Error types

func (*APIError) Error 

func (e *APIError) Error() string

type APIStatusError 

type APIStatusError struct{ APIError }

Error types matching SDK error hierarchy

type APITimeoutError 

type APITimeoutError struct{ APIConnectionError }

Error types matching SDK error hierarchy

type AuthenticationError 

type AuthenticationError struct{ APIStatusError }

Error types matching SDK error hierarchy

type BadRequestError 

type BadRequestError struct{ APIStatusError }

Error types matching SDK error hierarchy

type ChatCompletionChunk 

type ChatCompletionChunk struct {
	ID                string   `json:"id"`
	Object            string   `json:"object"`
	Created           int64    `json:"created"`
	Model             string   `json:"model"`
	SystemFingerprint string   `json:"system_fingerprint"`
	Choices           []Choice `json:"choices"`
}

Streaming types

type ChatCompletionRequest 

type ChatCompletionRequest struct {
	Model               string          `json:"model"`
	Messages            []Message       `json:"messages"`
	MaxCompletionTokens *int            `json:"max_completion_tokens,omitempty"`
	Temperature         *float32        `json:"temperature,omitempty"`
	TopP                *float32        `json:"top_p,omitempty"`
	Stream              bool            `json:"stream,omitempty"`
	Seed                *int            `json:"seed,omitempty"`
	Stop                []string        `json:"stop,omitempty"`
	User                string          `json:"user,omitempty"`
	LogProbs            *bool           `json:"logprobs,omitempty"`
	TopLogProbs         *int            `json:"top_logprobs,omitempty"`
	ResponseFormat      *ResponseFormat `json:"response_format,omitempty"`
	Tools               []Tool          `json:"tools,omitempty"`
	ToolChoice          interface{}     `json:"tool_choice,omitempty"`
	ParallelToolCalls   *bool           `json:"parallel_tool_calls,omitempty"`
}

Chat types

func (*ChatCompletionRequest) WithJSONMode 

func (r *ChatCompletionRequest) WithJSONMode() *ChatCompletionRequest

WithJSONMode enables JSON mode

func (*ChatCompletionRequest) WithJSONSchema 

func (r *ChatCompletionRequest) WithJSONSchema(name string, schema map[string]interface{}) *ChatCompletionRequest

WithJSONSchema enables structured output with a JSON schema

func (*ChatCompletionRequest) WithResponseFormat 

func (r *ChatCompletionRequest) WithResponseFormat(format *ResponseFormat) *ChatCompletionRequest

WithResponseFormat sets the response format for structured outputs

func (*ChatCompletionRequest) WithTools 

func (r *ChatCompletionRequest) WithTools(tools []Tool) *ChatCompletionRequest

WithTools adds tool definitions to a chat request

type ChatCompletionResponse 

type ChatCompletionResponse struct {
	ID                string   `json:"id"`
	Object            string   `json:"object"`
	Created           int64    `json:"created"`
	Model             string   `json:"model"`
	SystemFingerprint string   `json:"system_fingerprint"`
	Choices           []Choice `json:"choices"`
	Usage             Usage    `json:"usage"`
	TimeInfo          TimeInfo `json:"time_info"`
}

func CollectChatStream 

func CollectChatStream(stream *ChatCompletionStream) (*ChatCompletionResponse, error)

Helper function to collect all chunks from a chat stream

type ChatCompletionStream 

type ChatCompletionStream struct {
	// contains filtered or unexported fields
}

ChatCompletionStream handles streaming chat completions

func (*ChatCompletionStream) Close 

func (s *ChatCompletionStream) Close() error

func (*ChatCompletionStream) Next 

func (s *ChatCompletionStream) Next() (*ChatCompletionChunk, error)

type ChatService 

type ChatService struct {
	// contains filtered or unexported fields
}

ChatService handles chat completion operations

func (*ChatService) Create 

func (s *ChatService) Create(ctx context.Context, request *ChatCompletionRequest) (*ChatCompletionResponse, error)

Create sends a chat completion request

func (*ChatService) CreateStream 

func (s *ChatService) CreateStream(ctx context.Context, request *ChatCompletionRequest) (*ChatCompletionStream, error)

CreateStream sends a streaming chat completion request

func (*ChatService) CreateWithOptions 

func (s *ChatService) CreateWithOptions(ctx context.Context, request *ChatCompletionRequest) (interface{}, error)

CreateWithOptions is a convenience method that handles both streaming and non-streaming

type Choice 

type Choice struct {
	Index        int       `json:"index"`
	Message      Message   `json:"message"`
	Delta        *Delta    `json:"delta,omitempty"`
	FinishReason string    `json:"finish_reason"`
	LogProbs     *LogProbs `json:"logprobs,omitempty"`
}

type Client 

type Client struct {
	Chat        *ChatService
	Completions *CompletionsService
	Models      *ModelsService
	// contains filtered or unexported fields
}

func NewClient 

func NewClient(apiKey string) *Client

func NewClientFromEnv 

func NewClientFromEnv() (*Client, error)

func NewClientWithConfig 

func NewClientWithConfig(config Config) *Client

type CompletionChoice 

type CompletionChoice struct {
	Index        int       `json:"index"`
	Text         string    `json:"text"`
	FinishReason string    `json:"finish_reason"`
	LogProbs     *LogProbs `json:"logprobs"`
}

type CompletionChunk 

type CompletionChunk struct {
	ID                string             `json:"id"`
	Object            string             `json:"object"`
	Created           int64              `json:"created"`
	Model             string             `json:"model"`
	SystemFingerprint string             `json:"system_fingerprint"`
	Choices           []CompletionChoice `json:"choices"`
}

type CompletionRequest 

type CompletionRequest struct {
	Model           string   `json:"model"`
	Prompt          string   `json:"prompt"`
	MaxTokens       *int     `json:"max_tokens,omitempty"`
	MinTokens       *int     `json:"min_tokens,omitempty"`
	Temperature     *float32 `json:"temperature,omitempty"`
	TopP            *float32 `json:"top_p,omitempty"`
	Stream          bool     `json:"stream,omitempty"`
	Seed            *int     `json:"seed,omitempty"`
	Stop            []string `json:"stop,omitempty"`
	Echo            bool     `json:"echo,omitempty"`
	ReturnRawTokens bool     `json:"return_raw_tokens,omitempty"`
	GrammarRoot     *string  `json:"grammar_root,omitempty"`
	User            string   `json:"user,omitempty"`
}

Completion types

func NewCompletionRequest 

func NewCompletionRequest(model, prompt string) *CompletionRequest

NewCompletionRequest creates a new completion request with defaults

func (*CompletionRequest) WithEcho 

func (r *CompletionRequest) WithEcho(echo bool) *CompletionRequest

WithEcho enables echoing the prompt in the response

func (*CompletionRequest) WithGrammarRoot 

func (r *CompletionRequest) WithGrammarRoot(grammar string) *CompletionRequest

WithGrammarRoot sets the grammar root for structured generation

func (*CompletionRequest) WithMaxTokens 

func (r *CompletionRequest) WithMaxTokens(tokens int) *CompletionRequest

WithMaxTokens sets the maximum tokens to generate

func (*CompletionRequest) WithMinTokens 

func (r *CompletionRequest) WithMinTokens(tokens int) *CompletionRequest

WithMinTokens sets the minimum tokens to generate

func (*CompletionRequest) WithReturnRawTokens 

func (r *CompletionRequest) WithReturnRawTokens(returnTokens bool) *CompletionRequest

WithReturnRawTokens enables returning raw tokens instead of text

func (*CompletionRequest) WithSeed 

func (r *CompletionRequest) WithSeed(seed int) *CompletionRequest

WithSeed sets the random seed for deterministic generation

func (*CompletionRequest) WithStop 

func (r *CompletionRequest) WithStop(sequences ...string) *CompletionRequest

WithStop adds stop sequences

func (*CompletionRequest) WithTemperature 

func (r *CompletionRequest) WithTemperature(temp float32) *CompletionRequest

WithTemperature sets the sampling temperature

func (*CompletionRequest) WithTopP 

func (r *CompletionRequest) WithTopP(topP float32) *CompletionRequest

WithTopP sets the nucleus sampling parameter

type CompletionResponse 

type CompletionResponse struct {
	ID                string             `json:"id"`
	Object            string             `json:"object"`
	Created           int64              `json:"created"`
	Model             string             `json:"model"`
	SystemFingerprint string             `json:"system_fingerprint"`
	Choices           []CompletionChoice `json:"choices"`
	Usage             Usage              `json:"usage"`
	TimeInfo          TimeInfo           `json:"time_info"`
}

func CollectCompletionStream 

func CollectCompletionStream(stream *CompletionStream) (*CompletionResponse, error)

Helper function to collect all chunks from a completion stream

type CompletionStream 

type CompletionStream struct {
	// contains filtered or unexported fields
}

CompletionStream handles streaming completions

func (*CompletionStream) Close 

func (s *CompletionStream) Close() error

func (*CompletionStream) Next 

func (s *CompletionStream) Next() (*CompletionChunk, error)

type CompletionsService 

type CompletionsService struct {
	// contains filtered or unexported fields
}

CompletionsService handles text completion operations

func (*CompletionsService) Create 

func (s *CompletionsService) Create(ctx context.Context, request *CompletionRequest) (*CompletionResponse, error)

Create sends a completion request

func (*CompletionsService) CreateStream 

func (s *CompletionsService) CreateStream(ctx context.Context, request *CompletionRequest) (*CompletionStream, error)

CreateStream sends a streaming completion request

func (*CompletionsService) CreateWithOptions 

func (s *CompletionsService) CreateWithOptions(ctx context.Context, request *CompletionRequest) (interface{}, error)

CreateWithOptions is a convenience method that handles both streaming and non-streaming

type Config 

type Config struct {
	APIKey     string
	BaseURL    string
	HTTPClient *http.Client
	Timeout    time.Duration
	MaxRetries int
	UserAgent  string
}

type Delta 

type Delta struct {
	Role      string     `json:"role,omitempty"`
	Content   string     `json:"content,omitempty"`
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
}

type Function 

type Function struct {
	Name        string                 `json:"name"`
	Description string                 `json:"description,omitempty"`
	Parameters  map[string]interface{} `json:"parameters"`
	Strict      bool                   `json:"strict,omitempty"`
}

type FunctionCall 

type FunctionCall struct {
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
}

type InternalServerError 

type InternalServerError struct{ APIStatusError }

Error types matching SDK error hierarchy

type JSONSchema 

type JSONSchema struct {
	Name   string                 `json:"name"`
	Strict bool                   `json:"strict"`
	Schema map[string]interface{} `json:"schema"`
}

type LogProbContent 

type LogProbContent struct {
	Token       string       `json:"token"`
	LogProb     float64      `json:"logprob"`
	Bytes       []int        `json:"bytes"`
	TopLogProbs []TopLogProb `json:"top_logprobs"`
}

type LogProbs 

type LogProbs struct {
	Content []LogProbContent `json:"content"`
}

type Message 

type Message struct {
	Role       string     `json:"role"`
	Content    string     `json:"content"`
	ToolCalls  []ToolCall `json:"tool_calls,omitempty"`
	ToolCallID string     `json:"tool_call_id,omitempty"`
}

func CreateToolResponse 

func CreateToolResponse(toolCallID string, response interface{}) (Message, error)

CreateToolResponse creates a tool response message

func NewAssistantMessage 

func NewAssistantMessage(content string) Message

NewAssistantMessage creates an assistant message

func NewChatMessage 

func NewChatMessage(role, content string) Message

NewChatMessage creates a new chat message

func NewSystemMessage 

func NewSystemMessage(content string) Message

NewSystemMessage creates a system message

func NewToolMessage 

func NewToolMessage(content, toolCallID string) Message

NewToolMessage creates a tool response message

func NewUserMessage 

func NewUserMessage(content string) Message

NewUserMessage creates a user message

type Model 

type Model struct {
	ID      string `json:"id"`
	Object  string `json:"object"`
	Created int64  `json:"created"`
	OwnedBy string `json:"owned_by"`
}

Model types

type ModelsResponse 

type ModelsResponse struct {
	Object string  `json:"object"`
	Data   []Model `json:"data"`
}

type ModelsService 

type ModelsService struct {
	// contains filtered or unexported fields
}

ModelsService handles operations related to models

func (*ModelsService) Get 

func (s *ModelsService) Get(ctx context.Context, modelID string) (*Model, error)

Get retrieves a specific model by ID

func (*ModelsService) List 

func (s *ModelsService) List(ctx context.Context) (*ModelsResponse, error)

List retrieves all available models

type NotFoundError 

type NotFoundError struct{ APIStatusError }

Error types matching SDK error hierarchy

type PermissionDeniedError 

type PermissionDeniedError struct{ APIStatusError }

Error types matching SDK error hierarchy

type PropertyBuilder 

type PropertyBuilder struct {
	// contains filtered or unexported fields
}

PropertyBuilder helps construct individual properties

func AnyOf 

func AnyOf(description string, types ...PropertyBuilder) PropertyBuilder

AnyOf creates a union type property

func Array 

func Array(description string, items PropertyBuilder) PropertyBuilder

Array creates an array property

func Boolean 

func Boolean(description string) PropertyBuilder

Boolean creates a boolean property

func Enum 

func Enum(description string, values ...string) PropertyBuilder

Enum creates an enum property

func Integer 

func Integer(description string) PropertyBuilder

Integer creates an integer property

func Number 

func Number(description string) PropertyBuilder

Number creates a number property

func Object 

func Object(description string) PropertyBuilder

Object creates an object property

func StringProp 

func StringProp(description string) PropertyBuilder

StringProp creates a string property

func (PropertyBuilder) Build 

func (pb PropertyBuilder) Build() map[string]interface{}

Build returns the constructed property

func (PropertyBuilder) WithMaxLength 

func (pb PropertyBuilder) WithMaxLength(max int) PropertyBuilder

WithMaxLength adds a maximum length constraint

func (PropertyBuilder) WithMaximum 

func (pb PropertyBuilder) WithMaximum(max float64) PropertyBuilder

WithMaximum adds a maximum value constraint

func (PropertyBuilder) WithMinLength 

func (pb PropertyBuilder) WithMinLength(min int) PropertyBuilder

WithMinLength adds a minimum length constraint

func (PropertyBuilder) WithMinimum 

func (pb PropertyBuilder) WithMinimum(min float64) PropertyBuilder

WithMinimum adds a minimum value constraint

func (PropertyBuilder) WithPattern 

func (pb PropertyBuilder) WithPattern(pattern string) PropertyBuilder

WithPattern adds a regex pattern constraint

func (PropertyBuilder) WithProperty 

func (pb PropertyBuilder) WithProperty(name string, prop PropertyBuilder, required bool) PropertyBuilder

WithProperty adds a property to an object type

type RateLimitError 

type RateLimitError struct{ APIStatusError }

Error types matching SDK error hierarchy

type ResponseFormat 

type ResponseFormat struct {
	Type       string      `json:"type"`
	JSONSchema *JSONSchema `json:"json_schema,omitempty"`
}

type SchemaBuilder 

type SchemaBuilder struct {
	// contains filtered or unexported fields
}

SchemaBuilder helps construct JSON schemas for structured outputs

func NewSchema 

func NewSchema() *SchemaBuilder

NewSchema creates a new schema builder

func (*SchemaBuilder) Build 

func (sb *SchemaBuilder) Build() map[string]interface{}

Build returns the constructed schema

func (*SchemaBuilder) WithDefinition 

func (sb *SchemaBuilder) WithDefinition(name string, def map[string]interface{}) *SchemaBuilder

WithDefinition adds a reusable definition

func (*SchemaBuilder) WithProperty 

func (sb *SchemaBuilder) WithProperty(name string, prop PropertyBuilder, required bool) *SchemaBuilder

WithProperty adds a property to the schema

func (*SchemaBuilder) WithRef 

func (sb *SchemaBuilder) WithRef(name, ref string, required bool) *SchemaBuilder

WithRef adds a reference property

type TimeInfo 

type TimeInfo struct {
	QueueTime      float64 `json:"queue_time"`
	PromptTime     float64 `json:"prompt_time"`
	CompletionTime float64 `json:"completion_time"`
	TotalTime      float64 `json:"total_time"`
	Created        int64   `json:"created"`
}

type Tool 

type Tool struct {
	Type     string   `json:"type"`
	Function Function `json:"function"`
}

func CreateCalculatorTool 

func CreateCalculatorTool() Tool

CreateCalculatorTool creates a calculator tool

func CreateSearchTool 

func CreateSearchTool() Tool

CreateSearchTool creates a web search tool

func CreateWeatherTool 

func CreateWeatherTool() Tool

CreateWeatherTool creates a weather lookup tool

type ToolBuilder 

type ToolBuilder struct {
	// contains filtered or unexported fields
}

ToolBuilder helps construct tool definitions

func NewTool 

func NewTool(name, description string) *ToolBuilder

NewTool creates a new tool builder

func (*ToolBuilder) Build 

func (tb *ToolBuilder) Build() Tool

Build returns the constructed tool

func (*ToolBuilder) Strict 

func (tb *ToolBuilder) Strict() *ToolBuilder

Strict enables strict mode for the tool

func (*ToolBuilder) WithArrayParameter 

func (tb *ToolBuilder) WithArrayParameter(name, description, itemType string, required bool) *ToolBuilder

WithArrayParameter adds an array parameter

func (*ToolBuilder) WithEnumParameter 

func (tb *ToolBuilder) WithEnumParameter(name, description string, values []string, required bool) *ToolBuilder

WithEnumParameter adds an enum parameter to the tool

func (*ToolBuilder) WithObjectParameter 

func (tb *ToolBuilder) WithObjectParameter(name, description string, properties map[string]interface{}, required bool) *ToolBuilder

WithObjectParameter adds a nested object parameter

func (*ToolBuilder) WithParameter 

func (tb *ToolBuilder) WithParameter(name, paramType, description string, required bool) *ToolBuilder

WithParameter adds a parameter to the tool

type ToolCall 

type ToolCall struct {
	ID       string       `json:"id"`
	Type     string       `json:"type"`
	Function FunctionCall `json:"function"`
}

type ToolChoice 

type ToolChoice struct {
	Type string
	Name string
}

ToolChoice represents the tool choice parameter

func SpecificTool 

func SpecificTool(name string) ToolChoice

SpecificTool creates a tool choice for a specific function

func (ToolChoice) MarshalJSON 

func (tc ToolChoice) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler for ToolChoice

type TopLogProb 

type TopLogProb struct {
	Token   string  `json:"token"`
	LogProb float64 `json:"logprob"`
	Bytes   []int   `json:"bytes"`
}

type UnprocessableEntityError 

type UnprocessableEntityError struct{ APIStatusError }

Error types matching SDK error hierarchy

type Usage 

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.