relay

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 3, 2026 License: MIT Imports: 22 Imported by: 0

README

relay

Unified LLM relay layer for calling 40+ AI/LLM providers through a single, framework-agnostic API with integrated usage metering, retry, and circuit breaking.

Structure

relay/
├── client.go          # Client: unified entry point (Chat, Embed, Image, ...)
├── go.mod             # Separate Go module
├── channel/           # 40 provider adaptors (see below)
├── common/            # Adaptor interface + RelayInfo + shared types
├── common_handler/    # Shared response handlers (rerank, etc.)
├── constant/          # API type constants
├── helper/            # SSE streaming + response ID helpers
├── meter/             # Usage metering (tokens, images, audio seconds)
├── realtime/          # WebSocket realtime API connectors
├── relaykit/          # DTO types, reason maps, relay converters
├── relaymode/         # Relay mode constants (chat, embed, image, ...)
├── service/           # Adaptor-to-relaykit bridge utilities
├── setting/           # Global relay settings stubs
├── task/              # Async task types
└── types/             # Price data and shared types

Key Types

// Client is the unified entry point for all AI API calls.
type Client struct { ... }

// Provider is the high-level interface for AI API providers.
type Provider interface {
    Name() string
    ApiType() int
    Adaptor() common.Adaptor
}

// ChatRequest is the unified chat completion request.
type ChatRequest struct {
    Model       string    `json:"model"`
    Messages    []Message `json:"messages"`
    Temperature *float64  `json:"temperature,omitempty"`
    MaxTokens   *int      `json:"max_tokens,omitempty"`
    Stream      bool      `json:"stream,omitempty"`
    Tools       []Tool    `json:"tools,omitempty"`
    // ...
}

// ChatResponse is the unified chat completion response.
type ChatResponse struct {
    ID       string       `json:"id"`
    Model    string       `json:"model"`
    Choices  []ChatChoice `json:"choices"`
    Usage    meter.Usage  `json:"usage"`
    Provider string       `json:"provider"`
}

// ChatStreamResult holds the stream channel and final usage.
type ChatStreamResult struct {
    Ch    chan ChatStreamChunk
    Usage meter.Usage
}

Client Methods

Method Description
Chat Synchronous chat completion
ChatStream Streaming chat completion (channel)
Embed Text embeddings
Image Image generation
Audio Audio transcription
AudioTranslation Audio translation
Rerank Document reranking
Responses OpenAI Responses API
Completions Legacy completions
Moderations Content moderation
SubmitTask Async task submission (Midjourney)
FetchTask Async task polling
MidjourneySubmit Midjourney image generation
SubmitSunoTask Suno music generation

Supported Channels (40)

advancedcustom  ai360       ali          aws          baidu
baidu_v2        claude      cloudflare   codex        cohere
coze            deepseek    dify         gemini       huggingface
jimeng          jina        lingyiwanwu  minimax      mistral
mokaai          moonshot    newapi       ollama       openai
openrouter      palm        perplexity   replicate    siliconflow
sub2api         submodel    tencent      vertex       volcengine
xai             xinference  xunfei       zhipu        zhipu_4v

Quick Start

import (
    "github.com/LingByte/ling-base/relay"
    "github.com/LingByte/ling-base/relay/meter"
    "github.com/LingByte/ling-base/relay/channel/openai"
)

client := relay.New(
    relay.WithProvider(openai.New("sk-xxx")),
    relay.WithMeter(meter.NewMemoryMeter()),
)

// Synchronous chat
resp, err := client.Chat(ctx, &relay.ChatRequest{
    Model:    "gpt-4o",
    Messages: []relay.Message{{Role: "user", Content: json.RawMessage(`"Hello"`)}},
})

// Streaming chat
result, err := client.ChatStream(ctx, &relay.ChatRequest{
    Model:    "gpt-4o",
    Messages: []relay.Message{{Role: "user", Content: json.RawMessage(`"Tell me a story"`)}},
    Stream:   true,
})
for chunk := range result.Ch {
    fmt.Print(chunk.Delta)
}

Sub-packages

Package Description
channel 40 provider adaptors (OpenAI, Claude, Gemini, etc.)
common Adaptor interface, RelayInfo, shared request types
common_handler Shared response handlers (rerank, etc.)
constant API type and mode constants
helper SSE streaming, response ID generation helpers
meter Usage metering (tokens, images, audio/video seconds)
realtime WebSocket realtime API connectors (OpenAI, etc.)
relaykit DTO types, reason maps, relay format converters
relaymode Relay mode constants (chat, embed, image, audio, ...)
service Adaptor-to-relaykit bridge and response utilities
setting Global relay settings stubs (overridable by app)
task Async task types
types Price data and shared types

Documentation

Overview

Package relay is the unified entry point for calling AI/LLM providers. It wraps the relay adaptor system with a clean, framework-agnostic API and integrates usage metering.

Quick start: 

import (
	"github.com/LingByte/ling-base/relay"
	"github.com/LingByte/ling-base/relay/meter"
	"github.com/LingByte/ling-base/relay/channel/openai"
)

client := relay.New(
	relay.WithProvider(openai.New("sk-xxx")),
	relay.WithMeter(meter.NewMemoryMeter()),
)
resp, err := client.Chat(ctx, &relay.ChatRequest{
	Model:    "gpt-4o",
	Messages: []relay.Message{{Role: "user", Content: json.RawMessage(`"Hello"`)}},
})

Index

Constants

View Source 
const (
	SunoActionMusic  = "MUSIC"
	SunoActionLyrics = "LYRICS"
)

SunoActionMusic and SunoActionLyrics are the supported Suno actions.

View Source 
const (
	BlockTypeText       = "text"
	BlockTypeToolUse    = "tool_use"
	BlockTypeToolResult = "tool_result"
	BlockTypeThinking   = "thinking"
	BlockTypeImage      = "image"
)

BlockType constants.

View Source 
const (
	ChunkTypeTextDelta     = "text_delta"
	ChunkTypeToolUseDelta  = "tool_use_delta"
	ChunkTypeThinkingDelta = "thinking_delta"
	ChunkTypeFinish        = "finish"
	ChunkTypeUsage         = "usage"
	ChunkTypeError         = "error"
)

Stream chunk type constants.

Variables

This section is empty.

Functions

func DefaultHTTPClient 

func DefaultHTTPClient() *http.Client

DefaultHTTPClient returns a production-ready HTTP client with sensible timeouts and connection pooling. The underlying *http.Client is created via common/netutil so that transport tuning stays centralised.

Types

type AlphaSearchRequest 

type AlphaSearchRequest struct {
	Model string          `json:"model"`
	Query json.RawMessage `json:"query"`
}

AlphaSearchRequest is the /v1/alpha/search request body.

type AudioRequest 

type AudioRequest struct {
	Model          string
	Input          json.RawMessage // text for TTS, file data for ASR
	Voice          string
	ResponseFormat string
	Speed          *float64
	Language       string
}

AudioRequest is the unified audio request for TTS and ASR.

type AudioResponse 

type AudioResponse struct {
	Data    []byte
	Usage   meter.Usage
	Headers http.Header
}

AudioResponse holds the audio response data.

type ChatChoice 

type ChatChoice struct {
	Index        int     `json:"index"`
	Message      Message `json:"message"`
	FinishReason string  `json:"finish_reason"`
}

ChatChoice is one choice in a chat completion response.

type ChatRequest 

type ChatRequest struct {
	Model           string          `json:"model"`
	Messages        []Message       `json:"messages"`
	Temperature     *float64        `json:"temperature,omitempty"`
	TopP            *float64        `json:"top_p,omitempty"`
	MaxTokens       *int            `json:"max_tokens,omitempty"`
	Stream          bool            `json:"stream,omitempty"`
	Tools           []Tool          `json:"tools,omitempty"`
	ToolChoice      json.RawMessage `json:"tool_choice,omitempty"`
	Stop            json.RawMessage `json:"stop,omitempty"`
	N               *int            `json:"n,omitempty"`
	User            string          `json:"user,omitempty"`
	ReasoningEffort string          `json:"reasoning_effort,omitempty"` // low/medium/high for reasoning models

	// System is the top-level system prompt (Anthropic Messages API style).
	// For OpenAI-compatible providers, it is prepended as a system message.
	// Empty = no system prompt.
	System string `json:"system,omitempty"`

	// Betas are provider-specific beta feature flags (e.g. Anthropic beta headers).
	// Passed through to the Anthropic channel as anthropic-beta headers.
	Betas []string `json:"betas,omitempty"`
}

ChatRequest is the unified chat completion request.

type ChatResponse 

type ChatResponse struct {
	ID       string       `json:"id"`
	Model    string       `json:"model"`
	Choices  []ChatChoice `json:"choices"`
	Usage    meter.Usage  `json:"usage"`
	Provider string       `json:"provider"`
}

ChatResponse is the unified chat completion response.

type ChatStreamChunk 

type ChatStreamChunk struct {
	Delta        string                 // content delta (text)
	Reasoning    string                 // reasoning_content delta (DeepSeek/o1-style)
	ToolCalls    []dto.ToolCallResponse // tool_call delta (OpenAI streaming format)
	FinishReason string                 // "stop" | "tool_calls" | "length" | ...
	Err          error
	Done         bool
	Usage        *meter.Usage // present on final chunk if provider reports it
}

ChatStreamChunk is one chunk in a streaming chat response.

type ChatStreamResult 

type ChatStreamResult struct {
	Ch    chan ChatStreamChunk
	Usage meter.Usage // filled after channel closes
}

ChatStreamResult holds the stream channel and final usage.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is the unified entry point. It routes requests to the configured provider, records usage via the Meter, and returns results.

func New 

func New(opts ...Option) *Client

New creates a new Client.

func (*Client) AlphaSearch 

func (c *Client) AlphaSearch(ctx context.Context, req *AlphaSearchRequest) (*ResponsesResponse, error)

AlphaSearch sends a web search request (/v1/alpha/search). The response body is returned as raw bytes.

func (*Client) Audio 

func (c *Client) Audio(ctx context.Context, req *AudioRequest, isTranscription bool) (*AudioResponse, error)

Audio sends an audio request (TTS or ASR) to the provider. The response body is returned as raw bytes (audio data for TTS, transcript JSON for ASR).

func (*Client) AudioTranslation 

func (c *Client) AudioTranslation(ctx context.Context, req *AudioRequest) (*AudioResponse, error)

AudioTranslation sends an audio translation request (ASR translation) to the provider. Similar to transcription but translates the audio to English.

func (*Client) Chat 

func (c *Client) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

Chat sends a chat completion request. If model fallback is configured via WithFallback, the Client retries the request with each fallback model in order until one succeeds or all are exhausted.

func (*Client) ChatStream 

func (c *Client) ChatStream(ctx context.Context, req *ChatRequest) (*ChatStreamResult, error)

ChatStream sends a streaming chat completion request. It returns a channel of chunks. The channel closes when the stream ends. After the channel closes, ChatStreamResult.Usage contains the final usage.

If model fallback is configured via WithFallback, the Client retries the request setup with each fallback model in order until one successfully establishes the stream. Fallback only applies to setup-time errors; once a stream has started it is not retried on mid-stream failures.

func (*Client) Completions 

func (c *Client) Completions(ctx context.Context, req *CompletionsRequest) (*CompletionsResponse, error)

Completions sends a legacy text completion request (/v1/completions).

func (*Client) Edit 

func (c *Client) Edit(ctx context.Context, req *EditRequest) (*CompletionsResponse, error)

Edit sends a legacy text edit request (/v1/edits).

func (*Client) Embed 

func (c *Client) Embed(ctx context.Context, req *EmbedRequest) (*EmbedResponse, error)

Embed sends an embedding request.

func (*Client) FetchSunoTask 

func (c *Client) FetchSunoTask(ctx context.Context, baseURL, apiKey string, taskIDs []string) ([]SunoTaskData, error)

FetchSunoTask fetches Suno task status by task IDs (batch supported).

func (*Client) FetchSunoTaskByID 

func (c *Client) FetchSunoTaskByID(ctx context.Context, baseURL, apiKey, taskID string) (*SunoTaskData, error)

FetchSunoTaskByID fetches a single Suno task by its ID.

func (*Client) FetchTask 

func (c *Client) FetchTask(ctx context.Context, tp TaskProvider, baseURL, key string, body map[string]any, proxy string) (*common.TaskInfo, error)

FetchTask polls the status of an async task.

func (*Client) GeminiChat 

func (c *Client) GeminiChat(ctx context.Context, req *GeminiChatRequest, model string) (*GeminiChatResponse, error)

GeminiChat sends a native Gemini generateContent request.

func (*Client) GeminiChatStream 

func (c *Client) GeminiChatStream(ctx context.Context, req *GeminiChatRequest, model string) (*ChatStreamResult, error)

GeminiChatStream sends a native Gemini streamGenerateContent request. It returns a channel of chunks. The channel closes when the stream ends.

func (*Client) Image 

func (c *Client) Image(ctx context.Context, req *ImageRequest) (*ImageResponse, error)

Image sends an image generation request.

func (*Client) ImageEdit 

func (c *Client) ImageEdit(ctx context.Context, req *ImageEditRequest) (*ImageResponse, error)

ImageEdit sends an image edit request (/v1/images/edits). The response body is returned as raw bytes.

func (*Client) Meter 

func (c *Client) Meter() meter.Meter

Meter returns the current meter.

func (*Client) MidjourneyFetch 

func (c *Client) MidjourneyFetch(ctx context.Context, baseURL, apiKey, taskID string) (*MidjourneyTask, error)

MidjourneyFetch fetches a Midjourney task by ID.

func (*Client) MidjourneyFetchByCondition 

func (c *Client) MidjourneyFetchByCondition(ctx context.Context, baseURL, apiKey string, condition map[string]any) ([]MidjourneyTask, error)

MidjourneyFetchByCondition fetches Midjourney tasks by condition (e.g. user_id, status).

func (*Client) MidjourneyImageSeed 

func (c *Client) MidjourneyImageSeed(ctx context.Context, baseURL, apiKey, taskID string) (json.RawMessage, error)

MidjourneyImageSeed fetches the image seed for a completed Midjourney task.

func (*Client) MidjourneyNotify 

func (c *Client) MidjourneyNotify(ctx context.Context, body io.Reader) (*MidjourneyNotifyRequest, error)

MidjourneyNotify handles a Midjourney webhook callback. It parses the notification and returns it for the caller to persist. Unlike new-api-main which updates a database, this library-level method simply returns the parsed notification.

func (*Client) MidjourneySubmit 

func (c *Client) MidjourneySubmit(ctx context.Context, baseURL, apiKey string, mode int, req *MidjourneyRequest) (*MidjourneyResponse, error)

MidjourneySubmit submits a Midjourney task. The mode should be one of the relaymode.RelayModeMidjourney* constants.

func (*Client) Moderations 

func (c *Client) Moderations(ctx context.Context, req *ModerationsRequest) (*dto.ModerationResponse, error)

Moderations sends a content moderation request.

func (*Client) Provider 

func (c *Client) Provider() Provider

Provider returns the current provider.

func (*Client) Rerank 

func (c *Client) Rerank(ctx context.Context, req *RerankRequest) (*RerankResponse, error)

Rerank sends a rerank request to the provider.

func (*Client) Responses 

func (c *Client) Responses(ctx context.Context, req *ResponsesRequest) (*ResponsesResponse, error)

Responses sends an OpenAI Responses API request.

func (*Client) ResponsesCompact 

func (c *Client) ResponsesCompact(ctx context.Context, req *ResponsesRequest) (*ResponsesResponse, error)

ResponsesCompact sends a compact OpenAI Responses API request (/v1/responses/compact). Only a subset of fields (model, input, instructions, previous_response_id, prompt_cache_*) are forwarded.

func (*Client) RichChat 

func (c *Client) RichChat(ctx context.Context, req *RichChatRequest) (*RichResponse, error)

RichChat sends a non-streaming rich chat request and returns the assembled response.

func (*Client) RichChatStream 

func (c *Client) RichChatStream(ctx context.Context, req *RichChatRequest) (*RichChatResult, error)

RichChatStream sends a streaming rich chat request. It returns a channel of rich stream chunks. The channel closes when the stream ends.

For the Claude/Anthropic channel, it calls /v1/messages directly with native content blocks. For OpenAI-compatible channels, it translates to the flat Chat Completions format and back.

func (*Client) SetMeter 

func (c *Client) SetMeter(m meter.Meter)

SetMeter replaces the meter at runtime.

func (*Client) SetProvider 

func (c *Client) SetProvider(p Provider)

SetProvider replaces the provider at runtime.

func (*Client) SubmitSunoTask 

func (c *Client) SubmitSunoTask(ctx context.Context, baseURL, apiKey, action string, req *SunoSubmitRequest) (*SunoSubmitResponse, error)

SubmitSunoTask submits a Suno music or lyrics generation task. action must be SunoActionMusic or SunoActionLyrics.

func (*Client) SubmitTask 

func (c *Client) SubmitTask(ctx context.Context, tp TaskProvider, body io.Reader) (*TaskSubmitResult, error)

SubmitTask submits an async task (video/music generation).

type CompletionsRequest 

type CompletionsRequest struct {
	Model       string          `json:"model"`
	Prompt      json.RawMessage `json:"prompt"` // string or []string
	MaxTokens   *int            `json:"max_tokens,omitempty"`
	Temperature *float64        `json:"temperature,omitempty"`
	TopP        *float64        `json:"top_p,omitempty"`
	Stream      bool            `json:"stream,omitempty"`
	Stop        json.RawMessage `json:"stop,omitempty"`
	N           *int            `json:"n,omitempty"`
	User        string          `json:"user,omitempty"`
}

CompletionsRequest is the legacy OpenAI /v1/completions request.

type CompletionsResponse 

type CompletionsResponse struct {
	ID       string            `json:"id"`
	Model    string            `json:"model"`
	Choices  []json.RawMessage `json:"choices"`
	Usage    meter.Usage       `json:"usage"`
	Provider string            `json:"provider"`
}

CompletionsResponse is the legacy completions response.

type ConfiguredProvider 

type ConfiguredProvider interface {
	BaseURL() string
	APIKey() string
}

ConfiguredProvider is an optional interface implemented by providers that expose their API endpoint and key. The Client uses this to populate RelayInfo.ChannelBaseUrl and RelayInfo.ApiKey, which most adaptors rely on for URL construction and authentication.

type ContentBlock 

type ContentBlock struct {
	Type string `json:"type"`

	// text block
	Text string `json:"text,omitempty"`

	// tool_use block (assistant)
	ID    string          `json:"id,omitempty"`
	Name  string          `json:"name,omitempty"`
	Input json.RawMessage `json:"input,omitempty"`

	// tool_result block (user, replying to a tool_use)
	ToolUseID string          `json:"tool_use_id,omitempty"`
	Content   json.RawMessage `json:"content,omitempty"` // string or array of blocks
	IsError   bool            `json:"is_error,omitempty"`

	// thinking block (extended thinking)
	Thinking  string `json:"thinking,omitempty"`
	Signature string `json:"signature,omitempty"`

	// image block
	Source *ImageSource `json:"source,omitempty"`

	// CacheControl places a prompt-cache breakpoint on this block.
	// "ephemeral" is the only supported value today. Empty = no breakpoint.
	CacheControl json.RawMessage `json:"cache_control,omitempty"`
}

ContentBlock is a discriminated union of content block types. Exactly one of the pointer fields is non-nil (except Image which carries its data inline).

func NewImageBlock 

func NewImageBlock(mediaType, base64Data string) ContentBlock

NewImageBlock creates an image content block from base64 data.

func NewTextBlock 

func NewTextBlock(text string) ContentBlock

NewTextBlock creates a text content block.

func NewThinkingBlock 

func NewThinkingBlock(thinking, signature string) ContentBlock

NewThinkingBlock creates a thinking content block.

func NewToolResultBlock 

func NewToolResultBlock(toolUseID string, content string, isError bool) ContentBlock

NewToolResultBlock creates a tool_result content block. content can be a plain string (will be JSON-encoded) or a JSON array of blocks.

func NewToolResultBlockRaw 

func NewToolResultBlockRaw(toolUseID string, content json.RawMessage, isError bool) ContentBlock

NewToolResultBlockRaw creates a tool_result with raw JSON content.

func NewToolUseBlock 

func NewToolUseBlock(id, name string, input json.RawMessage) ContentBlock

NewToolUseBlock creates a tool_use content block.

func (ContentBlock) GetText 

func (b ContentBlock) GetText() string

GetText returns the text content (empty for non-text blocks).

func (ContentBlock) GetToolResultText 

func (b ContentBlock) GetToolResultText() string

GetToolResultText extracts the text from a tool_result's Content field. If Content is a JSON string, it unquotes it. If it's an array of blocks, it concatenates the text blocks. Returns "" for non-tool_result blocks.

func (ContentBlock) IsImage 

func (b ContentBlock) IsImage() bool

IsImage returns true if this is an image block.

func (ContentBlock) IsText 

func (b ContentBlock) IsText() bool

IsText returns true if this is a text block.

func (ContentBlock) IsThinking 

func (b ContentBlock) IsThinking() bool

IsThinking returns true if this is a thinking block.

func (ContentBlock) IsToolResult 

func (b ContentBlock) IsToolResult() bool

IsToolResult returns true if this is a tool_result block.

func (ContentBlock) IsToolUse 

func (b ContentBlock) IsToolUse() bool

IsToolUse returns true if this is a tool_use block.

func (ContentBlock) WithCacheControl 

func (b ContentBlock) WithCacheControl() ContentBlock

WithCacheControl attaches a prompt-cache breakpoint to this block.

type EditRequest 

type EditRequest struct {
	Model       string   `json:"model"`
	Input       string   `json:"input"`
	Instruction string   `json:"instruction"`
	N           *int     `json:"n,omitempty"`
	Temperature *float64 `json:"temperature,omitempty"`
	TopP        *float64 `json:"top_p,omitempty"`
}

EditRequest is the legacy /v1/edits request.

type EmbedData 

type EmbedData struct {
	Index     int       `json:"index"`
	Embedding []float32 `json:"embedding"`
}

EmbedData is one embedding vector.

type EmbedRequest 

type EmbedRequest struct {
	Model string          `json:"model"`
	Input json.RawMessage `json:"input"`
	User  string          `json:"user,omitempty"`
}

EmbedRequest is the unified embedding request.

type EmbedResponse 

type EmbedResponse struct {
	Model    string      `json:"model"`
	Data     []EmbedData `json:"data"`
	Usage    meter.Usage `json:"usage"`
	Provider string      `json:"provider"`
}

EmbedResponse is the unified embedding response.

type FallbackConfig 

type FallbackConfig struct {
	// FallbackModels is a list of models to try if the primary model fails.
	// The Client will retry the request with each fallback model in order.
	FallbackModels []string
	// RetryOnErrors, if non-nil, determines which errors trigger a fallback.
	// If nil, all errors trigger fallback.
	RetryOnErrors func(err error) bool
}

FallbackConfig configures model fallback behavior. When configured, the Client retries a failed request with each fallback model in order. Fallback is opt-in: it is only active if WithFallback is used.

type GeminiChatRequest 

type GeminiChatRequest = dto.GeminiChatRequest

GeminiChatRequest is the native Gemini generateContent request.

type GeminiChatResponse 

type GeminiChatResponse = dto.GeminiChatResponse

GeminiChatResponse is the native Gemini generateContent response.

type GenericProvider 

type GenericProvider struct {
	// contains filtered or unexported fields
}

GenericProvider wraps any common.Adaptor with a name, API type, base URL, and API key. It is the standard way to use non-OpenAI/Claude/ Gemini providers with the Client.

func NewProvider 

func NewProvider(name string, apiType int, adaptor common.Adaptor, baseURL string, apiKey string) *GenericProvider

NewProvider creates a GenericProvider from the given adaptor and config. This is the primary constructor for providers that don't have their own Provider type (i.e. all providers except openai, claude, and gemini).

func (*GenericProvider) APIKey 

func (p *GenericProvider) APIKey() string

func (*GenericProvider) Adaptor 

func (p *GenericProvider) Adaptor() common.Adaptor

func (*GenericProvider) ApiType 

func (p *GenericProvider) ApiType() int

func (*GenericProvider) BaseURL 

func (p *GenericProvider) BaseURL() string

func (*GenericProvider) Name 

func (p *GenericProvider) Name() string

type ImageData 

type ImageData struct {
	URL           string `json:"url,omitempty"`
	B64JSON       string `json:"b64_json,omitempty"`
	RevisedPrompt string `json:"revised_prompt,omitempty"`
}

ImageData is one generated image.

type ImageEditRequest 

type ImageEditRequest struct {
	Model  string
	Prompt string
	Image  []byte // original image bytes
	Mask   []byte // optional mask bytes
	N      *int
	Size   string
	User   string
}

ImageEditRequest is the /v1/images/edits request (multipart form).

type ImageRequest 

type ImageRequest struct {
	Model          string `json:"model"`
	Prompt         string `json:"prompt"`
	N              *int   `json:"n,omitempty"`
	Size           string `json:"size,omitempty"`
	Quality        string `json:"quality,omitempty"`
	ResponseFormat string `json:"response_format,omitempty"`
	Style          string `json:"style,omitempty"`
	User           string `json:"user,omitempty"`
}

ImageRequest is the unified image generation request.

type ImageResponse 

type ImageResponse struct {
	Created  int64       `json:"created"`
	Data     []ImageData `json:"data"`
	Usage    meter.Usage `json:"usage"`
	Provider string      `json:"provider"`
}

ImageResponse is the unified image generation response.

type ImageSource 

type ImageSource struct {
	Type      string `json:"type"`       // "base64"
	MediaType string `json:"media_type"` // "image/png", "image/jpeg", ...
	Data      string `json:"data"`       // base64-encoded
}

ImageSource is the source of an image content block.

type Message 

type Message = dto.Message

Message is a chat message. Content is any to support both plain string content and multimodal content arrays.

type MidjourneyNotifyRequest 

type MidjourneyNotifyRequest struct {
	MjId     string `json:"mjId"`
	Status   string `json:"status"`
	Progress string `json:"progress"`
	PromptEn string `json:"promptEn"`
	ImageUrl string `json:"imageUrl"`
	State    string `json:"state"`
}

MidjourneyNotifyRequest is the body of a Midjourney webhook callback.

type MidjourneyRequest 

type MidjourneyRequest struct {
	Prompt      string   `json:"prompt,omitempty"`
	CustomId    string   `json:"customId,omitempty"`
	BotType     string   `json:"bot_type,omitempty"`
	NotifyHook  string   `json:"notifyHook,omitempty"`
	Action      string   `json:"action,omitempty"`
	Index       int      `json:"index,omitempty"`
	State       string   `json:"state,omitempty"`
	TaskId      string   `json:"taskId,omitempty"`
	Base64Array []string `json:"base64Array,omitempty"`
	Content     string   `json:"content,omitempty"`
	MaskBase64  string   `json:"maskBase64,omitempty"`
}

MidjourneyRequest represents a Midjourney task submission request.

type MidjourneyResponse 

type MidjourneyResponse struct {
	Code        int            `json:"code"`
	Description string         `json:"description,omitempty"`
	Properties  map[string]any `json:"properties,omitempty"`
	Result      string         `json:"result,omitempty"`
}

MidjourneyResponse represents the Midjourney API response.

type MidjourneyTask 

type MidjourneyTask struct {
	ID         string           `json:"id"`
	Action     string           `json:"action,omitempty"`
	Status     string           `json:"status,omitempty"`
	Progress   string           `json:"progress,omitempty"`
	Prompt     string           `json:"prompt,omitempty"`
	PromptEn   string           `json:"promptEn,omitempty"`
	ImageUrl   string           `json:"imageUrl,omitempty"`
	VideoUrl   string           `json:"videoUrl,omitempty"`
	FailReason string           `json:"failReason,omitempty"`
	Buttons    []map[string]any `json:"buttons,omitempty"`
	Properties map[string]any   `json:"properties,omitempty"`
	SubmitTime int64            `json:"submitTime,omitempty"`
	StartTime  int64            `json:"startTime,omitempty"`
	FinishTime int64            `json:"finishTime,omitempty"`
}

MidjourneyTask represents a fetched Midjourney task state.

type ModerationsRequest 

type ModerationsRequest struct {
	Model string `json:"model,omitempty"`
	Input any    `json:"input"` // string or []string
}

ModerationsRequest is the /v1/moderations request.

type Option 

type Option func(*Client)

Option configures a Client.

func WithCircuitBreaker 

func WithCircuitBreaker(cb *circuitbreaker.CircuitBreaker) Option

WithCircuitBreaker configures a circuit breaker for HTTP requests. When set, each (retried) attempt is wrapped in the breaker's Execute. By default no circuit breaker is used.

func WithFallback 

func WithFallback(cfg FallbackConfig) Option

WithFallback configures model fallback.

func WithHTTPClient 

func WithHTTPClient(hc *http.Client) Option

WithHTTPClient sets a custom HTTP client.

func WithMaxIdleConns 

func WithMaxIdleConns(n int) Option

WithMaxIdleConns sets the max idle connections per host on the default transport. If a custom transport is in use that is not an *http.Transport, this option is a no-op.

func WithMeter 

func WithMeter(m meter.Meter) Option

WithMeter sets the usage meter.

func WithProvider 

func WithProvider(p Provider) Option

WithProvider sets the provider.

func WithRequestHook 

func WithRequestHook(h RequestHook) Option

WithRequestHook sets a hook called before each request. The returned function is called after the request completes with the resulting error. This enables tracing/metrics without the relay package depending on a specific observability SDK.

func WithRetry 

func WithRetry(opts ...retry.Option) Option

WithRetry configures retry behaviour for HTTP requests. When set, the Client retries failed requests (network errors, 429, and 5xx responses) according to the supplied retry options. By default no retry is performed.

func WithTimeout 

func WithTimeout(d time.Duration) Option

WithTimeout sets the HTTP client timeout.

type Provider 

type Provider interface {
	Name() string
	ApiType() int
	Adaptor() common.Adaptor
}

Provider is the high-level interface for AI API providers. It wraps the low-level common.Adaptor with a simpler, type-safe API.

type RelayError 

type RelayError struct {
	Provider   string
	Model      string
	StatusCode int
	Code       string // error code from provider
	Message    string
	Err        error // wrapped error
}

RelayError is a unified error type for relay operations. It captures the provider and model involved, the HTTP status code, the provider-specific error code (when available), and a human-readable message. The underlying error (if any) is wrapped and accessible via Unwrap.

func (*RelayError) Error 

func (e *RelayError) Error() string

Error implements the error interface.

func (*RelayError) IsRetryable 

func (e *RelayError) IsRetryable() bool

IsRetryable returns true if the error is likely transient (429, 500, 502, 503, 504). Such errors are good candidates for retry or model fallback.

func (*RelayError) Unwrap 

func (e *RelayError) Unwrap() error

Unwrap returns the wrapped error, if any.

type RequestHook 

type RequestHook func(ctx context.Context, provider, model string, mode int) func(err error)

RequestHook is called before each request. It returns a function that is called after the request completes (with the resulting error, which may be nil on success). This lets consumers add tracing/metrics/logging without the relay package depending on a specific observability SDK. 

mode is one of the relaymode.RelayMode* constants.

type RerankRequest 

type RerankRequest struct {
	Model     string
	Query     string
	Documents []string
	TopN      *int
}

RerankRequest is the unified rerank request.

type RerankResponse 

type RerankResponse struct {
	Results []dto.RerankResponseResult
	Usage   meter.Usage
}

RerankResponse holds the rerank response.

type ResponsesRequest 

type ResponsesRequest struct {
	Model   string
	Input   json.RawMessage
	Stream  bool
	Tools   json.RawMessage
	Options json.RawMessage
}

ResponsesRequest is the unified OpenAI Responses API request.

type ResponsesResponse 

type ResponsesResponse struct {
	Data  []byte
	Usage meter.Usage
}

ResponsesResponse holds the Responses API response.

type RichChatRequest 

type RichChatRequest struct {
	Model           string          `json:"model"`
	Messages        []RichMessage   `json:"messages"`
	System          string          `json:"system,omitempty"`
	Temperature     *float64        `json:"temperature,omitempty"`
	MaxTokens       int             `json:"max_tokens,omitempty"`
	Tools           []RichTool      `json:"tools,omitempty"`
	ToolChoice      json.RawMessage `json:"tool_choice,omitempty"`
	Betas           []string        `json:"betas,omitempty"`
	ReasoningEffort string          `json:"reasoning_effort,omitempty"`
	Stream          bool            `json:"stream,omitempty"`
}

RichChatRequest is the rich content-block version of ChatRequest. It uses RichMessage (with ContentBlock[]) instead of the flat OpenAI Message format, so the agent loop can work with structured content (text, tool_use, tool_result, thinking, images) without depending on any specific SDK.

type RichChatResult 

type RichChatResult struct {
	Ch    chan RichStreamChunk
	Final RichResponse
}

RichChatResult holds the stream channel and final usage for a rich stream.

type RichMessage 

type RichMessage struct {
	Role    string         `json:"role"`
	Content []ContentBlock `json:"content"`
}

RichMessage is a conversation message with structured content blocks. It is the provider-neutral equivalent of anthropic.BetaMessageParam. Role is "user", "assistant", or "system".

func NewAssistantMessageBlocks 

func NewAssistantMessageBlocks(blocks ...ContentBlock) RichMessage

NewAssistantMessage creates an assistant message from content blocks.

func NewSystemMessage 

func NewSystemMessage(text string) RichMessage

NewSystemMessage creates a system message (rarely used; system is usually a top-level field on the request).

func NewUserMessage 

func NewUserMessage(text string) RichMessage

NewUserMessage creates a user message with text content.

func NewUserMessageBlocks 

func NewUserMessageBlocks(blocks ...ContentBlock) RichMessage

NewUserMessageBlocks creates a user message from content blocks.

type RichResponse 

type RichResponse struct {
	ID           string         `json:"id"`
	Role         string         `json:"role"` // always "assistant"
	Content      []ContentBlock `json:"content"`
	StopReason   string         `json:"stop_reason"` // "end_turn" | "tool_use" | "max_tokens" | ...
	Model        string         `json:"model"`
	InputTokens  int64          `json:"input_tokens"`
	OutputTokens int64          `json:"output_tokens"`
	// Prompt-cache usage (Anthropic-specific; zero for other providers).
	CacheReadInputTokens     int64 `json:"cache_read_input_tokens,omitempty"`
	CacheCreationInputTokens int64 `json:"cache_creation_input_tokens,omitempty"`
}

RichResponse is the assembled response from a model turn, in content-block form. It is the provider-neutral equivalent of anthropic.BetaMessage.

func (RichResponse) Text 

func (r RichResponse) Text() string

Text concatenates all text blocks.

func (RichResponse) ToolUses 

func (r RichResponse) ToolUses() []ContentBlock

ToolUses returns all tool_use blocks in order.

type RichStreamChunk 

type RichStreamChunk struct {
	// Type identifies what kind of update this is:
	// "text_delta" — Text carries the incremental text
	// "tool_use_delta" — ToolUseIndex + ToolUseID/Name/InputFragment
	// "thinking_delta" — Thinking carries the incremental thinking text
	// "finish" — FinishReason is set
	// "usage" — InputTokens/OutputTokens are set (final usage)
	// "error" — Err is set
	Type string `json:"type"`

	// text delta
	Text string `json:"text,omitempty"`

	// thinking delta
	Thinking string `json:"thinking,omitempty"`

	// tool_use delta (accumulated by index)
	ToolUseIndex int    `json:"tool_use_index,omitempty"`
	ToolUseID    string `json:"tool_use_id,omitempty"`
	ToolUseName  string `json:"tool_use_name,omitempty"`
	// InputFragment is a piece of the tool input JSON (accumulated by index).
	InputFragment string `json:"input_fragment,omitempty"`

	// finish
	FinishReason string `json:"finish_reason,omitempty"`

	// usage (final chunk)
	InputTokens  int64 `json:"input_tokens,omitempty"`
	OutputTokens int64 `json:"output_tokens,omitempty"`

	// cache usage
	CacheReadInputTokens     int64 `json:"cache_read_input_tokens,omitempty"`
	CacheCreationInputTokens int64 `json:"cache_creation_input_tokens,omitempty"`

	Err  error `json:"-"`
	Done bool  `json:"-"`
}

RichStreamChunk is one chunk in a streaming response, in content-block form. It carries incremental updates as the model generates.

type RichTool 

type RichTool struct {
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	InputSchema json.RawMessage `json:"input_schema,omitempty"`
	// Type is "function" for custom tools. Some providers have built-in
	// tools (e.g. Anthropic web_search) with different types.
	Type string `json:"type,omitempty"`
}

RichTool is a tool definition in the rich format.

type SunoSubmitRequest 

type SunoSubmitRequest struct {
	GptDescriptionPrompt string  `json:"gpt_description_prompt,omitempty"`
	Prompt               string  `json:"prompt,omitempty"`
	Mv                   string  `json:"mv,omitempty"`
	Title                string  `json:"title,omitempty"`
	Tags                 string  `json:"tags,omitempty"`
	ContinueAt           float64 `json:"continue_at,omitempty"`
	TaskID               string  `json:"task_id,omitempty"`
	ContinueClipId       string  `json:"continue_clip_id,omitempty"`
	MakeInstrumental     bool    `json:"make_instrumental"`
}

SunoSubmitRequest is the request body for Suno music/lyrics generation.

type SunoSubmitResponse 

type SunoSubmitResponse struct {
	Code    int    `json:"code"`
	Message string `json:"message,omitempty"`
	Data    string `json:"data,omitempty"`
}

SunoSubmitResponse is the response from a Suno submit request.

type SunoTaskData 

type SunoTaskData struct {
	TaskID     string          `json:"task_id,omitempty"`
	Action     string          `json:"action,omitempty"`
	Status     string          `json:"status,omitempty"`
	FailReason string          `json:"fail_reason,omitempty"`
	SubmitTime int64           `json:"submit_time,omitempty"`
	StartTime  int64           `json:"start_time,omitempty"`
	FinishTime int64           `json:"finish_time,omitempty"`
	Data       json.RawMessage `json:"data,omitempty"`
}

SunoTaskData represents one Suno task entry returned by the fetch endpoint.

type TaskProvider 

type TaskProvider interface {
	Name() string
	ApiType() int
	TaskAdaptor() common.TaskAdaptor
}

TaskProvider is the interface for async task providers.

type TaskSubmitResult 

type TaskSubmitResult struct {
	TaskID   string
	TaskData []byte
}

TaskSubmitResult holds the result of submitting an async task.

type Tool 

type Tool = dto.ToolCallRequest

Tool represents a tool the model may call (alias to ToolCallRequest).

type ToolCall 

type ToolCall = dto.ToolCallRequest

ToolCall represents a tool/function call request.

Source Files

View all Source files

Directories

Path Synopsis
Package channel provides shared utilities for provider adaptors.
 Package channel provides shared utilities for provider adaptors.
advancedcustom
ai360
ali
aws
baidu
baidu_v2
claude
Package claude provides an Anthropic Claude AI provider adaptor.
 Package claude provides an Anthropic Claude AI provider adaptor.
cloudflare
codex
cohere
coze
deepseek
dify
gemini
Package gemini provides a Google Gemini AI provider adaptor.
 Package gemini provides a Google Gemini AI provider adaptor.
huggingface
Package huggingface provides a HuggingFace Inference Router adaptor.
 Package huggingface provides a HuggingFace Inference Router adaptor.
jimeng
jina
lingyiwanwu
minimax
mistral
mokaai
moonshot
newapi
ollama
openai
Package openai provides an OpenAI-compatible AI provider adaptor.
 Package openai provides an OpenAI-compatible AI provider adaptor.
openrouter
palm
perplexity
replicate
siliconflow
sub2api
submodel
tencent
vertex
volcengine
xai
xinference
xunfei
zhipu
zhipu_4v
Package relay provides the core relay types: a stripped-down RelayInfo (no user/token/billing/DB fields) and a clean Adaptor interface that uses context.Context instead of gin.Context.
 Package relay provides the core relay types: a stripped-down RelayInfo (no user/token/billing/DB fields) and a clean Adaptor interface that uses context.Context instead of gin.Context.
Package common_handler provides shared response handlers for relay providers.
 Package common_handler provides shared response handlers for relay providers.
example
basic command
Example: basic chat completion with OpenAI provider and usage metering.
 Example: basic chat completion with OpenAI provider and usage metering.
gemini command
Example: Gemini native format chat completion.
 Example: Gemini native format chat completion.
midjourney command
Example: Midjourney task submission and polling.
 Example: Midjourney task submission and polling.
realtime command
Example: OpenAI Realtime API WebSocket session.
 Example: OpenAI Realtime API WebSocket session.
suno command
Example: Suno music generation task submission and polling.
 Example: Suno music generation task submission and polling.
Package helper provides utility functions for relay providers, adapted from LingRein's pkg/relay/helper with gin.Context dependencies removed.
 Package helper provides utility functions for relay providers, adapted from LingRein's pkg/relay/helper with gin.Context dependencies removed.
Package meter provides usage metering for AI API calls.
 Package meter provides usage metering for AI API calls.
Package realtime provides a framework-independent WebSocket abstraction for OpenAI-compatible Realtime API sessions.
 Package realtime provides a framework-independent WebSocket abstraction for OpenAI-compatible Realtime API sessions.
relaykit module
Package service provides shared utility functions for relay providers, adapted from LingRein's internal/service package with gin.Context dependencies removed.
 Package service provides shared utility functions for relay providers, adapted from LingRein's internal/service package with gin.Context dependencies removed.
Package setting provides minimal configuration stubs for relay providers.
 Package setting provides minimal configuration stubs for relay providers.
task
ali
doubao
gemini
hailuo
jimeng
kling
sora
suno
taskcommon
Package taskcommon provides shared utilities for task providers.
 Package taskcommon provides shared utilities for task providers.
taskmodel
Package taskmodel provides a minimal Task struct and status constants for async task providers.
 Package taskmodel provides a minimal Task struct and status constants for async task providers.
vertex
vidu

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.