virtualmodels

Documentation

Overview

Package virtualmodels implements public virtual model pools such as ail-compound. It deliberately lives below internal/ so SDK clients only see the stable config shape, not selector internals.

Index

• Constants
• func FallbackEnabled(cfg *config.SDKConfig, model string) bool
• func IsVirtualModel(cfg *config.SDKConfig, model string) bool
• func PublicCatalog(cfg *config.SDKConfig) []map[string]any
• func RewriteModelField(payload []byte, publicModel string) []byte
• type NoEligibleBackendError
  • func (e NoEligibleBackendError) Code() string
  • func (e NoEligibleBackendError) Error() string
• type Requirements
  • func DetectRequirements(rawJSON []byte, endpoint string) Requirements
• type Route
• type Router
  • func NewRouter() *Router
  • func (r *Router) Select(cfg *config.SDKConfig, model string, rawJSON []byte, endpoint string) (Route, error)
  • func (r *Router) SelectExcluding(cfg *config.SDKConfig, model string, rawJSON []byte, endpoint string, ...) (Route, error)

Constants

const (
	ClassChatText               = "chat_text"
	ClassChatTextTools          = "chat_text_tools"
	ClassChatMultiturnTools     = "chat_multiturn_tools"
	ClassChatImageUnderstanding = "chat_image_understanding"
	ClassChatAudioUnderstanding = "chat_audio_understanding"
	ClassAudioTranscription     = "audio_transcription"
	ClassImageGeneration        = "image_generation"
	ClassSpeechGeneration       = "speech_generation"
	ClassMusicGeneration        = "music_generation"
	ClassLyricsGeneration       = "lyrics_generation"
	ClassEmbeddings             = "embeddings"
)

Functions

func FallbackEnabled

func FallbackEnabled(cfg *config.SDKConfig, model string) bool

FallbackEnabled reports whether a virtual pool should try another eligible member on recoverable upstream failures.

func IsVirtualModel

func IsVirtualModel(cfg *config.SDKConfig, model string) bool

IsVirtualModel reports whether model maps to a configured virtual pool.

func PublicCatalog

func PublicCatalog(cfg *config.SDKConfig) []map[string]any

PublicCatalog returns provider-neutral public virtual models for /v1/models.

func RewriteModelField

func RewriteModelField(payload []byte, publicModel string) []byte

RewriteModelField rewrites a JSON object model field when present. It also supports raw SSE payload text; data: [DONE] is preserved.

Types

type NoEligibleBackendError

type NoEligibleBackendError struct {
	Model        string
	Requirements Requirements
}

NoEligibleBackendError reports explicit capability mismatch without trying an upstream provider.

func (NoEligibleBackendError) Code

func (e NoEligibleBackendError) Code() string

func (NoEligibleBackendError) Error

func (e NoEligibleBackendError) Error() string

type Requirements

type Requirements struct {
	Class          string
	InputText      bool
	InputImage     bool
	InputAudio     bool
	NeedsTools     bool
	HasToolHistory bool
	OutputText     bool
	OutputImage    bool
	OutputAudio    bool
	MinContext     int
	Endpoint       string
}

Requirements captures the capability contract required by a request.

func DetectRequirements

func DetectRequirements(rawJSON []byte, endpoint string) Requirements

DetectRequirements classifies OpenAI-compatible requests by endpoint and body.

type Route

type Route struct {
	PublicModel  string
	Provider     string
	NativeModel  string
	MemberID     string
	Requirements Requirements
}

Route is the concrete provider/model choice for one request.

type Router

type Router struct {
	// contains filtered or unexported fields
}

Router keeps process-local weighted round-robin cursors.

func NewRouter

func NewRouter() *Router

func (*Router) Select

func (r *Router) Select(cfg *config.SDKConfig, model string, rawJSON []byte, endpoint string) (Route, error)

Select chooses one eligible member for a virtual model.

func (*Router) SelectExcluding

func (r *Router) SelectExcluding(cfg *config.SDKConfig, model string, rawJSON []byte, endpoint string, exclude map[string]struct{}) (Route, error)

SelectExcluding chooses an eligible member while skipping member IDs in exclude.