configprofiles

Documentation

Overview

Package configprofiles parses the model.aibrix.ai/config annotation (or ConfigMap) and supports multiple named profiles selectable at runtime via config-profile header. See docs/source/designs/model-config-profiles.rst for the design.

Index

• Constants
• type ModelConfigProfile
  • func ResolveProfile(pods []*v1.Pod, headerProfile string) *ModelConfigProfile
  • func ResolveProfileFromPod(pod *v1.Pod, headerProfile string) *ModelConfigProfile
• type ModelConfigProfiles
  • func ParseModelConfig(jsonStr string) (*ModelConfigProfiles, error)
  • func (c *ModelConfigProfiles) GetProfile(name string) *ModelConfigProfile

Constants

const (
	// DefaultProfileName is used when defaultProfile is not set in the JSON.
	DefaultProfileName = "default"
)

Types

type ModelConfigProfile

type ModelConfigProfile struct {
	RoutingStrategy   string          `json:"routingStrategy"`
	RoutingConfig     json.RawMessage `json:"routingConfig,omitempty"`
	RequestsPerSecond int64           `json:"requestsPerSecond,omitempty"`
}

ModelConfigProfile holds gateway options for a single profile.

func ResolveProfile

func ResolveProfile(pods []*v1.Pod, headerProfile string) *ModelConfigProfile

ResolveProfile resolves the model config from pods (annotation), then returns the profile selected by headerProfile (from config-profile). configMapGetter can be nil; it is checked first when provided.

func ResolveProfileFromPod

func ResolveProfileFromPod(pod *v1.Pod, headerProfile string) *ModelConfigProfile

ResolveProfileFromPod resolves the model config from a single pod annotation and returns the selected profile.

type ModelConfigProfiles

type ModelConfigProfiles struct {
	DefaultProfile string                        `json:"defaultProfile"`
	Profiles       map[string]ModelConfigProfile `json:"profiles"`
}

ModelConfigProfiles is the root JSON structure from model.aibrix.ai/config.

func ParseModelConfig

func ParseModelConfig(jsonStr string) (*ModelConfigProfiles, error)

ParseModelConfig parses the JSON from annotation data. Returns nil if jsonStr is empty or invalid.

func (*ModelConfigProfiles) GetProfile

func (c *ModelConfigProfiles) GetProfile(name string) *ModelConfigProfile

GetProfile returns the profile for the given name, or the default profile. Falls back to defaultProfile/"default" when the requested profile does not exist. Returns nil only if no default profile exists.