ollama

Documentation

Index

• func SetEnvVarsFromConfig(c config.OllamaConfig) error
• type Manager
  • func New(contextLengthsByModelID map[string]int) *Manager
  • func (m *Manager) CreateNewModel(modelName string, spec *manager.ModelSpec) error
  • func (m *Manager) IsReady() (bool, string)
  • func (m *Manager) PullBaseModel(modelName string) error
  • func (m *Manager) Run() error
  • func (m *Manager) UpdateModelTemplateToLatest(modelName string) error
  • func (m *Manager) WaitForReady() error

Functions

func SetEnvVarsFromConfig

func SetEnvVarsFromConfig(c config.OllamaConfig) error

SetEnvVarsFromConfig sets environment variables from the given configuration.

Types

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager manages the Ollama service.

TODO(kenji): Refactor this class once we completely switch to the one-odel-per-pod implementation where inference-manager-engine doesn't directly run vLLM or Ollama.

func New

func New(contextLengthsByModelID map[string]int) *Manager

New returns a new Manager.

func (*Manager) CreateNewModel

func (m *Manager) CreateNewModel(modelName string, spec *manager.ModelSpec) error

CreateNewModel creates a new model with the given name and spec.

func (*Manager) IsReady

func (m *Manager) IsReady() (bool, string)

IsReady returns true if the processor is ready. If not, it returns a message describing why it is not ready.

func (*Manager) PullBaseModel

func (m *Manager) PullBaseModel(modelName string) error

PullBaseModel pulls the base model from the given path.

func (*Manager) Run

func (m *Manager) Run() error

Run starts the Ollama service.

func (*Manager) UpdateModelTemplateToLatest

func (m *Manager) UpdateModelTemplateToLatest(modelName string) error

UpdateModelTemplateToLatest updates the model template to the latest.

func (*Manager) WaitForReady

func (m *Manager) WaitForReady() error

WaitForReady waits for the Ollama service to be ready.