ollama

Documentation

Index

• func DeleteOrphanedRunnersDir(c config.OllamaConfig) error
• func SetEnvVarsFromConfig(c config.OllamaConfig) error
• type Manager
  • func New(contextLengthsByModelID map[string]int, s3Client s3Client) *Manager
  • func (m *Manager) CreateNewModelOfGGUF(modelName string, spec *manager.ModelSpec) error
  • func (m *Manager) DownloadAndCreateNewModel(modelName string, resp *mv1.GetBaseModelPathResponse) error
  • func (m *Manager) IsReady() (bool, string)
  • func (m *Manager) PullBaseModel(modelName string) error
  • func (m *Manager) Run() error
  • func (m *Manager) UpdateModelTemplateToLatest(modelName string) error
  • func (m *Manager) WaitForReady() error

Functions

func DeleteOrphanedRunnersDir

func DeleteOrphanedRunnersDir(c config.OllamaConfig) error

DeleteOrphanedRunnersDir deletes orphaned payload directories.

func SetEnvVarsFromConfig

func SetEnvVarsFromConfig(c config.OllamaConfig) error

SetEnvVarsFromConfig sets environment variables from the given configuration.

Types

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager manages the Ollama service.

TODO(kenji): Refactor this class once we completely switch to the one-odel-per-pod implementation where inference-manager-engine doesn't directly run vLLM or Ollama.

func New

func New(contextLengthsByModelID map[string]int, s3Client s3Client) *Manager

New returns a new Manager.

func (*Manager) CreateNewModelOfGGUF

func (m *Manager) CreateNewModelOfGGUF(modelName string, spec *manager.ModelSpec) error

CreateNewModelOfGGUF creates a new model with the given name and spec that uses a GGUF model file.

func (*Manager) DownloadAndCreateNewModel

func (m *Manager) DownloadAndCreateNewModel(modelName string, resp *mv1.GetBaseModelPathResponse) error

DownloadAndCreateNewModel downloads the model from the given path and creates a new model.

func (*Manager) IsReady

func (m *Manager) IsReady() (bool, string)

IsReady returns true if the processor is ready. If not, it returns a message describing why it is not ready.

func (*Manager) PullBaseModel

func (m *Manager) PullBaseModel(modelName string) error

PullBaseModel pulls the base model from the given path.

func (*Manager) Run

func (m *Manager) Run() error

Run starts the Ollama service.

func (*Manager) UpdateModelTemplateToLatest

func (m *Manager) UpdateModelTemplateToLatest(modelName string) error

UpdateModelTemplateToLatest updates the model template to the latest.

func (*Manager) WaitForReady

func (m *Manager) WaitForReady() error

WaitForReady waits for the Ollama service to be ready.