ollama

Documentation

Index

• func CreateModelfile(filePath string, modelName string, spec *ModelSpec, ...) error
• func ModelfilePath(modelDir string, modelName string) string
• func SetEnvVarsFromConfig(c config.OllamaConfig) error
• type Manager
  • func New(contextLengthsByModelID map[string]int, s3Client s3Client) *Manager
  • func (m *Manager) CreateNewModelOfGGUF(modelName string, spec *ModelSpec) error
  • func (m *Manager) DownloadAndCreateNewModel(ctx context.Context, modelName string, resp *mv1.GetBaseModelPathResponse) error
  • func (m *Manager) IsReady() (bool, string)
  • func (m *Manager) PullBaseModel(modelName string) error
  • func (m *Manager) Run() error
  • func (m *Manager) UpdateModelTemplateToLatest(modelName string) error
  • func (m *Manager) WaitForReady() error
• type ModelSpec

Functions

func CreateModelfile

func CreateModelfile(
	filePath string,
	modelName string,
	spec *ModelSpec,
	contextLengthsByModelID map[string]int,
) error

CreateModelfile creates a new model file.

func ModelfilePath

func ModelfilePath(modelDir string, modelName string) string

ModelfilePath returns the model file path.

func SetEnvVarsFromConfig

func SetEnvVarsFromConfig(c config.OllamaConfig) error

SetEnvVarsFromConfig sets environment variables from the given configuration.

Types

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager manages the Ollama service.

TODO(kenji): Refactor this class once we completely switch to the one-odel-per-pod implementation where inference-manager-engine doesn't directly run vLLM or Ollama.

func New

func New(contextLengthsByModelID map[string]int, s3Client s3Client) *Manager

New returns a new Manager.

func (*Manager) CreateNewModelOfGGUF

func (m *Manager) CreateNewModelOfGGUF(modelName string, spec *ModelSpec) error

CreateNewModelOfGGUF creates a new model with the given name and spec that uses a GGUF model file.

func (*Manager) DownloadAndCreateNewModel

func (m *Manager) DownloadAndCreateNewModel(ctx context.Context, modelName string, resp *mv1.GetBaseModelPathResponse) error

DownloadAndCreateNewModel downloads the model from the given path and creates a new model.

func (*Manager) IsReady

func (m *Manager) IsReady() (bool, string)

IsReady returns true if the processor is ready. If not, it returns a message describing why it is not ready.

func (*Manager) PullBaseModel

func (m *Manager) PullBaseModel(modelName string) error

PullBaseModel pulls the base model from the given path.

func (*Manager) Run

func (m *Manager) Run() error

Run starts the Ollama service.

func (*Manager) UpdateModelTemplateToLatest

func (m *Manager) UpdateModelTemplateToLatest(modelName string) error

UpdateModelTemplateToLatest updates the model template to the latest.

func (*Manager) WaitForReady

func (m *Manager) WaitForReady() error

WaitForReady waits for the Ollama service to be ready.

type ModelSpec

type ModelSpec struct {
	From        string
	AdapterPath string
}

ModelSpec is the specification for a new model.