vllm

Documentation

Index

• type Manager
  • func New(c *config.Config) *Manager
  • func (m *Manager) CreateNewModel(modelName string, spec *manager.ModelSpec) error
  • func (m *Manager) IsReady() (bool, string)
  • func (m *Manager) Run() error
  • func (m *Manager) UpdateModelTemplateToLatest(modelName string) error
  • func (m *Manager) WaitForReady() error

Types

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager manages the Ollama service.

func New

func New(c *config.Config) *Manager

New returns a new Manager.

func (*Manager) CreateNewModel

func (m *Manager) CreateNewModel(modelName string, spec *manager.ModelSpec) error

CreateNewModel creates a new model with the given name and spec.

func (*Manager) IsReady

func (m *Manager) IsReady() (bool, string)

IsReady returns true if the processor is ready. If not, it returns a message describing why it is not ready.

func (*Manager) Run

func (m *Manager) Run() error

Run starts the vLLM service.

func (*Manager) UpdateModelTemplateToLatest

func (m *Manager) UpdateModelTemplateToLatest(modelName string) error

UpdateModelTemplateToLatest updates the model template to the latest.

func (*Manager) WaitForReady

func (m *Manager) WaitForReady() error

WaitForReady waits for the vllm service to be ready.