vllm

Documentation

Index

• type Manager
  • func New(c *config.Config) *Manager
  • func (m *Manager) CreateNewModel(modelName string, spec *manager.ModelSpec) error
  • func (m *Manager) DeleteModel(ctx context.Context, modelName string) error
  • func (m *Manager) Run() error
  • func (m *Manager) WaitForReady() error

Types

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager manages the Ollama service.

func New

func New(c *config.Config) *Manager

New returns a new Manager.

func (*Manager) CreateNewModel

func (m *Manager) CreateNewModel(modelName string, spec *manager.ModelSpec) error

CreateNewModel creates a new model with the given name and spec.

func (*Manager) DeleteModel

func (m *Manager) DeleteModel(ctx context.Context, modelName string) error

DeleteModel deletes the model.

func (*Manager) Run

func (m *Manager) Run() error

Run starts the vLLM service.

func (*Manager) WaitForReady

func (m *Manager) WaitForReady() error

WaitForReady waits for the vllm service to be ready.