router

Documentation

Index

• type R
  • func New(c config.InferenceManagerEngineConfig, k8sClient *k8s.Client) *R
  • func (r *R) GetEngineForModel(ctx context.Context, modelID string) (string, error)
  • func (r *R) Run(ctx context.Context, errCh chan error) error

Types

type R

type R struct {
	// contains filtered or unexported fields
}

R manages the routing table of models to engines.

func New

func New(c config.InferenceManagerEngineConfig, k8sClient *k8s.Client) *R

New creates a new router.

func (*R) GetEngineForModel

func (r *R) GetEngineForModel(ctx context.Context, modelID string) (string, error)

GetEngineForModel returns the engine address (ip and port) for the given model.

func (*R) Run

func (r *R) Run(ctx context.Context, errCh chan error) error

Run updates the routes periodically and upon pod events.