local

package
v0.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 25, 2023 License: MIT Imports: 10 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrEmptyResponse is returned when the local LLM binary returns an empty response.
	ErrEmptyResponse = errors.New("no response")
	// ErrMissingBin is returned when the LOCAL_LLM_BIN environment variable is not set.
	ErrMissingBin = errors.New("missing the local LLM binary path, set the LOCAL_LLM_BIN environment variable")
)

Functions

This section is empty.

Types

type LLM

type LLM struct {
	CallbacksHandler callbacks.Handler
	// contains filtered or unexported fields
}

LLM is a local LLM implementation.

func New

func New(opts ...Option) (*LLM, error)

New creates a new local LLM implementation.

func (*LLM) Call

func (o *LLM) Call(ctx context.Context, prompt string, options ...llms.CallOption) (string, error)

Call calls the local LLM binary with the given prompt.

func (*LLM) Generate

func (o *LLM) Generate(ctx context.Context, prompts []string, options ...llms.CallOption) ([]*llms.Generation, error)

Generate generates completions using the local LLM binary.

func (*LLM) GeneratePrompt

func (o *LLM) GeneratePrompt(
	ctx context.Context,
	prompts []schema.PromptValue,
	options ...llms.CallOption,
) (llms.LLMResult, error)

func (*LLM) GetNumTokens

func (o *LLM) GetNumTokens(text string) int

type Option

type Option func(*options)

func WithArgs

func WithArgs(args string) Option

WithArgs passes the CLI arguments to the local LLM binary. If not set, then will be used the LOCAL_LLM_ARGS environment variable.

func WithBin

func WithBin(bin string) Option

WithBin passes the path to the local LLM binary to the client. If not set, then will be used the LOCAL_LLM_BIN environment variable.

func WithGlobalAsArgs

func WithGlobalAsArgs() Option

WithGlobalAsArgs passes the CLI arguments to the local LLM binary formed from global llms.Options.

Directories

Path Synopsis
internal
localclient
Package localclient provides a client for local LLMs.
Package localclient provides a client for local LLMs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL