openai

package module
v0.10.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2024 License: MIT Imports: 17 Imported by: 3

README

Pragmatic OpenAI SDK for Go focused on ChatGPT

Zero Dependencies • Tokenizer • Simple Code • Best way to use ChatGPT from Go

Install:

go get github.com/andreyvit/openai@latest

Run the example:

export OPENAI_API_KEY=...
go run github.com/andreyvit/openai/cmd/openai-example-bot

Batteries included:

  • Use ChatGPT 3 & 4, text-davinci-003 and fine-tuned models
  • Use Embeddings API to add knowledge base excerpts (“context”) to your prompts
  • Stream chat completions
  • Compute token count (plus a full tokenizer with encoding/decoding)
  • Compute costs
  • Utilities to trim history

Pragmatic:

  • No dependencies
  • No abstractions
  • Under 1500 lines of code (you can actually read and understand the entire code here)
  • Sensible error handling

Status

Used in production. Note: if you're looking to hire someone to build an AI bot, ping me at andrey@tarantsov.com.

Example Bot

func main() {
    opt := openai.DefaultChatOptions()
    opt.MaxTokens = 768 // limit response length
    chat := []openai.Msg{
        openai.SystemMsg("You are a helpful assistant. Many would say you are way too cheerful and over the top. Answer concisely, adding jokes and exclamantions."),
    }
    client := &http.Client{
        Timeout: 2 * time.Minute, // sometimes this stuff takes a long time to respond
    }

    scanner := bufio.NewScanner(bufio.NewReader(os.Stdin))
    for scanner.Scan() {
        input := scanner.Text()

        // Limit input message length to 768 tokens
        if openai.TokenCount(input, opt.Model) > 768 {
            fmt.Printf("** message too long\n")
            continue
        }

        // Add message to history and truncate oldest messages if history no longer fits
        chat = append(chat, openai.UserMsg(input))
        chat, _ = openai.DropChatHistoryIfNeeded(chat, 1, openai.MaxTokens(opt.Model), opt.Model)

        msgs, usage, err := openai.Chat(context.Background(), chat, opt, client, creds)
        if err != nil {
            fmt.Printf("** %v\n", err)
            continue
        }

        fmt.Printf("\nChatGPT: %s\n\n", strings.TrimSpace(msgs[0].Content))
        chat = append(chat, msgs[0])
    }
}

See cmd/openai-example-bot/example.go for the full runnable code.

Contributing

Contributions are welcome, but keep in mind that I want to keep this library short and focused.

Auto-testing via modd (go install github.com/cortesi/modd/cmd/modd@latest):

modd

Updating binary files:

src=~/Downloads/GPT-3-Encoder-master
cp $src/vocab.bpe tokenizer-bpe.bin
go run ./_convert-encoder-json-to-tokens-bin.go $src/encoder.json tokenizer-tokens.bin

TODO:

  • Rewrite tokenizer based on code and binary data from tiktoken

MIT License

Copyright © 2023, Andrey Tarantsov. Distributed under the MIT license.

Documentation

Index

Constants

View Source 
const (
	// ModelChatGPT4o is the current best chat model, gpt-4o, with 128k context.
	ModelChatGPT4o = "gpt-4o"

	// ModelChatGPT4oMini is the current best low-cost model, gpt-4o-mini, with 128k context.
	ModelChatGPT4oMini = "gpt-4o-mini"

	// ModelChatGPT4Turbo is the first 128k context model, gpt-4-turbo.
	ModelChatGPT4Turbo = "gpt-4-turbo"

	// ModelChatGPT4TurboPreview is the preview of GPT-4 Turbo, with 128k context.
	ModelChatGPT4TurboPreview = "gpt-4-turbo-preview"

	// ModelChatGPT4 is the current best chat model, gpt-4, with 8k context.
	ModelChatGPT4 = "gpt-4"

	// ModelChatGPT4With32k is a version of ModelChatGPT4 with a 32k context.
	ModelChatGPT4With32k = "gpt-4-32k"

	// ModelChatGPT35Turbo is the current best, cheapest and universally available ChatGPT 3.5 model.
	ModelChatGPT35Turbo = "gpt-3.5-turbo"

	// ModelDefaultChat is a chat model used by default. This is going to be set to whatever
	// default choice the author of this library feels appropriate going forward, but really,
	// you should be specifying a specific model like ModelChatGPT4 or ModelChatGPT35Turbo.
	ModelDefaultChat = ModelChatGPT4Turbo

	// ModelDefaultCompletion is the current best instruction-following model for text completion.
	// Not recommended for basically anything any more because gpt-3.5-turbo is 10x cheaper and just as good.
	ModelDefaultCompletion = ModelTextDavinci003

	// ModelTextDavinci003 is the current best instruction-following model.
	// Not recommended for basically anything any more because gpt-3.5-turbo is 10x cheaper and just as good.
	ModelTextDavinci003 = "text-davinci-003"

	// ModelBaseDavinci is an older GPT 3 (not GPT 3.5) family base model. Only useful for fine-tuning.
	ModelBaseDavinci = "davinci"

	// ModelEmbedding3Large is the best embedding model so far.
	ModelEmbedding3Large = "text-embedding-3-large"

	// ModelEmbedding3Small is a better version of ModelEmbeddingAda002.
	ModelEmbedding3Small = "text-embedding-3-small"

	// ModelEmbeddingAda002 is the original embedding model, its use is no longer recommended.
	ModelEmbeddingAda002 = "text-embedding-ada-002"
)

Variables

This section is empty.

Functions

func Chat 

func Chat(ctx context.Context, messages []Msg, opt Options, client *http.Client, creds Credentials) ([]Msg, Usage, error)

Chat suggests the next assistant's message for the given prompt via ChatGPT.. When successful, always returns at least one Msg, more if you set opt.N (these are multiple choices for the next message, not multiple messages). Options should originate from DefaultChatOptions, not DefaultCompleteOptions.

func ChatTokenCount 

func ChatTokenCount(msgs []Msg, model string) int

func Complete 

func Complete(ctx context.Context, prompt string, opt Options, client *http.Client, creds Credentials) ([]Completion, Usage, error)

Complete generates a completion for the given prompt using a non-chat model. This is mainly of interest when using fine-tuned models now. When successful, always returns at least one Completion; more if you set opt.N. Options should originate from DefaultCompleteOptions, not DefaultChatOptions.

func Decode 

func Decode(tokens []int, model string) string

func Encode 

func Encode(text, model string) []int

func EncodeEnum 

func EncodeEnum(text, model string, f func(int))

func MaxTokens 

func MaxTokens(model string) int

MaxTokens returns the maximum number of tokens the given model supports. This is a sum of prompt and completion tokens.

func MsgTokenCount 

func MsgTokenCount(msg Msg, model string) int

func TokenCount 

func TokenCount(text, model string) int

TokenCount counts GPT-3 tokens in the given text for the given model.

Types

type Completion 

type Completion struct {
	Text         string       `json:"text"`
	FinishReason FinishReason `json:"finish_reason"`
}

type Credentials 

type Credentials struct {
	APIKey         string
	OrganizationID string
}

Credentials are used to authenticate with OpenAI.

type Error 

type Error struct {
	CallID            string
	IsNetwork         bool
	StatusCode        int
	Type              string
	Message           string
	RawResponseBody   []byte
	PrintResponseBody bool
	Cause             error
}

func (*Error) Error 

func (e *Error) Error() string

func (*Error) Unwrap 

func (e *Error) Unwrap() error

type FinishReason 

type FinishReason string
const (
	FinishReasonStop   FinishReason = "stop"
	FinishReasonLength FinishReason = "length"
)

type ForceFunctionCall 

type ForceFunctionCall struct {
	Name string `json:"name"`
}

ForceFunctionCall is a value to use in Options.FunctionCallMode.

type FunctionCall 

type FunctionCall struct {
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
}

func (*FunctionCall) UnmarshalArguments 

func (call *FunctionCall) UnmarshalArguments(out any) error

type Msg 

type Msg struct {
	Role         Role          `json:"role"`
	Content      string        `json:"content"`
	FunctionCall *FunctionCall `json:"function_call,omitempty"`
}

Msg is a single chat message.

func AssistantMsg 

func AssistantMsg(content string) Msg

AssistantMsg makes an Msg with an Assistant role.

func DropChatHistoryIfNeeded 

func DropChatHistoryIfNeeded(chat []Msg, fixedSuffixLen int, maxTokens int, model string) ([]Msg, int)

func FitChatContext 

func FitChatContext(candidates []Msg, maxTokenCount int, model string) ([]Msg, int)

FitChatContext returns messages that fit into maxTokenCount, skipping those that don't fit. This is meant for including knowledge base context into ChatGPT prompts.

func StreamChat 

func StreamChat(ctx context.Context, messages []Msg, opt Options, client *http.Client, creds Credentials, f func(msg *Msg, delta string) error) (Msg, error)

StreamChat suggests the next assistant's message for the given prompt via ChatGPT, streaming the response. Options should originate from DefaultChatOptions, not DefaultCompleteOptions. Options.N must be 0 or 1.

func SystemMsg 

func SystemMsg(content string) Msg

SystemMsg makes an Msg with a System role.

func UserMsg 

func UserMsg(content string) Msg

UserMsg makes an Msg with a User role.

func (*Msg) UnmarshalCallArguments 

func (msg *Msg) UnmarshalCallArguments(out any) error

type Options 

type Options struct {
	// Model is the OpenAI model to use, see https://platform.openai.com/docs/models/.
	//
	Model string `json:"model"`

	// MaxTokens is upper limit on completion length. In chat API, use 0 to allow the maximum possible length (4096 minus prompt length).
	MaxTokens int `json:"max_tokens,omitempty"`

	Functions        []any `json:"functions,omitempty"`
	FunctionCallMode any   `json:"function_call,omitempty"`
	Tools            []any `json:"tools,omitempty"`
	ToolChoice       any   `json:"tool_choice,omitempty"`

	Temperature float64 `json:"temperature"`

	TopP float64 `json:"top_p"`

	// N determines how many choices to return for each prompt. Defaults to 1. Must be less or equal to BestOf if both are specified.
	N int `json:"n,omitempty"`

	// BestOf determines how many choices to create for each prompt. Defaults to 1. Must be greater or equal to N if both are specified.
	BestOf int `json:"best_of,omitempty"`

	// Stop is up to 4 sequences where the API will stop generating tokens. Response will not contain the stop sequence.
	Stop []string `json:"stop,omitempty"`

	// PresencePenalty number between 0 and 1 that penalizes tokens that have already appeared in the text so far.
	PresencePenalty float64 `json:"presence_penalty"`

	// FrequencyPenalty number between 0 and 1 that penalizes tokens on existing frequency in the text so far.
	FrequencyPenalty float64 `json:"frequency_penalty"`
}

Options adjust details of how Chat and Complete calls behave.

func DefaultChatOptions 

func DefaultChatOptions() Options

DefaultChatOptions provides a safe and conservative starting point for Chat call options. Note that it sets Temperature to 0 and enables unlimited MaxTokens.

func DefaultCompleteOptions 

func DefaultCompleteOptions() Options

DefaultCompleteOptions provides a safe and conservative starting point for Complete call options. Note that it sets Temperature to 0 and MaxTokens to 256.

type Price 

type Price int64

Price is an amount in 1/1_000_000 of a cent. I.e. $2 per 1M tokens = $0.002 per 1K tokens = Price(200) per token. Max price is thus $92_233_720_368 ($92 billion). But looks like we'll have to scale it soon!

func Cost 

func Cost(promptTokens, completionTokens int, model string) Price

Cost estimates the cost of processing the given number of prompt & completion tokens with the given model.

func FineTuningCost 

func FineTuningCost(tokens int, model string) Price

FineTuningCost estimates the cost of fine-tuning the given model using the given number of tokens of sample data.

func (Price) String 

func (p Price) String() string

String formats the price as dollars and cents, e.g. $3.14.

type Role 

type Role string
const (
	System    Role = "system"
	User      Role = "user"
	Assistant Role = "assistant"
)

type Usage 

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

func ComputeEmbedding 

func ComputeEmbedding(ctx context.Context, input string, client *http.Client, creds Credentials) ([]float64, Usage, error)

Source Files

View all Source files

Directories

Path Synopsis
cmd
convert-encoder-json-to-tokens-bin
openai-example-bot
openai-token-count

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.