pegwings

package module
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 18, 2024 License: MIT Imports: 16 Imported by: 0

README

pegwings-go

Go Reference Go Report Card Coverage Status PhormAI

Features

  • Supports all models from Groq in a type-safe way.
  • Supports streaming.
  • Supports moderation.
  • Supports audio transcription.
  • Supports audio translation.
  • Supports Tool Use.
  • Supports Function Calling.
  • JSON Schema Generation from structs.
  • Supports Toolhouse function calling. Extention
  • Supports E2b function calling. Extention
  • Supports Composio function calling. Extention
  • Supports Jigsaw Stack function calling. Extention

Installation

go get github.com/pegwings/pegwings-go

Examples

For introductory examples, see the examples directory.

External Repositories using pegwings-go:

Development

This project uses devenv for development.

To start development:

devenv allow

To run tests:

test

If you fork the repository, you should set up the following environment variables in ci/cd:

export E2B_API_KEY=your-e2b-api-key
export GROQ_KEY=your-groq-key
export TOOLHOUSE_API_KEY=your-toolhouse-api-key

Documentation

The following documentation is generated from the source code using gomarkdoc.

groq

import "github.com/pegwings/pegwings-go"

Package groq provides a unofficial client for the Groq API.

With specially designed hardware, the Groq API is a super fast way to query open source llms.

API Documentation: https://console.pegwings.com/docs/quickstart

Index

Constants

const (
    // RoleSystem is the system chat message role.
    RoleSystem Role = "system"
    // RoleUser is the user chat message role.
    RoleUser Role = "user"
    // RoleAssistant is the assistant chat message role.
    RoleAssistant Role = "assistant"
    // RoleFunction is the function chat message role.
    RoleFunction Role = "function"
    // RoleTool is the tool chat message role.
    RoleTool Role = "tool"

    // ImageURLDetailHigh is the high image url detail.
    ImageURLDetailHigh ImageURLDetail = "high"
    // ImageURLDetailLow is the low image url detail.
    ImageURLDetailLow ImageURLDetail = "low"
    // ImageURLDetailAuto is the auto image url detail.
    ImageURLDetailAuto ImageURLDetail = "auto"

    // ChatMessagePartTypeText is the text chat message part type.
    ChatMessagePartTypeText ChatMessagePartType = "text"
    // ChatMessagePartTypeImageURL is the image url chat message part type.
    ChatMessagePartTypeImageURL ChatMessagePartType = "image_url"
)

Variables

var (
    // ModelGemma29BIt is an AI text chat model.
    //
    // It is created/provided by Google.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelGemma29BIt ChatModel = "gemma2-9b-it"
    // ModelGemma7BIt is an AI text chat model.
    //
    // It is created/provided by Google.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelGemma7BIt ChatModel = "gemma-7b-it"
    // ModelLlama3170BVersatile is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 32768 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama3170BVersatile ChatModel = "llama-3.1-70b-versatile"
    // ModelLlama318BInstant is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 131072 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama318BInstant ChatModel = "llama-3.1-8b-instant"
    // ModelLlama3211BVisionPreview is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama3211BVisionPreview ChatModel = "llama-3.2-11b-vision-preview"
    // ModelLlama321BPreview is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama321BPreview ChatModel = "llama-3.2-1b-preview"
    // ModelLlama323BPreview is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama323BPreview ChatModel = "llama-3.2-3b-preview"
    // ModelLlama3290BVisionPreview is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama3290BVisionPreview ChatModel = "llama-3.2-90b-vision-preview"
    // ModelLlama3370BSpecdec is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama3370BSpecdec ChatModel = "llama-3.3-70b-specdec"
    // ModelLlama3370BVersatile is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 32768 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama3370BVersatile ChatModel = "llama-3.3-70b-versatile"
    // ModelLlama370B8192 is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama370B8192 ChatModel = "llama3-70b-8192"
    // ModelLlama38B8192 is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama38B8192 ChatModel = "llama3-8b-8192"
    // ModelLlama3Groq70B8192ToolUsePreview is an AI text chat model.
    //
    // It is created/provided by pegwings.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama3Groq70B8192ToolUsePreview ChatModel = "llama3-groq-70b-8192-tool-use-preview"
    // ModelLlama3Groq8B8192ToolUsePreview is an AI text chat model.
    //
    // It is created/provided by pegwings.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelLlama3Groq8B8192ToolUsePreview ChatModel = "llama3-groq-8b-8192-tool-use-preview"
    // ModelMixtral8X7B32768 is an AI text chat model.
    //
    // It is created/provided by Mistral AI.
    //
    // It has 32768 context window.
    //
    // It can be used with the following client methods:
    //	- ChatCompletion
    // 	- ChatCompletionStream
    // 	- ChatCompletionJSON
    ModelMixtral8X7B32768 ChatModel = "mixtral-8x7b-32768"
    // ModelDistilWhisperLargeV3En is an AI audio transcription model.
    //
    // It is created/provided by Hugging Face.
    //
    // It has 448 context window.
    //
    // It can be used with the following client methods:
    //	- CreateTranscription
    // 	- CreateTranslation
    ModelDistilWhisperLargeV3En AudioModel = "distil-whisper-large-v3-en"
    // ModelWhisperLargeV3 is an AI audio transcription model.
    //
    // It is created/provided by OpenAI.
    //
    // It has 448 context window.
    //
    // It can be used with the following client methods:
    //	- CreateTranscription
    // 	- CreateTranslation
    ModelWhisperLargeV3 AudioModel = "whisper-large-v3"
    // ModelWhisperLargeV3Turbo is an AI audio transcription model.
    //
    // It is created/provided by OpenAI.
    //
    // It has 448 context window.
    //
    // It can be used with the following client methods:
    //	- CreateTranscription
    // 	- CreateTranslation
    ModelWhisperLargeV3Turbo AudioModel = "whisper-large-v3-turbo"
    // ModelLlamaGuard38B is an AI moderation model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- Moderate
    ModelLlamaGuard38B ModerationModel = "llama-guard-3-8b"
)

func AudioMultipartForm

func AudioMultipartForm(request AudioRequest, b builders.FormBuilder) error

AudioMultipartForm creates a form with audio file contents and the name of the model to use for audio processing.

type AudioModel

AudioModel is the type for audio models present on the groq api.

type AudioModel Model

type AudioRequest

AudioRequest represents a request structure for audio API.

type AudioRequest struct {
    // Model is the model to use for the transcription.
    Model AudioModel
    // FilePath is either an existing file in your filesystem or a
    // filename representing the contents of Reader.
    FilePath string
    // Reader is an optional io.Reader when you do not want to use
    // an existing file.
    Reader io.Reader
    // Prompt is the prompt for the transcription.
    Prompt string
    // Temperature is the temperature for the transcription.
    Temperature float32
    // Language is the language for the transcription. Only for
    // transcription.
    Language string
    // Format is the format for the response.
    Format Format
}

type AudioResponse

AudioResponse represents a response structure for audio API.

type AudioResponse struct {
    // Task is the task of the response.
    Task string `json:"task"`
    // Language is the language of the response.
    Language string `json:"language"`
    // Duration is the duration of the response.
    Duration float64 `json:"duration"`
    // Segments is the segments of the response.
    Segments Segments `json:"segments"`
    // Words is the words of the response.
    Words Words `json:"words"`
    // Text is the text of the response.
    Text string `json:"text"`

    Header http.Header // Header is the header of the response.
}

func (*AudioResponse) SetHeader
func (r *AudioResponse) SetHeader(header http.Header)

SetHeader sets the header of the response.

type ChatCompletionChoice

ChatCompletionChoice represents the chat completion choice.

type ChatCompletionChoice struct {
    Index int `json:"index"` // Index is the index of the choice.
    // Message is the chat completion message of the choice.
    Message ChatCompletionMessage `json:"message"`
    // FinishReason is the finish reason of the choice.
    FinishReason FinishReason `json:"finish_reason"`
    // LogProbs is the log probs of the choice.
    //
    // This is basically the probability of the model choosing the
    // token.
    LogProbs *LogProbs `json:"logprobs,omitempty"`
}

type ChatCompletionMessage

ChatCompletionMessage represents the chat completion message.

type ChatCompletionMessage struct {
    // Name is the name of the chat completion message.
    Name string `json:"name"`
    // Role is the role of the chat completion message.
    Role Role `json:"role"`
    // Content is the content of the chat completion message.
    Content string `json:"content"`
    // MultiContent is the multi content of the chat completion
    // message.
    MultiContent []ChatMessagePart `json:"-"`
    // FunctionCall setting for Role=assistant prompts this may be
    // set to the function call generated by the model.
    FunctionCall *tools.FunctionCall `json:"function_call,omitempty"`
    // ToolCalls setting for Role=assistant prompts this may be set
    // to the tool calls generated by the model, such as function
    // calls.
    ToolCalls []tools.ToolCall `json:"tool_calls,omitempty"`
    // ToolCallID is setting for Role=tool prompts this should be
    // set to the ID given in the assistant's prior request to call
    // a tool.
    ToolCallID string `json:"tool_call_id,omitempty"`
}

func (ChatCompletionMessage) MarshalJSON
func (m ChatCompletionMessage) MarshalJSON() ([]byte, error)

MarshalJSON method implements the json.Marshaler interface.

It exists to allow for the use of the multi-part content field.

func (*ChatCompletionMessage) UnmarshalJSON
func (m *ChatCompletionMessage) UnmarshalJSON(bs []byte) (err error)

UnmarshalJSON method implements the json.Unmarshaler interface.

It exists to allow for the use of the multi-part content field.

type ChatCompletionRequest

ChatCompletionRequest represents a request structure for the chat completion API.

type ChatCompletionRequest struct {
    // Model is the model of the chat completion request.
    Model ChatModel `json:"model"`
    // Messages is the messages of the chat completion request.
    //
    // These act as the prompt for the model.
    Messages []ChatCompletionMessage `json:"messages"`
    // MaxTokens is the max tokens of the chat completion request.
    MaxTokens int `json:"max_tokens,omitempty"`
    // Temperature is the temperature of the chat completion
    // request.
    Temperature float32 `json:"temperature,omitempty"`
    // TopP is the top p of the chat completion request.
    TopP float32 `json:"top_p,omitempty"`
    // N is the n of the chat completion request.
    N   int `json:"n,omitempty"`
    // Stream is the stream of the chat completion request.
    Stream bool `json:"stream,omitempty"`
    // Stop is the stop of the chat completion request.
    Stop []string `json:"stop,omitempty"`
    // PresencePenalty is the presence penalty of the chat
    // completion request.
    PresencePenalty float32 `json:"presence_penalty,omitempty"`
    // ResponseFormat is the response format of the chat completion
    // request.
    ResponseFormat *ChatCompletionResponseFormat `json:"response_format,omitempty"`
    // Seed is the seed of the chat completion request.
    Seed *int `json:"seed,omitempty"`
    // FrequencyPenalty is the frequency penalty of the chat
    // completion request.
    FrequencyPenalty float32 `json:"frequency_penalty,omitempty"`
    // LogitBias is must be a token id string (specified by their
    // token ID in the tokenizer), not a word string. incorrect: `"logit_bias":{ "You": 6}`, correct: `"logit_bias":{"1639": 6}` refs: https://platform.openai.com/docs/api-reference/chat/create#chat/create-logit_bias
    LogitBias map[string]int `json:"logit_bias,omitempty"`
    // LogProbs indicates whether to return log probabilities of the
    // output tokens or not. If true, returns the log probabilities
    // of each output token returned in the content of message.
    //
    // This option is currently not available on the
    // gpt-4-vision-preview model.
    LogProbs bool `json:"logprobs,omitempty"`
    // TopLogProbs is an integer between 0 and 5 specifying the
    // number of most likely tokens to return at each token
    // position, each with an associated log probability. Logprobs
    // must be set to true if this parameter is used.
    TopLogProbs int `json:"top_logprobs,omitempty"`
    // User is the user of the chat completion request.
    User string `json:"user,omitempty"`
    // Tools is the tools of the chat completion request.
    Tools []tools.Tool `json:"tools,omitempty"`
    // This can be either a string or an ToolChoice object.
    ToolChoice any `json:"tool_choice,omitempty"`
    // Options for streaming response. Only set this when you set stream: true.
    StreamOptions *StreamOptions `json:"stream_options,omitempty"`
    // Disable the default behavior of parallel tool calls by setting it: false.
    ParallelToolCalls any `json:"parallel_tool_calls,omitempty"`
    // RetryDelay is the delay between retries.
    RetryDelay time.Duration `json:"-"`
}

type ChatCompletionResponse

ChatCompletionResponse represents a response structure for chat completion API.

type ChatCompletionResponse struct {
    // ID is the id of the response.
    ID  string `json:"id"`
    // Object is the object of the response.
    Object string `json:"object"`
    // Created is the created time of the response.
    Created int64 `json:"created"`
    // Model is the model of the response.
    Model ChatModel `json:"model"`
    // Choices is the choices of the response.
    Choices []ChatCompletionChoice `json:"choices"`
    // Usage is the usage of the response.
    Usage Usage `json:"usage"`
    // SystemFingerprint is the system fingerprint of the response.
    SystemFingerprint string `json:"system_fingerprint"`
    // Header is the header of the response.
    http.Header
}

func (*ChatCompletionResponse) SetHeader
func (r *ChatCompletionResponse) SetHeader(h http.Header)

SetHeader sets the header of the response.

type ChatCompletionResponseFormat

ChatCompletionResponseFormat is the chat completion response format.

type ChatCompletionResponseFormat struct {
    // Type is the type of the chat completion response format.
    Type Format `json:"type,omitempty"`
    // JSONSchema is the json schema of the chat completion response
    // format.
    JSONSchema *ChatCompletionResponseFormatJSONSchema `json:"json_schema,omitempty"`
}

type ChatCompletionResponseFormatJSONSchema

ChatCompletionResponseFormatJSONSchema is the chat completion response format json schema.

type ChatCompletionResponseFormatJSONSchema struct {
    // Name is the name of the chat completion response format json
    // schema.
    //
    // it is used to further identify the schema in the response.
    Name string `json:"name"`
    // Description is the description of the chat completion
    // response format json schema.
    Description string `json:"description,omitempty"`
    // Schema is the schema of the chat completion response format
    // json schema.
    Schema schema.Schema `json:"schema"`
    // Strict determines whether to enforce the schema upon the
    // generated content.
    Strict bool `json:"strict"`
}

type ChatCompletionStream

ChatCompletionStream is a stream of ChatCompletionStreamResponse.

type ChatCompletionStream struct {
    // contains filtered or unexported fields
}

type ChatCompletionStreamChoice

ChatCompletionStreamChoice represents a response structure for chat completion API.

type ChatCompletionStreamChoice struct {
    // Index is the index of the choice.
    Index int `json:"index"`
    // Delta is the delta of the choice.
    Delta ChatCompletionStreamChoiceDelta `json:"delta"`
    // FinishReason is the finish reason of the choice.
    FinishReason FinishReason `json:"finish_reason"`
}

type ChatCompletionStreamChoiceDelta

ChatCompletionStreamChoiceDelta represents a response structure for chat completion API.

type ChatCompletionStreamChoiceDelta struct {
    // Content is the content of the response.
    Content string `json:"content,omitempty"`
    // Role is the role of the creator of the completion.
    Role string `json:"role,omitempty"`
    // FunctionCall is the function call of the response.
    FunctionCall *tools.FunctionCall `json:"function_call,omitempty"`
    // ToolCalls are the tool calls of the response.
    ToolCalls []tools.ToolCall `json:"tool_calls,omitempty"`
}

type ChatCompletionStreamResponse

ChatCompletionStreamResponse represents a response structure for chat completion API.

type ChatCompletionStreamResponse struct {
    // ID is the identifier for the chat completion stream response.
    ID  string `json:"id"`
    // Object is the object type of the chat completion stream
    // response.
    Object string `json:"object"`
    // Created is the creation time of the chat completion stream
    // response.
    Created int64 `json:"created"`
    // Model is the model used for the chat completion stream
    // response.
    Model ChatModel `json:"model"`
    // Choices is the choices for the chat completion stream
    // response.
    Choices []ChatCompletionStreamChoice `json:"choices"`
    // SystemFingerprint is the system fingerprint for the chat
    // completion stream response.
    SystemFingerprint string `json:"system_fingerprint"`
    // PromptAnnotations is the prompt annotations for the chat
    // completion stream response.
    PromptAnnotations []PromptAnnotation `json:"prompt_annotations,omitempty"`
    // PromptFilterResults is the prompt filter results for the chat
    // completion stream response.
    PromptFilterResults []struct {
        Index int `json:"index"`
    }   `json:"prompt_filter_results,omitempty"`
    // Usage is an optional field that will only be present when you
    // set stream_options: {"include_usage": true} in your request.
    //
    // When present, it contains a null value except for the last
    // chunk which contains the token usage statistics for the
    // entire request.
    Usage *Usage `json:"usage,omitempty"`
}

type ChatMessageImageURL

ChatMessageImageURL represents the chat message image url.

type ChatMessageImageURL struct {
    // URL is the url of the image.
    URL string `json:"url,omitempty"`
    // Detail is the detail of the image url.
    Detail ImageURLDetail `json:"detail,omitempty"`
}

type ChatMessagePart

ChatMessagePart represents the chat message part of a chat completion message.

type ChatMessagePart struct {
    // Text is the text of the chat message part.
    Text string `json:"text,omitempty"`
    // Type is the type of the chat message part.
    Type ChatMessagePartType `json:"type,omitempty"`
    // ImageURL is the image url of the chat message part.
    ImageURL *ChatMessageImageURL `json:"image_url,omitempty"`
}

type ChatMessagePartType

ChatMessagePartType is the chat message part type.

string

type ChatMessagePartType string

type ChatModel

ChatModel is the type for chat models present on the groq api.

type ChatModel Model

type Client

Client is a Groq api client.

type Client struct {
    // contains filtered or unexported fields
}

func NewClient
func NewClient(groqAPIKey string, opts ...Opts) (*Client, error)

NewClient creates a new Groq client.

func (*Client) ChatCompletion
func (c *Client) ChatCompletion(ctx context.Context, request ChatCompletionRequest) (response ChatCompletionResponse, err error)

ChatCompletion method is an API call to create a chat completion.

func (*Client) ChatCompletionJSON
func (c *Client) ChatCompletionJSON(ctx context.Context, request ChatCompletionRequest, output any) (err error)

ChatCompletionJSON method is an API call to create a chat completion w/ object output.

func (*Client) ChatCompletionStream
func (c *Client) ChatCompletionStream(ctx context.Context, request ChatCompletionRequest) (stream *ChatCompletionStream, err error)

ChatCompletionStream method is an API call to create a chat completion w/ streaming support.

func (*Client) CreateTranscription
func (c *Client) CreateTranscription(ctx context.Context, request AudioRequest) (AudioResponse, error)

CreateTranscription calls the transcriptions endpoint with the given request.

Returns transcribed text in the response_format specified in the request.

func (*Client) CreateTranslation
func (c *Client) CreateTranslation(ctx context.Context, request AudioRequest) (AudioResponse, error)

CreateTranslation calls the translations endpoint with the given request.

Returns the translated text in the response_format specified in the request.

func (*Client) Moderate
func (c *Client) Moderate(ctx context.Context, messages []ChatCompletionMessage, model ModerationModel) (response Moderation, err error)

Moderate performs a moderation api call over a string. Input can be an array or slice but a string will reduce the complexity.

type FinishReason

FinishReason is the finish reason.

string

type FinishReason string


const (
    // ReasonStop is the stop finish reason for a chat completion.
    ReasonStop FinishReason = "stop"
    // ReasonLength is the length finish reason for a chat completion.
    ReasonLength FinishReason = "length"
    // ReasonFunctionCall is the function call finish reason for a chat
    // completion.
    ReasonFunctionCall FinishReason = "function_call"
    // ReasonToolCalls is the tool calls finish reason for a chat
    // completion.
    ReasonToolCalls FinishReason = "tool_calls"
    // ReasonContentFilter is the content filter finish reason for a chat
    // completion.
    ReasonContentFilter FinishReason = "content_filter"
    // ReasonNull is the null finish reason for a chat completion.
    ReasonNull FinishReason = "null"
)

func (FinishReason) MarshalJSON
func (r FinishReason) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

type Format

Format is the format of a response. string

type Format string


const (
    // FormatText is the text format. It is the default format of a
    // response.
    FormatText Format = "text"
    // FormatJSON is the JSON format. There is no support for streaming with
    // JSON format selected.
    FormatJSON Format = "json"
    // FormatSRT is the SRT format. This is a text format that is only
    // supported for the transcription API.
    // SRT format selected.
    FormatSRT Format = "srt"
    // FormatVTT is the VTT format. This is a text format that is only
    // supported for the transcription API.
    FormatVTT Format = "vtt"
    // FormatVerboseJSON is the verbose JSON format. This is a JSON format
    // that is only supported for the transcription API.
    FormatVerboseJSON Format = "verbose_json"
    // FormatJSONObject is the json object chat
    // completion response format type.
    FormatJSONObject Format = "json_object"
    // FormatJSONSchema is the json schema chat
    // completion response format type.
    FormatJSONSchema Format = "json_schema"
)

type ImageURLDetail

ImageURLDetail is the detail of the image at the URL.

string

type ImageURLDetail string

type LogProbs

LogProbs is the top-level structure containing the log probability information.

type LogProbs struct {
    // Content is a list of message content tokens with log
    // probability information.
    Content []struct {
        // Token is the token of the log prob.
        Token string `json:"token"`
        // LogProb is the log prob of the log prob.
        LogProb float64 `json:"logprob"`
        // Omitting the field if it is null
        Bytes []byte `json:"bytes,omitempty"`
        // TopLogProbs is a list of the most likely tokens and
        // their log probability, at this token position. In
        // rare cases, there may be fewer than the number of
        // requested top_logprobs returned.
        TopLogProbs []TopLogProbs `json:"top_logprobs"`
    } `json:"content"`
}

type Model

Model is a ai model accessible through the groq api.

type Model string

type Moderation

Moderation represents the response of a moderation request.

type Moderation struct {
    // Categories is the categories of the result.
    Categories []moderation.HarmfulCategory `json:"categories"`
    // Flagged is the flagged status of the result.
    Flagged bool `json:"flagged"`
}

type ModerationModel

ModerationModel is the type for moderation models present on the groq api.

type ModerationModel Model

type Opts

Opts is a function that sets options for a Groq client.

type Opts func(*Client)

func WithBaseURL
func WithBaseURL(baseURL string) Opts

WithBaseURL sets the base URL for the Groq client.

func WithClient
func WithClient(client *http.Client) Opts

WithClient sets the client for the Groq client.

func WithLogger
func WithLogger(logger *slog.Logger) Opts

WithLogger sets the logger for the Groq client.

type PromptAnnotation

PromptAnnotation represents the prompt annotation.

type PromptAnnotation struct {
    PromptIndex int `json:"prompt_index,omitempty"`
}

type RateLimitHeaders

RateLimitHeaders struct represents Groq rate limits headers.

type RateLimitHeaders struct {
    // LimitRequests is the limit requests of the rate limit
    // headers.
    LimitRequests int `json:"x-ratelimit-limit-requests"`
    // LimitTokens is the limit tokens of the rate limit headers.
    LimitTokens int `json:"x-ratelimit-limit-tokens"`
    // RemainingRequests is the remaining requests of the rate
    // limit headers.
    RemainingRequests int `json:"x-ratelimit-remaining-requests"`
    // RemainingTokens is the remaining tokens of the rate limit
    // headers.
    RemainingTokens int `json:"x-ratelimit-remaining-tokens"`
    // ResetRequests is the reset requests of the rate limit
    // headers.
    ResetRequests ResetTime `json:"x-ratelimit-reset-requests"`
    // ResetTokens is the reset tokens of the rate limit headers.
    ResetTokens ResetTime `json:"x-ratelimit-reset-tokens"`
}

type ResetTime

ResetTime is a time.Time wrapper for the rate limit reset time. string

type ResetTime string

func (ResetTime) String
func (r ResetTime) String() string

String returns the string representation of the ResetTime.

func (ResetTime) Time
func (r ResetTime) Time() time.Time

Time returns the time.Time representation of the ResetTime.

type Role

Role is the role of the chat completion message.

string

type Role string

type Segments

Segments is the segments of the response.

type Segments []struct {
    // ID is the ID of the segment.
    ID  int `json:"id"`
    // Seek is the seek of the segment.
    Seek int `json:"seek"`
    // Start is the start of the segment.
    Start float64 `json:"start"`
    // End is the end of the segment.
    End float64 `json:"end"`
    // Text is the text of the segment.
    Text string `json:"text"`
    // Tokens is the tokens of the segment.
    Tokens []int `json:"tokens"`
    // Temperature is the temperature of the segment.
    Temperature float64 `json:"temperature"`
    // AvgLogprob is the avg log prob of the segment.
    AvgLogprob float64 `json:"avg_logprob"`
    // CompressionRatio is the compression ratio of the segment.
    CompressionRatio float64 `json:"compression_ratio"`
    // NoSpeechProb is the no speech prob of the segment.
    NoSpeechProb float64 `json:"no_speech_prob"`
    // Transient is the transient of the segment.
    Transient bool `json:"transient"`
}

type StreamOptions

StreamOptions represents the stream options.

type StreamOptions struct {
    // IncludeUsage is the include usage option of the stream
    // options.
    //
    // If set, an additional chunk will be streamed before the data:
    // [DONE] message.
    // The usage field on this chunk shows the token usage
    // statistics for the entire request, and the choices field will
    // always be an empty array.
    //
    // All other chunks will also include a usage field, but with a
    // null value.
    IncludeUsage bool `json:"include_usage,omitempty"`
}

type TopLogProbs

TopLogProbs represents the top log probs.

type TopLogProbs struct {
    // Token is the token of the top log probs.
    Token string `json:"token"`
    // LogProb is the log prob of the top log probs.
    LogProb float64 `json:"logprob"`
    // Bytes is the bytes of the top log probs.
    Bytes []byte `json:"bytes,omitempty"`
}

type TranscriptionTimestampGranularity

TranscriptionTimestampGranularity is the timestamp granularity for the transcription.

string

type TranscriptionTimestampGranularity string


const (
    // TranscriptionTimestampGranularityWord is the word timestamp
    // granularity.
    TranscriptionTimestampGranularityWord TranscriptionTimestampGranularity = "word"
    // TranscriptionTimestampGranularitySegment is the segment timestamp
    // granularity.
    TranscriptionTimestampGranularitySegment TranscriptionTimestampGranularity = "segment"
)

type Usage

Usage Represents the total token usage per request to pegwings.

type Usage struct {
    PromptTokens     int `json:"prompt_tokens"`
    CompletionTokens int `json:"completion_tokens"`
    TotalTokens      int `json:"total_tokens"`
}

type Words

Words is the words of the audio response.

type Words []struct {
    // Word is the textual representation of a word in the audio
    // response.
    Word string `json:"word"`
    // Start is the start of the words in seconds.
    Start float64 `json:"start"`
    // End is the end of the words in seconds.
    End float64 `json:"end"`
}

Generated by gomarkdoc

pegwings-go

Documentation

Overview

Package pegwings provides a unofficial client for the Groq API.

With specially designed hardware, the Groq API is a super fast way to query open source llms.

API Documentation: https://console.pegwings.com/docs/quickstart

Index

Constants

View Source 
const (
	// RoleSystem is the system chat message role.
	RoleSystem Role = "system"
	// RoleUser is the user chat message role.
	RoleUser Role = "user"
	// RoleAssistant is the assistant chat message role.
	RoleAssistant Role = "assistant"
	// RoleFunction is the function chat message role.
	RoleFunction Role = "function"
	// RoleTool is the tool chat message role.
	RoleTool Role = "tool"

	// ImageURLDetailHigh is the high image url detail.
	ImageURLDetailHigh ImageURLDetail = "high"
	// ImageURLDetailLow is the low image url detail.
	ImageURLDetailLow ImageURLDetail = "low"
	// ImageURLDetailAuto is the auto image url detail.
	ImageURLDetailAuto ImageURLDetail = "auto"

	// ChatMessagePartTypeText is the text chat message part type.
	ChatMessagePartTypeText ChatMessagePartType = "text"
	// ChatMessagePartTypeImageURL is the image url chat message part type.
	ChatMessagePartTypeImageURL ChatMessagePartType = "image_url"
)

Variables

View Source 
var (
	// ModelGemma29BIt is an AI text chat model.
	//
	// It is created/provided by Google.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelGemma29BIt ChatModel = "gemma2-9b-it"
	// ModelGemma7BIt is an AI text chat model.
	//
	// It is created/provided by Google.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelGemma7BIt ChatModel = "gemma-7b-it"
	// ModelLlama3170BVersatile is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 32768 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama3170BVersatile ChatModel = "llama-3.1-70b-versatile"
	// ModelLlama318BInstant is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 131072 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama318BInstant ChatModel = "llama-3.1-8b-instant"
	// ModelLlama3211BVisionPreview is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama3211BVisionPreview ChatModel = "llama-3.2-11b-vision-preview"
	// ModelLlama321BPreview is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama321BPreview ChatModel = "llama-3.2-1b-preview"
	// ModelLlama323BPreview is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama323BPreview ChatModel = "llama-3.2-3b-preview"
	// ModelLlama3290BVisionPreview is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama3290BVisionPreview ChatModel = "llama-3.2-90b-vision-preview"
	// ModelLlama3370BSpecdec is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama3370BSpecdec ChatModel = "llama-3.3-70b-specdec"
	// ModelLlama3370BVersatile is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 32768 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama3370BVersatile ChatModel = "llama-3.3-70b-versatile"
	// ModelLlama370B8192 is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama370B8192 ChatModel = "llama3-70b-8192"
	// ModelLlama38B8192 is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama38B8192 ChatModel = "llama3-8b-8192"
	// ModelLlama3Groq70B8192ToolUsePreview is an AI text chat model.
	//
	// It is created/provided by pegwings.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama3Groq70B8192ToolUsePreview ChatModel = "llama3-groq-70b-8192-tool-use-preview"
	// ModelLlama3Groq8B8192ToolUsePreview is an AI text chat model.
	//
	// It is created/provided by pegwings.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelLlama3Groq8B8192ToolUsePreview ChatModel = "llama3-groq-8b-8192-tool-use-preview"
	// ModelMixtral8X7B32768 is an AI text chat model.
	//
	// It is created/provided by Mistral AI.
	//
	// It has 32768 context window.
	//
	// It can be used with the following client methods:
	//	- ChatCompletion
	// 	- ChatCompletionStream
	// 	- ChatCompletionJSON
	ModelMixtral8X7B32768 ChatModel = "mixtral-8x7b-32768"
	// ModelDistilWhisperLargeV3En is an AI audio transcription model.
	//
	// It is created/provided by Hugging Face.
	//
	// It has 448 context window.
	//
	// It can be used with the following client methods:
	//	- CreateTranscription
	// 	- CreateTranslation
	ModelDistilWhisperLargeV3En AudioModel = "distil-whisper-large-v3-en"
	// ModelWhisperLargeV3 is an AI audio transcription model.
	//
	// It is created/provided by OpenAI.
	//
	// It has 448 context window.
	//
	// It can be used with the following client methods:
	//	- CreateTranscription
	// 	- CreateTranslation
	ModelWhisperLargeV3 AudioModel = "whisper-large-v3"
	// ModelWhisperLargeV3Turbo is an AI audio transcription model.
	//
	// It is created/provided by OpenAI.
	//
	// It has 448 context window.
	//
	// It can be used with the following client methods:
	//	- CreateTranscription
	// 	- CreateTranslation
	ModelWhisperLargeV3Turbo AudioModel = "whisper-large-v3-turbo"
	// ModelLlamaGuard38B is an AI moderation model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- Moderate
	ModelLlamaGuard38B ModerationModel = "llama-guard-3-8b"
)

Functions

func AudioMultipartForm 

func AudioMultipartForm(request AudioRequest, b builders.FormBuilder) error

AudioMultipartForm creates a form with audio file contents and the name of the model to use for audio processing.

Types

type AudioModel 

type AudioModel Model

AudioModel is the type for audio models present on the groq api.

type AudioRequest 

type AudioRequest struct {
	// Model is the model to use for the transcription.
	Model AudioModel
	// FilePath is either an existing file in your filesystem or a
	// filename representing the contents of Reader.
	FilePath string
	// Reader is an optional io.Reader when you do not want to use
	// an existing file.
	Reader io.Reader
	// Prompt is the prompt for the transcription.
	Prompt string
	// Temperature is the temperature for the transcription.
	Temperature float32
	// Language is the language for the transcription. Only for
	// transcription.
	Language string
	// Format is the format for the response.
	Format Format
}

AudioRequest represents a request structure for audio API.

type AudioResponse 

type AudioResponse struct {
	// Task is the task of the response.
	Task string `json:"task"`
	// Language is the language of the response.
	Language string `json:"language"`
	// Duration is the duration of the response.
	Duration float64 `json:"duration"`
	// Segments is the segments of the response.
	Segments Segments `json:"segments"`
	// Words is the words of the response.
	Words Words `json:"words"`
	// Text is the text of the response.
	Text string `json:"text"`
	// contains filtered or unexported fields
}

AudioResponse represents a response structure for audio API.

func (*AudioResponse) SetHeader 

func (r *AudioResponse) SetHeader(header http.Header)

SetHeader sets the header of the response.

type ChatCompletionChoice 

type ChatCompletionChoice struct {
	Index int `json:"index"` // Index is the index of the choice.
	// Message is the chat completion message of the choice.
	Message ChatCompletionMessage `json:"message"`
	// FinishReason is the finish reason of the choice.
	FinishReason FinishReason `json:"finish_reason"`
	// LogProbs is the log probs of the choice.
	//
	// This is basically the probability of the model choosing the
	// token.
	LogProbs *LogProbs `json:"logprobs,omitempty"`
}

ChatCompletionChoice represents the chat completion choice.

type ChatCompletionMessage 

type ChatCompletionMessage struct {
	// Name is the name of the chat completion message.
	Name string `json:"name"`
	// Role is the role of the chat completion message.
	Role Role `json:"role"`
	// Content is the content of the chat completion message.
	Content string `json:"content"`
	// MultiContent is the multi content of the chat completion
	// message.
	MultiContent []ChatMessagePart `json:"-"`
	// FunctionCall setting for Role=assistant prompts this may be
	// set to the function call generated by the model.
	FunctionCall *tools.FunctionCall `json:"function_call,omitempty"`
	// ToolCalls setting for Role=assistant prompts this may be set
	// to the tool calls generated by the model, such as function
	// calls.
	ToolCalls []tools.ToolCall `json:"tool_calls,omitempty"`
	// ToolCallID is setting for Role=tool prompts this should be
	// set to the ID given in the assistant's prior request to call
	// a tool.
	ToolCallID string `json:"tool_call_id,omitempty"`
}

ChatCompletionMessage represents the chat completion message.

func (ChatCompletionMessage) MarshalJSON 

func (m ChatCompletionMessage) MarshalJSON() ([]byte, error)

MarshalJSON method implements the json.Marshaler interface.

It exists to allow for the use of the multi-part content field.

func (*ChatCompletionMessage) UnmarshalJSON 

func (m *ChatCompletionMessage) UnmarshalJSON(bs []byte) (err error)

UnmarshalJSON method implements the json.Unmarshaler interface.

It exists to allow for the use of the multi-part content field.

type ChatCompletionRequest 

type ChatCompletionRequest struct {
	// Model is the model of the chat completion request.
	Model ChatModel `json:"model"`
	// Messages is the messages of the chat completion request.
	//
	// These act as the prompt for the model.
	Messages []ChatCompletionMessage `json:"messages"`
	// MaxTokens is the max tokens of the chat completion request.
	MaxTokens int `json:"max_tokens,omitempty"`
	// Temperature is the temperature of the chat completion
	// request.
	Temperature float32 `json:"temperature,omitempty"`
	// TopP is the top p of the chat completion request.
	TopP float32 `json:"top_p,omitempty"`
	// N is the n of the chat completion request.
	N int `json:"n,omitempty"`
	// Stream is the stream of the chat completion request.
	Stream bool `json:"stream,omitempty"`
	// Stop is the stop of the chat completion request.
	Stop []string `json:"stop,omitempty"`
	// PresencePenalty is the presence penalty of the chat
	// completion request.
	PresencePenalty float32 `json:"presence_penalty,omitempty"`
	// ResponseFormat is the response format of the chat completion
	// request.
	ResponseFormat *ChatCompletionResponseFormat `json:"response_format,omitempty"`
	// Seed is the seed of the chat completion request.
	Seed *int `json:"seed,omitempty"`
	// FrequencyPenalty is the frequency penalty of the chat
	// completion request.
	FrequencyPenalty float32 `json:"frequency_penalty,omitempty"`
	// LogitBias is must be a token id string (specified by their
	// token ID in the tokenizer), not a word string.
	// incorrect: `"logit_bias":{ "You": 6}`, correct: `"logit_bias":{"1639": 6}`
	// refs: https://platform.openai.com/docs/api-reference/chat/create#chat/create-logit_bias
	LogitBias map[string]int `json:"logit_bias,omitempty"`
	// LogProbs indicates whether to return log probabilities of the
	// output tokens or not. If true, returns the log probabilities
	// of each output token returned in the content of message.
	//
	// This option is currently not available on the
	// gpt-4-vision-preview model.
	LogProbs bool `json:"logprobs,omitempty"`
	// TopLogProbs is an integer between 0 and 5 specifying the
	// number of most likely tokens to return at each token
	// position, each with an associated log probability. Logprobs
	// must be set to true if this parameter is used.
	TopLogProbs int `json:"top_logprobs,omitempty"`
	// User is the user of the chat completion request.
	User string `json:"user,omitempty"`
	// Tools is the tools of the chat completion request.
	Tools []tools.Tool `json:"tools,omitempty"`
	// This can be either a string or an ToolChoice object.
	ToolChoice any `json:"tool_choice,omitempty"`
	// Options for streaming response. Only set this when you set stream: true.
	StreamOptions *StreamOptions `json:"stream_options,omitempty"`
	// Disable the default behavior of parallel tool calls by setting it: false.
	ParallelToolCalls any `json:"parallel_tool_calls,omitempty"`
	// RetryDelay is the delay between retries.
	RetryDelay time.Duration `json:"-"`
}

ChatCompletionRequest represents a request structure for the chat completion API.

type ChatCompletionResponse 

type ChatCompletionResponse struct {
	// ID is the id of the response.
	ID string `json:"id"`
	// Object is the object of the response.
	Object string `json:"object"`
	// Created is the created time of the response.
	Created int64 `json:"created"`
	// Model is the model of the response.
	Model ChatModel `json:"model"`
	// Choices is the choices of the response.
	Choices []ChatCompletionChoice `json:"choices"`
	// Usage is the usage of the response.
	Usage Usage `json:"usage"`
	// SystemFingerprint is the system fingerprint of the response.
	SystemFingerprint string `json:"system_fingerprint"`
	// Header is the header of the response.
	http.Header
}

ChatCompletionResponse represents a response structure for chat completion API.

func (*ChatCompletionResponse) SetHeader 

func (r *ChatCompletionResponse) SetHeader(h http.Header)

SetHeader sets the header of the response.

type ChatCompletionResponseFormat 

type ChatCompletionResponseFormat struct {
	// Type is the type of the chat completion response format.
	Type Format `json:"type,omitempty"`
	// JSONSchema is the json schema of the chat completion response
	// format.
	JSONSchema *ChatCompletionResponseFormatJSONSchema `json:"json_schema,omitempty"`
}

ChatCompletionResponseFormat is the chat completion response format.

type ChatCompletionResponseFormatJSONSchema 

type ChatCompletionResponseFormatJSONSchema struct {
	// Name is the name of the chat completion response format json
	// schema.
	//
	// it is used to further identify the schema in the response.
	Name string `json:"name"`
	// Description is the description of the chat completion
	// response format json schema.
	Description string `json:"description,omitempty"`
	// Schema is the schema of the chat completion response format
	// json schema.
	Schema schema.Schema `json:"schema"`
	// Strict determines whether to enforce the schema upon the
	// generated content.
	Strict bool `json:"strict"`
}

ChatCompletionResponseFormatJSONSchema is the chat completion response format json schema.

type ChatCompletionStream 

type ChatCompletionStream struct {
	*streams.StreamReader[*ChatCompletionStreamResponse]
}

ChatCompletionStream is a stream of ChatCompletionStreamResponse.

type ChatCompletionStreamChoice 

type ChatCompletionStreamChoice struct {
	// Index is the index of the choice.
	Index int `json:"index"`
	// Delta is the delta of the choice.
	Delta ChatCompletionStreamChoiceDelta `json:"delta"`
	// FinishReason is the finish reason of the choice.
	FinishReason FinishReason `json:"finish_reason"`
}

ChatCompletionStreamChoice represents a response structure for chat completion API.

type ChatCompletionStreamChoiceDelta 

type ChatCompletionStreamChoiceDelta struct {
	// Content is the content of the response.
	Content string `json:"content,omitempty"`
	// Role is the role of the creator of the completion.
	Role string `json:"role,omitempty"`
	// FunctionCall is the function call of the response.
	FunctionCall *tools.FunctionCall `json:"function_call,omitempty"`
	// ToolCalls are the tool calls of the response.
	ToolCalls []tools.ToolCall `json:"tool_calls,omitempty"`
}

ChatCompletionStreamChoiceDelta represents a response structure for chat completion API.

type ChatCompletionStreamResponse 

type ChatCompletionStreamResponse struct {
	// ID is the identifier for the chat completion stream response.
	ID string `json:"id"`
	// Object is the object type of the chat completion stream
	// response.
	Object string `json:"object"`
	// Created is the creation time of the chat completion stream
	// response.
	Created int64 `json:"created"`
	// Model is the model used for the chat completion stream
	// response.
	Model ChatModel `json:"model"`
	// Choices is the choices for the chat completion stream
	// response.
	Choices []ChatCompletionStreamChoice `json:"choices"`
	// SystemFingerprint is the system fingerprint for the chat
	// completion stream response.
	SystemFingerprint string `json:"system_fingerprint"`
	// PromptAnnotations is the prompt annotations for the chat
	// completion stream response.
	PromptAnnotations []PromptAnnotation `json:"prompt_annotations,omitempty"`
	// PromptFilterResults is the prompt filter results for the chat
	// completion stream response.
	PromptFilterResults []struct {
		Index int `json:"index"`
	} `json:"prompt_filter_results,omitempty"`
	// Usage is an optional field that will only be present when you
	// set stream_options: {"include_usage": true} in your request.
	//
	// When present, it contains a null value except for the last
	// chunk which contains the token usage statistics for the
	// entire request.
	Usage *Usage `json:"usage,omitempty"`
}

ChatCompletionStreamResponse represents a response structure for chat completion API.

type ChatMessageImageURL 

type ChatMessageImageURL struct {
	// URL is the url of the image.
	URL string `json:"url,omitempty"`
	// Detail is the detail of the image url.
	Detail ImageURLDetail `json:"detail,omitempty"`
}

ChatMessageImageURL represents the chat message image url.

type ChatMessagePart 

type ChatMessagePart struct {
	// Text is the text of the chat message part.
	Text string `json:"text,omitempty"`
	// Type is the type of the chat message part.
	Type ChatMessagePartType `json:"type,omitempty"`
	// ImageURL is the image url of the chat message part.
	ImageURL *ChatMessageImageURL `json:"image_url,omitempty"`
}

ChatMessagePart represents the chat message part of a chat completion message.

type ChatMessagePartType 

type ChatMessagePartType string

ChatMessagePartType is the chat message part type.

string

type ChatModel 

type ChatModel Model

ChatModel is the type for chat models present on the groq api.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is a Groq api client.

func NewClient 

func NewClient(groqAPIKey string, opts ...Opts) (*Client, error)

NewClient creates a new Groq client.

func (*Client) ChatCompletion 

func (c *Client) ChatCompletion(
	ctx context.Context,
	request ChatCompletionRequest,
) (response ChatCompletionResponse, err error)

ChatCompletion method is an API call to create a chat completion.

func (*Client) ChatCompletionJSON 

func (c *Client) ChatCompletionJSON(
	ctx context.Context,
	request ChatCompletionRequest,
	output any,
) (err error)

ChatCompletionJSON method is an API call to create a chat completion w/ object output.

func (*Client) ChatCompletionStream 

func (c *Client) ChatCompletionStream(
	ctx context.Context,
	request ChatCompletionRequest,
) (stream *ChatCompletionStream, err error)

ChatCompletionStream method is an API call to create a chat completion w/ streaming support.

func (*Client) Embedding added in v0.0.3 

func (c *Client) Embedding(ctx context.Context, request EmbeddingRequest) (EmbeddingResponse, error)

Embedding method is an API call to create an embedding.

func (*Client) Moderate 

func (c *Client) Moderate(
	ctx context.Context,
	messages []ChatCompletionMessage,
	model ModerationModel,
) (response []Moderation, err error)

Moderate performs a moderation api call over a string. Input can be an array or slice but a string will reduce the complexity.

func (*Client) Transcribe 

func (c *Client) Transcribe(
	ctx context.Context,
	request AudioRequest,
) (AudioResponse, error)

Transcribe calls the transcriptions endpoint with the given request.

Returns transcribed text in the response_format specified in the request.

func (*Client) Translate 

func (c *Client) Translate(
	ctx context.Context,
	request AudioRequest,
) (AudioResponse, error)

Translate calls the translations endpoint with the given request.

Returns the translated text in the response_format specified in the request.

type CreateEmbeddingResponseObject added in v0.0.3 

type CreateEmbeddingResponseObject string

CreateEmbeddingResponseObject is the object type for the create embedding response. It's value is always "list". string 

const (
	// CreateEmbeddingResponseObjectList is the list object type.
	CreateEmbeddingResponseObjectList CreateEmbeddingResponseObject = "list"
)

type EmbeddingModel added in v0.0.3 

type EmbeddingModel Model

EmbeddingModel is the type for embedding models present on the groq api.

type EmbeddingObject added in v0.0.3 

type EmbeddingObject string

EmbeddingObject is the object type for the embedding response. It's value is always "embedding". string 

const (
	// EmbeddingObjectEmbedding is the embedding object type.
	EmbeddingObjectEmbedding EmbeddingObject = "embedding"
)

type EmbeddingRequest added in v0.0.3 

type EmbeddingRequest struct {
	// Input text to embed, encoded as a string or array of tokens. To embed multiple
	// inputs in a single request, pass an array of strings or array of token arrays.
	// The input must not exceed the max input tokens for the model (8192 tokens for
	// `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
	// dimensions or less.
	// [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
	// for counting tokens.
	Input string `json:"input,required"`
	// ID of the model to use. You can use the
	// [List models](https://platform.openai.com/docs/api-reference/models/list) API to
	// see all of your available models, or see our
	// [Model overview](https://platform.openai.com/docs/models) for descriptions of
	// them.
	Model EmbeddingModel `json:"model,required"`
	// The format to return the embeddings in. Can be either `float` or
	// [`base64`](https://pypi.org/project/pybase64/).
	EncodingFormat EmbeddingsFormat `json:"encoding_format"`
	// The number of dimensions the resulting output embeddings should have. Only
	// supported in `text-embedding-3` and later models.
	Dimensions int64 `json:"dimensions"`
	// A unique identifier representing your end-user, which can help OpenAI to monitor
	// and detect abuse.
	// [Learn more](https://platform.openai.com/docs/guides/safety-best-practices#end-user-ids).
	User string `json:"user"`
}

EmbeddingRequest represents a request structure for embedding API. TODO: update reference to the official API.

type EmbeddingResponse added in v0.0.3 

type EmbeddingResponse struct {
	Object string `json:"object"`
	Data   []struct {
		Object    string    `json:"object"`
		Index     int       `json:"index"`
		Embedding []float64 `json:"embedding"`
	} `json:"data"`
	Model string `json:"model"`
	Usage struct {
		PromptTokens int `json:"prompt_tokens"`
		TotalTokens  int `json:"total_tokens"`
	} `json:"usage"`
	Header http.Header `json:"-"`
}

EmbeddingResponse represents the response structure for embedding API.

func (EmbeddingResponse) SetHeader added in v0.0.3 

func (r EmbeddingResponse) SetHeader(header http.Header)

SetHeader sets the header of the embedding response.

type EmbeddingsFormat added in v0.0.3 

type EmbeddingsFormat string

EmbeddingsFormat is the format for the embedding response. string 

const (
	// EncodingFormatFloat is the float encoding format.
	EncodingFormatFloat EmbeddingsFormat = "float"
	// EncodingFormatBase64 is the base64 encoding format.
	EncodingFormatBase64 EmbeddingsFormat = "base64"
)

type FinishReason 

type FinishReason string

FinishReason is the finish reason.

string 

const (
	// ReasonStop is the stop finish reason for a chat completion.
	ReasonStop FinishReason = "stop"
	// ReasonLength is the length finish reason for a chat completion.
	ReasonLength FinishReason = "length"
	// ReasonFunctionCall is the function call finish reason for a chat
	// completion.
	ReasonFunctionCall FinishReason = "function_call"
	// ReasonToolCalls is the tool calls finish reason for a chat
	// completion.
	ReasonToolCalls FinishReason = "tool_calls"
	// ReasonContentFilter is the content filter finish reason for a chat
	// completion.
	ReasonContentFilter FinishReason = "content_filter"
	// ReasonNull is the null finish reason for a chat completion.
	ReasonNull FinishReason = "null"
)

func (FinishReason) MarshalJSON 

func (r FinishReason) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

type Format 

type Format string

Format is the format of a response. string 

const (
	// FormatText is the text format. It is the default format of a
	// response.
	FormatText Format = "text"
	// FormatJSON is the JSON format. There is no support for streaming with
	// JSON format selected.
	FormatJSON Format = "json"
	// FormatSRT is the SRT format. This is a text format that is only
	// supported for the transcription API.
	// SRT format selected.
	FormatSRT Format = "srt"
	// FormatVTT is the VTT format. This is a text format that is only
	// supported for the transcription API.
	FormatVTT Format = "vtt"
	// FormatVerboseJSON is the verbose JSON format. This is a JSON format
	// that is only supported for the transcription API.
	FormatVerboseJSON Format = "verbose_json"
	// FormatJSONObject is the json object chat
	// completion response format type.
	FormatJSONObject Format = "json_object"
	// FormatJSONSchema is the json schema chat
	// completion response format type.
	FormatJSONSchema Format = "json_schema"
)

type ImageURLDetail 

type ImageURLDetail string

ImageURLDetail is the detail of the image at the URL.

string

type LogProbs 

type LogProbs struct {
	// Content is a list of message content tokens with log
	// probability information.
	Content []struct {
		// Token is the token of the log prob.
		Token string `json:"token"`
		// LogProb is the log prob of the log prob.
		LogProb float64 `json:"logprob"`
		// Omitting the field if it is null
		Bytes []byte `json:"bytes,omitempty"`
		// TopLogProbs is a list of the most likely tokens and
		// their log probability, at this token position. In
		// rare cases, there may be fewer than the number of
		// requested top_logprobs returned.
		TopLogProbs []TopLogProbs `json:"top_logprobs"`
	} `json:"content"`
}

LogProbs is the top-level structure containing the log probability information.

type Model 

type Model string

Model is a ai model accessible through the groq api.

type Moderation 

type Moderation string

Moderation is a category of harmful content that can be used to categorize a chat history using Llama Guard 3.

[Llama Guard 3](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama-guard-3/)

Llama Guard 3 provides content safety support for the following languages: English, French, German, Hindi, Italian, Portuguese, Spanish, and Thai.

string 

const (
	// ModerationViolentCrimes (S1) is the violent crimes category.
	//
	// Responses that enable, encourage, or endorse the commission of
	// violent crimes, including: (1) unlawful violence toward people
	// (ex: terrorism, genocide, murder, hate-motivated violent crimes,
	// child abuse, assault, battery, kidnapping) and (2) unlawful violence
	// toward animals (ex: animal abuse)
	ModerationViolentCrimes Moderation = "violent_crimes"
	// ModerationNonviolentCrimes (S2) is the non-violent crimes category.
	ModerationNonviolentCrimes Moderation = "non_violent_crimes"
	// ModerationSexRelatedCrimes (S3) is the sex-related crimes category.
	ModerationSexRelatedCrimes Moderation = "sex_related_crimes"
	// ModerationChildSexualExploitation (S4) is the child sexual exploitation
	// category.
	//
	// Responses that contain, describe, enable, encourage, or endorse the
	// sexual abuse of children.
	ModerationChildSexualExploitation Moderation = "child_sexual_exploitation"
	// ModerationDefamation (S5) is the defamation category.
	//
	// Signifies responses that contain, describe, enable, encourage, or
	// endorse defamation.
	ModerationDefamation Moderation = "defamation"
	// ModerationSpecializedAdvice (S6) is the specialized advice category.
	//
	// Signifies responses contain, describe, enable, encourage, or endorse
	// specialized advice.
	ModerationSpecializedAdvice Moderation = "specialized_advice"
	// ModerationPrivacy (S7) is the privacy category.
	//
	// Responses contain, describe, enable, encourage, or endorse privacy.
	ModerationPrivacy Moderation = "privacy"
	// ModerationIntellectualProperty (S8) is the intellectual property
	// category. Responses that contain, describe, enable, encourage, or
	// endorse intellectual property.
	ModerationIntellectualProperty Moderation = "intellectual_property"
	// ModerationIndiscriminateWeapons (S9) is the indiscriminate weapons
	// category.
	//
	// Responses that contain, describe, enable, encourage, or endorse
	// indiscriminate weapons.
	ModerationIndiscriminateWeapons Moderation = "indiscriminate_weapons"
	// ModerationHate (S10) is the hate category.
	//
	// Responses contain, describe, enable, encourage, or endorse hate.
	ModerationHate Moderation = "hate"
	// ModerationSuicideOrSelfHarm (S11) is the suicide/self-harm category.
	//
	// Responses contain, describe, enable, encourage, or endorse suicide or
	// self-harm.
	ModerationSuicideOrSelfHarm Moderation = "suicide_and_self_harm"
	// ModerationSexualContent (S12) is the sexual content category.
	//
	// Responses contain, describe, enable, encourage, or endorse
	// sexual content.
	ModerationSexualContent Moderation = "sexual_content"
	// ModerationElections (S13) is the elections category.
	//
	// Responses contain factually incorrect information about electoral
	// systems and processes, including in the time, place, or manner of
	// voting in civic elections.
	ModerationElections Moderation = "elections"
	// ModerationCodeInterpreterAbuse (S14) is the code interpreter abuse
	// category.
	//
	// Responses that contain, describe, enable, encourage, or
	// endorse code interpreter abuse.
	ModerationCodeInterpreterAbuse Moderation = "code_interpreter_abuse"
)

type ModerationModel 

type ModerationModel Model

ModerationModel is the type for moderation models present on the groq api.

type Opts 

type Opts func(*Client)

Opts is a function that sets options for a Groq client.

func WithBaseURL 

func WithBaseURL(baseURL string) Opts

WithBaseURL sets the base URL for the Groq client.

func WithClient 

func WithClient(client *http.Client) Opts

WithClient sets the client for the Groq client.

func WithLogger 

func WithLogger(logger *slog.Logger) Opts

WithLogger sets the logger for the Groq client.

type PromptAnnotation 

type PromptAnnotation struct {
	PromptIndex int `json:"prompt_index,omitempty"`
}

PromptAnnotation represents the prompt annotation.

type Role 

type Role string

Role is the role of the chat completion message.

string

type Segments 

type Segments []struct {
	// ID is the ID of the segment.
	ID int `json:"id"`
	// Seek is the seek of the segment.
	Seek int `json:"seek"`
	// Start is the start of the segment.
	Start float64 `json:"start"`
	// End is the end of the segment.
	End float64 `json:"end"`
	// Text is the text of the segment.
	Text string `json:"text"`
	// Tokens is the tokens of the segment.
	Tokens []int `json:"tokens"`
	// Temperature is the temperature of the segment.
	Temperature float64 `json:"temperature"`
	// AvgLogprob is the avg log prob of the segment.
	AvgLogprob float64 `json:"avg_logprob"`
	// CompressionRatio is the compression ratio of the segment.
	CompressionRatio float64 `json:"compression_ratio"`
	// NoSpeechProb is the no speech prob of the segment.
	NoSpeechProb float64 `json:"no_speech_prob"`
	// Transient is the transient of the segment.
	Transient bool `json:"transient"`
}

Segments is the segments of the response.

type StreamOptions 

type StreamOptions struct {
	// IncludeUsage is the include usage option of the stream
	// options.
	//
	// If set, an additional chunk will be streamed before the data:
	// [DONE] message.
	// The usage field on this chunk shows the token usage
	// statistics for the entire request, and the choices field will
	// always be an empty array.
	//
	// All other chunks will also include a usage field, but with a
	// null value.
	IncludeUsage bool `json:"include_usage,omitempty"`
}

StreamOptions represents the stream options.

type TopLogProbs 

type TopLogProbs struct {
	// Token is the token of the top log probs.
	Token string `json:"token"`
	// LogProb is the log prob of the top log probs.
	LogProb float64 `json:"logprob"`
	// Bytes is the bytes of the top log probs.
	Bytes []byte `json:"bytes,omitempty"`
}

TopLogProbs represents the top log probs.

type TranscriptionTimestampGranularity 

type TranscriptionTimestampGranularity string

TranscriptionTimestampGranularity is the timestamp granularity for the transcription.

string 

const (
	// TranscriptionTimestampGranularityWord is the word timestamp
	// granularity.
	TranscriptionTimestampGranularityWord TranscriptionTimestampGranularity = "word"
	// TranscriptionTimestampGranularitySegment is the segment timestamp
	// granularity.
	TranscriptionTimestampGranularitySegment TranscriptionTimestampGranularity = "segment"
)

type Usage 

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

Usage Represents the total token usage per request to pegwings.

type Words 

type Words []struct {
	// Word is the textual representation of a word in the audio
	// response.
	Word string `json:"word"`
	// Start is the start of the words in seconds.
	Start float64 `json:"start"`
	// End is the end of the words in seconds.
	End float64 `json:"end"`
}

Words is the words of the audio response.

Source Files

View all Source files

Directories

Path Synopsis
examples
audio-house-translation command
Package main is an example of using the pegwings-go library to create a transcription/translation using the whisper-large-v3 model.
 Package main is an example of using the pegwings-go library to create a transcription/translation using the whisper-large-v3 model.
audio-lex-fridman command
Package main is an example of using pegwings-go to create a transcription using the whisper model.
 Package main is an example of using pegwings-go to create a transcription using the whisper model.
chat-terminal command
Package main demonstrates how to use pegwings-go to create a chat application using the groq api accessable through the terminal.
 Package main demonstrates how to use pegwings-go to create a chat application using the groq api accessable through the terminal.
composio-github-star command
Package main is an example of using the composio client.
 Package main is an example of using the composio client.
e2b-go-project command
Package main shows an example of using the e2b extension.
 Package main shows an example of using the e2b extension.
json-chat command
Package main demonstrates an example application of pegwings-go.
 Package main demonstrates an example application of pegwings-go.
llama-blind command
Package main demonstrates an example application of pegwings-go.
 Package main demonstrates an example application of pegwings-go.
moderation command
Package main is an example of using pegwings-go to create a chat moderation using the llama-3BGuard model.
 Package main is an example of using pegwings-go to create a chat moderation using the llama-3BGuard model.
toolhouse-python-code-interpreter command
Package main shows an example of using the toolhouse go package.
 Package main shows an example of using the toolhouse go package.
vhdl-documentor-json command
Package main is an example of using pegwings-go to create a chat completion using the llama-70B-tools-preview model to create headers for vhdl projects.
 Package main is an example of using pegwings-go to create a chat completion using the llama-70B-tools-preview model to create headers for vhdl projects.
extensions
composio
Package composio provides a composio client for pegwings-go.
 Package composio provides a composio client for pegwings-go.
e2b
Package e2b provides an e2b client for pegwings-go.
 Package e2b provides an e2b client for pegwings-go.
jigsawstack
Package jigsawstack provides a JigsawStack extension for pegwings-go.
 Package jigsawstack provides a JigsawStack extension for pegwings-go.
toolhouse
Package toolhouse provides a Toolhouse extension for pegwings-go.
 Package toolhouse provides a Toolhouse extension for pegwings-go.
pkg
builders
Package builders provides builders for HTTP requests and forms.
 Package builders provides builders for HTTP requests and forms.
list
Package list containes the implementation a doubly linked list.
 Package list containes the implementation a doubly linked list.
omap
Package omap provides an ordered map implementation.
 Package omap provides an ordered map implementation.
pegwingerrs
Package pegwingerrs provides error types for the pegwings-go library.
 Package pegwingerrs provides error types for the pegwings-go library.
schema
Package schema provides an interface for working with JSON Schemas.
 Package schema provides an interface for working with JSON Schemas.
streams
Package streams contains the interfaces for pegwings-go streamed responses.
 Package streams contains the interfaces for pegwings-go streamed responses.
test
Package test contains test helpers.
 Package test contains test helpers.
tools
Package tools contains the interfaces for pegwings-go tooling usable by llms.
 Package tools contains the interfaces for pegwings-go tooling usable by llms.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.