groq

package module
v0.9.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 26, 2024 License: MIT Imports: 15 Imported by: 4

README

groq-go

Go Reference Go Report Card Coverage Status PhormAI

Powered by Groq for fast inference.

Features

  • Supports all models from Groq in a type-safe way.
  • Supports streaming.
  • Supports moderation.
  • Supports audio transcription.
  • Supports audio translation.
  • Supports Tool Use.
  • Supports Function Calling.
  • JSON Schema Generation from structs.
  • Supports Toolhouse function calling. Extention
  • Supports E2b function calling. Extention

Installation

go get github.com/conneroisu/groq-go

Examples

For introductory examples, see the examples directory.

External Repositories using groq-go:

Development

To run the tests:

Make sure you have a groq key set in the environment variable GROQ_KEY.

task test

To run the linter:

task lint

If you fork the repository, you should set up the following environment variables in ci/cd:

export E2B_API_KEY=your-e2b-api-key
export GROQ_KEY=your-groq-key
export TOOLHOUSE_API_KEY=your-toolhouse-api-key

Documentation

The following documentation is generated from the source code using gomarkdoc.

groq

import "github.com/conneroisu/groq-go"

Package groq provides a unofficial client for the Groq API.

With specially designed hardware, the Groq API is a super fast way to query open source llms.

API Documentation: https://console.groq.com/docs/quickstart

Code generated by groq-modeler DO NOT EDIT.

Created at: 2024-10-26 10:01:35

groq-modeler Version 1.1.2

Index

Constants

const (
    AudioResponseFormatJSON        AudioResponseFormat = "json"         // AudioResponseFormatJSON is the JSON format of some audio.
    AudioResponseFormatText        AudioResponseFormat = "text"         // AudioResponseFormatText is the text format of some audio.
    AudioResponseFormatSRT         AudioResponseFormat = "srt"          // AudioResponseFormatSRT is the SRT format of some audio.
    AudioResponseFormatVerboseJSON AudioResponseFormat = "verbose_json" // AudioResponseFormatVerboseJSON is the verbose JSON format of some audio.
    AudioResponseFormatVTT         AudioResponseFormat = "vtt"          // AudioResponseFormatVTT is the VTT format of some audio.

    TranscriptionTimestampGranularityWord    TranscriptionTimestampGranularity = "word"    // TranscriptionTimestampGranularityWord is the word timestamp granularity.
    TranscriptionTimestampGranularitySegment TranscriptionTimestampGranularity = "segment" // TranscriptionTimestampGranularitySegment is the segment timestamp granularity.
)

const (
    ChatMessageRoleSystem                      Role                             = "system"         // ChatMessageRoleSystem is the system chat message role.
    ChatMessageRoleUser                        Role                             = "user"           // ChatMessageRoleUser is the user chat message role.
    ChatMessageRoleAssistant                   Role                             = "assistant"      // ChatMessageRoleAssistant is the assistant chat message role.
    ChatMessageRoleFunction                    Role                             = "function"       // ChatMessageRoleFunction is the function chat message role.
    ChatMessageRoleTool                        Role                             = "tool"           // ChatMessageRoleTool is the tool chat message role.
    ImageURLDetailHigh                         ImageURLDetail                   = "high"           // ImageURLDetailHigh is the high image url detail.
    ImageURLDetailLow                          ImageURLDetail                   = "low"            // ImageURLDetailLow is the low image url detail.
    ImageURLDetailAuto                         ImageURLDetail                   = "auto"           // ImageURLDetailAuto is the auto image url detail.
    ChatMessagePartTypeText                    ChatMessagePartType              = "text"           // ChatMessagePartTypeText is the text chat message part type.
    ChatMessagePartTypeImageURL                ChatMessagePartType              = "image_url"      // ChatMessagePartTypeImageURL is the image url chat message part type.
    ChatCompletionResponseFormatTypeJSONObject ChatCompletionResponseFormatType = "json_object"    // ChatCompletionResponseFormatTypeJSONObject is the json object chat completion response format type.
    ChatCompletionResponseFormatTypeJSONSchema ChatCompletionResponseFormatType = "json_schema"    // ChatCompletionResponseFormatTypeJSONSchema is the json schema chat completion response format type.
    ChatCompletionResponseFormatTypeText       ChatCompletionResponseFormatType = "text"           // ChatCompletionResponseFormatTypeText is the text chat completion response format type.
    FinishReasonStop                           FinishReason                     = "stop"           // FinishReasonStop is the stop finish reason.
    FinishReasonLength                         FinishReason                     = "length"         // FinishReasonLength is the length finish reason.
    FinishReasonFunctionCall                   FinishReason                     = "function_call"  // FinishReasonFunctionCall is the function call finish reason.
    FinishReasonToolCalls                      FinishReason                     = "tool_calls"     // FinishReasonToolCalls is the tool calls finish reason.
    FinishReasonContentFilter                  FinishReason                     = "content_filter" // FinishReasonContentFilter is the content filter finish reason.
    FinishReasonNull                           FinishReason                     = "null"           // FinishReasonNull is the null finish reason.
)

Variables

var (
    // ModelGemma29BIt is an AI text chat model.
    //
    // It is created/provided by Google.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelGemma29BIt ChatModel = "gemma2-9b-it"
    // ModelGemma7BIt is an AI text chat model.
    //
    // It is created/provided by Google.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelGemma7BIt ChatModel = "gemma-7b-it"
    // ModelLlama3170BVersatile is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 32768 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama3170BVersatile ChatModel = "llama-3.1-70b-versatile"
    // ModelLlama318BInstant is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 131072 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama318BInstant ChatModel = "llama-3.1-8b-instant"
    // ModelLlama3211BTextPreview is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama3211BTextPreview ChatModel = "llama-3.2-11b-text-preview"
    // ModelLlama3211BVisionPreview is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama3211BVisionPreview ChatModel = "llama-3.2-11b-vision-preview"
    // ModelLlama321BPreview is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama321BPreview ChatModel = "llama-3.2-1b-preview"
    // ModelLlama323BPreview is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama323BPreview ChatModel = "llama-3.2-3b-preview"
    // ModelLlama3290BTextPreview is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama3290BTextPreview ChatModel = "llama-3.2-90b-text-preview"
    // ModelLlama3290BVisionPreview is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama3290BVisionPreview ChatModel = "llama-3.2-90b-vision-preview"
    // ModelLlama370B8192 is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama370B8192 ChatModel = "llama3-70b-8192"
    // ModelLlama38B8192 is an AI text chat model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama38B8192 ChatModel = "llama3-8b-8192"
    // ModelLlama3Groq70B8192ToolUsePreview is an AI text chat model.
    //
    // It is created/provided by Groq.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama3Groq70B8192ToolUsePreview ChatModel = "llama3-groq-70b-8192-tool-use-preview"
    // ModelLlama3Groq8B8192ToolUsePreview is an AI text chat model.
    //
    // It is created/provided by Groq.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlama3Groq8B8192ToolUsePreview ChatModel = "llama3-groq-8b-8192-tool-use-preview"
    // ModelLlavaV157B4096Preview is an AI text chat model.
    //
    // It is created/provided by Other.
    //
    // It has 4096 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelLlavaV157B4096Preview ChatModel = "llava-v1.5-7b-4096-preview"
    // ModelMixtral8X7B32768 is an AI text chat model.
    //
    // It is created/provided by Mistral AI.
    //
    // It has 32768 context window.
    //
    // It can be used with the following client methods:
    //	- CreateChatCompletion
    // 	- CreateChatCompletionStream
    // 	- CreateChatCompletionJSON
    ModelMixtral8X7B32768 ChatModel = "mixtral-8x7b-32768"
    // ModelWhisperLargeV3 is an AI audio transcription model.
    //
    // It is created/provided by OpenAI.
    //
    // It has 448 context window.
    //
    // It can be used with the following client methods:
    //	- CreateTranscription
    // 	- CreateTranslation
    ModelWhisperLargeV3 AudioModel = "whisper-large-v3"
    // ModelLlamaGuard38B is an AI moderation model.
    //
    // It is created/provided by Meta.
    //
    // It has 8192 context window.
    //
    // It can be used with the following client methods:
    //	- Moderate
    ModelLlamaGuard38B ModerationModel = "llama-guard-3-8b"
)

func AudioMultipartForm

func AudioMultipartForm(request AudioRequest, b builders.FormBuilder) error

AudioMultipartForm creates a form with audio file contents and the name of the model to use for audio processing.

func CreateFileField

func CreateFileField(request AudioRequest, b builders.FormBuilder) (err error)

CreateFileField creates the "file" form field from either an existing file or by using the reader.

type APIError

APIError provides error information returned by the Groq API.

type APIError struct {
    Code           any     `json:"code,omitempty"`  // Code is the code of the error.
    Message        string  `json:"message"`         // Message is the message of the error.
    Param          *string `json:"param,omitempty"` // Param is the param of the error.
    Type           string  `json:"type"`            // Type is the type of the error.
    HTTPStatusCode int     `json:"-"`               // HTTPStatusCode is the status code of the error.
}

func (*APIError) Error
func (e *APIError) Error() string

Error method implements the error interface on APIError.

func (*APIError) UnmarshalJSON
func (e *APIError) UnmarshalJSON(data []byte) (err error)

UnmarshalJSON implements the json.Unmarshaler interface.

type AudioModel

AudioModel is the type for audio models present on the groq api.

type AudioModel model

type AudioRequest

AudioRequest represents a request structure for audio API.

type AudioRequest struct {
    Model       AudioModel          // Model is the model to use for the transcription.
    FilePath    string              // FilePath is either an existing file in your filesystem or a filename representing the contents of Reader.
    Reader      io.Reader           // Reader is an optional io.Reader when you do not want to use an existing file.
    Prompt      string              // Prompt is the prompt for the transcription.
    Temperature float32             // Temperature is the temperature for the transcription.
    Language    string              // Language is the language for the transcription. Only for transcription.
    Format      AudioResponseFormat // Format is the format for the response.
}

type AudioResponse

AudioResponse represents a response structure for audio API.

type AudioResponse struct {
    Task     string   `json:"task"`     // Task is the task of the response.
    Language string   `json:"language"` // Language is the language of the response.
    Duration float64  `json:"duration"` // Duration is the duration of the response.
    Segments Segments `json:"segments"` // Segments is the segments of the response.
    Words    Words    `json:"words"`    // Words is the words of the response.
    Text     string   `json:"text"`     // Text is the text of the response.

    Header http.Header // Header is the header of the response.
}

func (*AudioResponse) SetHeader
func (r *AudioResponse) SetHeader(header http.Header)

SetHeader sets the header of the response.

type AudioResponseFormat

AudioResponseFormat is the response format for the audio API.

Response formatted using AudioResponseFormatJSON by default.

string

type AudioResponseFormat string

type ChatCompletionChoice

ChatCompletionChoice represents the chat completion choice.

type ChatCompletionChoice struct {
    Index int `json:"index"` // Index is the index of the choice.
    // Message is the chat completion message of the choice.
    Message ChatCompletionMessage `json:"message"` // Message is the chat completion message of the choice.
    // FinishReason is the finish reason of the choice.
    //
    // stop: API returned complete message,
    // or a message terminated by one of the stop sequences provided via the stop parameter
    // length: Incomplete model output due to max_tokens parameter or token limit
    // function_call: The model decided to call a function
    // content_filter: Omitted content due to a flag from our content filters
    // null: API response still in progress or incomplete
    FinishReason FinishReason `json:"finish_reason"` // FinishReason is the finish reason of the choice.
    // LogProbs is the log probs of the choice.
    //
    // This is basically the probability of the model choosing the token.
    LogProbs *LogProbs `json:"logprobs,omitempty"` // LogProbs is the log probs of the choice.
}

type ChatCompletionMessage

ChatCompletionMessage represents the chat completion message.

type ChatCompletionMessage struct {
    Name         string              `json:"name"`                    // Name is the name of the chat completion message.
    Role         Role                `json:"role"`                    // Role is the role of the chat completion message.
    Content      string              `json:"content"`                 // Content is the content of the chat completion message.
    MultiContent []ChatMessagePart   `json:"-"`                       // MultiContent is the multi content of the chat completion message.
    FunctionCall *tools.FunctionCall `json:"function_call,omitempty"` // FunctionCall setting for Role=assistant prompts this may be set to the function call generated by the model.
    ToolCalls    []tools.ToolCall    `json:"tool_calls,omitempty"`    // ToolCalls setting for Role=assistant prompts this may be set to the tool calls generated by the model, such as function calls.
    ToolCallID   string              `json:"tool_call_id,omitempty"`  // ToolCallID is setting for Role=tool prompts this should be set to the ID given in the assistant's prior request to call a tool.
}

func (ChatCompletionMessage) MarshalJSON
func (m ChatCompletionMessage) MarshalJSON() ([]byte, error)

MarshalJSON method implements the json.Marshaler interface.

func (*ChatCompletionMessage) UnmarshalJSON
func (m *ChatCompletionMessage) UnmarshalJSON(bs []byte) (err error)

UnmarshalJSON method implements the json.Unmarshaler interface.

type ChatCompletionRequest

ChatCompletionRequest represents a request structure for the chat completion API.

type ChatCompletionRequest struct {
    Model             ChatModel                     `json:"model"`                         // Model is the model of the chat completion request.
    Messages          []ChatCompletionMessage       `json:"messages"`                      // Messages is the messages of the chat completion request. These act as the prompt for the model.
    MaxTokens         int                           `json:"max_tokens,omitempty"`          // MaxTokens is the max tokens of the chat completion request.
    Temperature       float32                       `json:"temperature,omitempty"`         // Temperature is the temperature of the chat completion request.
    TopP              float32                       `json:"top_p,omitempty"`               // TopP is the top p of the chat completion request.
    N                 int                           `json:"n,omitempty"`                   // N is the n of the chat completion request.
    Stream            bool                          `json:"stream,omitempty"`              // Stream is the stream of the chat completion request.
    Stop              []string                      `json:"stop,omitempty"`                // Stop is the stop of the chat completion request.
    PresencePenalty   float32                       `json:"presence_penalty,omitempty"`    // PresencePenalty is the presence penalty of the chat completion request.
    ResponseFormat    *ChatCompletionResponseFormat `json:"response_format,omitempty"`     // ResponseFormat is the response format of the chat completion request.
    Seed              *int                          `json:"seed,omitempty"`                // Seed is the seed of the chat completion request.
    FrequencyPenalty  float32                       `json:"frequency_penalty,omitempty"`   // FrequencyPenalty is the frequency penalty of the chat completion request.
    LogitBias         map[string]int                `json:"logit_bias,omitempty"`          // LogitBias is must be a token id string (specified by their token ID in the tokenizer), not a word string. incorrect: `"logit_bias":{ "You": 6}`, correct: `"logit_bias":{"1639": 6}` refs: https://platform.openai.com/docs/api-reference/chat/create#chat/create-logit_bias
    LogProbs          bool                          `json:"logprobs,omitempty"`            // LogProbs indicates whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message. This option is currently not available on the gpt-4-vision-preview model.
    TopLogProbs       int                           `json:"top_logprobs,omitempty"`        // TopLogProbs is an integer between 0 and 5 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.
    User              string                        `json:"user,omitempty"`                // User is the user of the chat completion request.
    Tools             []tools.Tool                  `json:"tools,omitempty"`               // Tools is the tools of the chat completion request.
    ToolChoice        any                           `json:"tool_choice,omitempty"`         // This can be either a string or an ToolChoice object.
    StreamOptions     *StreamOptions                `json:"stream_options,omitempty"`      // Options for streaming response. Only set this when you set stream: true.
    ParallelToolCalls any                           `json:"parallel_tool_calls,omitempty"` // Disable the default behavior of parallel tool calls by setting it: false.
    RetryDelay        time.Duration                 `json:"-"`                             // RetryDelay is the delay between retries.
}

type ChatCompletionResponse

ChatCompletionResponse represents a response structure for chat completion API.

type ChatCompletionResponse struct {
    ID                string                 `json:"id"`                 // ID is the id of the response.
    Object            string                 `json:"object"`             // Object is the object of the response.
    Created           int64                  `json:"created"`            // Created is the created time of the response.
    Model             ChatModel              `json:"model"`              // Model is the model of the response.
    Choices           []ChatCompletionChoice `json:"choices"`            // Choices is the choices of the response.
    Usage             Usage                  `json:"usage"`              // Usage is the usage of the response.
    SystemFingerprint string                 `json:"system_fingerprint"` // SystemFingerprint is the system fingerprint of the response.
    http.Header                              // Header is the header of the response.
}

func (*ChatCompletionResponse) SetHeader
func (r *ChatCompletionResponse) SetHeader(h http.Header)

SetHeader sets the header of the response.

type ChatCompletionResponseFormat

ChatCompletionResponseFormat is the chat completion response format.

type ChatCompletionResponseFormat struct {
    // Type is the type of the chat completion response format.
    Type ChatCompletionResponseFormatType `json:"type,omitempty"`
    // JSONSchema is the json schema of the chat completion response format.
    JSONSchema *ChatCompletionResponseFormatJSONSchema `json:"json_schema,omitempty"`
}

type ChatCompletionResponseFormatJSONSchema

ChatCompletionResponseFormatJSONSchema is the chat completion response format json schema.

type ChatCompletionResponseFormatJSONSchema struct {
    // Name is the name of the chat completion response format json
    // schema.
    //
    // it is used to further identify the schema in the response.
    Name string `json:"name"`
    // Description is the description of the chat completion response
    // format
    // json schema.
    Description string `json:"description,omitempty"`
    // description of the chat completion response format json schema.
    // Schema is the schema of the chat completion response format json schema.
    Schema schema.Schema `json:"schema"`
    // Strict determines whether to enforce the schema upon the generated
    // content.
    Strict bool `json:"strict"`
}

type ChatCompletionResponseFormatType

ChatCompletionResponseFormatType is the chat completion response format type.

string

type ChatCompletionResponseFormatType string

type ChatCompletionStream

ChatCompletionStream is a stream of ChatCompletionStreamResponse.

Note: Perhaps it is more elegant to abstract Stream using generics.

type ChatCompletionStream struct {
    // contains filtered or unexported fields
}

type ChatCompletionStreamChoice

ChatCompletionStreamChoice represents a response structure for chat completion API.

type ChatCompletionStreamChoice struct {
    Index        int                             `json:"index"`
    Delta        ChatCompletionStreamChoiceDelta `json:"delta"`
    FinishReason FinishReason                    `json:"finish_reason"`
}

type ChatCompletionStreamChoiceDelta

ChatCompletionStreamChoiceDelta represents a response structure for chat completion API.

type ChatCompletionStreamChoiceDelta struct {
    Content      string              `json:"content,omitempty"`
    Role         string              `json:"role,omitempty"`
    FunctionCall *tools.FunctionCall `json:"function_call,omitempty"`
    ToolCalls    []tools.ToolCall    `json:"tool_calls,omitempty"`
}

type ChatCompletionStreamResponse

ChatCompletionStreamResponse represents a response structure for chat completion API.

type ChatCompletionStreamResponse struct {
    ID                  string                       `json:"id"`                           // ID is the identifier for the chat completion stream response.
    Object              string                       `json:"object"`                       // Object is the object type of the chat completion stream response.
    Created             int64                        `json:"created"`                      // Created is the creation time of the chat completion stream response.
    Model               ChatModel                    `json:"model"`                        // Model is the model used for the chat completion stream response.
    Choices             []ChatCompletionStreamChoice `json:"choices"`                      // Choices is the choices for the chat completion stream response.
    SystemFingerprint   string                       `json:"system_fingerprint"`           // SystemFingerprint is the system fingerprint for the chat completion stream response.
    PromptAnnotations   []PromptAnnotation           `json:"prompt_annotations,omitempty"` // PromptAnnotations is the prompt annotations for the chat completion stream response.
    PromptFilterResults []struct {
        Index int `json:"index"`
    }   `json:"prompt_filter_results,omitempty"` // PromptFilterResults is the prompt filter results for the chat completion stream response.
    // Usage is an optional field that will only be present when you set stream_options: {"include_usage": true} in your request.
    //
    // When present, it contains a null value except for the last chunk which contains the token usage statistics
    // for the entire request.
    Usage *Usage `json:"usage,omitempty"`
}

type ChatMessageImageURL

ChatMessageImageURL represents the chat message image url.

type ChatMessageImageURL struct {
    URL    string         `json:"url,omitempty"`    // URL is the url of the image.
    Detail ImageURLDetail `json:"detail,omitempty"` // Detail is the detail of the image url.
}

type ChatMessagePart

ChatMessagePart represents the chat message part of a chat completion message.

type ChatMessagePart struct {
    Text     string               `json:"text,omitempty"`      // Text is the text of the chat message part.
    Type     ChatMessagePartType  `json:"type,omitempty"`      // Type is the type of the chat message part.
    ImageURL *ChatMessageImageURL `json:"image_url,omitempty"` // ImageURL is the image url of the chat message part.
}

type ChatMessagePartType

ChatMessagePartType is the chat message part type.

string

type ChatMessagePartType string

type ChatModel

ChatModel is the type for chat models present on the groq api.

type ChatModel model

type Client

Client is a Groq api client.

type Client struct {
    // contains filtered or unexported fields
}

func NewClient
func NewClient(groqAPIKey string, opts ...Opts) (*Client, error)

NewClient creates a new Groq client.

func (*Client) CreateChatCompletion
func (c *Client) CreateChatCompletion(ctx context.Context, request ChatCompletionRequest) (response ChatCompletionResponse, err error)

CreateChatCompletion method is an API call to create a chat completion.

func (*Client) CreateChatCompletionJSON
func (c *Client) CreateChatCompletionJSON(ctx context.Context, request ChatCompletionRequest, output any) (err error)

CreateChatCompletionJSON method is an API call to create a chat completion w/ object output.

func (*Client) CreateChatCompletionStream
func (c *Client) CreateChatCompletionStream(ctx context.Context, request ChatCompletionRequest) (stream *ChatCompletionStream, err error)

CreateChatCompletionStream method is an API call to create a chat completion w/ streaming support.

If set, tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message.

func (*Client) CreateTranscription
func (c *Client) CreateTranscription(ctx context.Context, request AudioRequest) (AudioResponse, error)

CreateTranscription calls the transcriptions endpoint with the given request.

Returns transcribed text in the response_format specified in the request.

func (*Client) CreateTranslation
func (c *Client) CreateTranslation(ctx context.Context, request AudioRequest) (AudioResponse, error)

CreateTranslation calls the translations endpoint with the given request.

Returns the translated text in the response_format specified in the request.

func (*Client) Moderate
func (c *Client) Moderate(ctx context.Context, request ModerationRequest) (response Moderation, err error)

Moderate performs a moderation api call over a string. Input can be an array or slice but a string will reduce the complexity.

func (*Client) MustCreateChatCompletion
func (c *Client) MustCreateChatCompletion(ctx context.Context, request ChatCompletionRequest) (response ChatCompletionResponse)

MustCreateChatCompletion method is an API call to create a chat completion.

It panics if an error occurs.

type DefaultErrorAccumulator

DefaultErrorAccumulator is a default implementation of ErrorAccumulator

type DefaultErrorAccumulator struct {
    Buffer errorBuffer
}

func (*DefaultErrorAccumulator) Bytes
func (e *DefaultErrorAccumulator) Bytes() (errBytes []byte)

Bytes method returns the bytes of the error accumulator.

func (*DefaultErrorAccumulator) Write
func (e *DefaultErrorAccumulator) Write(p []byte) error

Write method writes bytes to the error accumulator.

type Endpoint

Endpoint is the endpoint for the groq api. string

type Endpoint string

type ErrContentFieldsMisused

ErrContentFieldsMisused is an error that occurs when both Content and MultiContent properties are set.

type ErrContentFieldsMisused struct {
    // contains filtered or unexported fields
}

func (ErrContentFieldsMisused) Error
func (e ErrContentFieldsMisused) Error() string

Error implements the error interface.

type ErrTooManyEmptyStreamMessages

ErrTooManyEmptyStreamMessages is returned when the stream has sent too many empty messages.

type ErrTooManyEmptyStreamMessages struct{}

func (ErrTooManyEmptyStreamMessages) Error
func (e ErrTooManyEmptyStreamMessages) Error() string

Error returns the error message.

type FinishReason

FinishReason is the finish reason. string

type FinishReason string

func (FinishReason) MarshalJSON
func (r FinishReason) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

type Format

Format is the format of a response. string

type Format string

const (
    // FormatText is the text format. It is the default format of a
    // response.
    FormatText Format = "text"
    // FormatJSON is the JSON format. There is no support for streaming with
    // JSON format selected.
    FormatJSON Format = "json"
)

type HarmfulCategory

HarmfulCategory is a category of harmful content that can be used to categorize a chat history using Llama Guard 3.

[Llama Guard 3](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama-guard-3/)

Llama Guard 3 provides content safety support for the following languages: English, French, German, Hindi, Italian, Portuguese, Spanish, and Thai.

string

type HarmfulCategory string

const (
    // CategoryViolentCrimes (S1) is the violent crimes category.
    //
    // Responses that enable, encourage, or endorse the commission of
    // violent crimes, including: (1) unlawful violence toward people
    // (ex: terrorism, genocide, murder, hate-motivated violent crimes,
    // child abuse, assault, battery, kidnapping) and (2) unlawful violence
    // toward animals (ex: animal abuse)
    CategoryViolentCrimes HarmfulCategory = "violent_crimes"
    // CategoryNonviolentCrimes (S2) is the non-violent crimes category.
    CategoryNonviolentCrimes HarmfulCategory = "non_violent_crimes"
    // CategorySexRelatedCrimes (S3) is the sex-related crimes category.
    CategorySexRelatedCrimes HarmfulCategory = "sex_related_crimes"
    // CategoryChildSexualExploitation (S4) is the child sexual exploitation
    // category.
    //
    // Responses that contain, describe, enable, encourage, or endorse the
    // sexual abuse of children.
    CategoryChildSexualExploitation HarmfulCategory = "child_sexual_exploitation"
    // CategoryDefamation (S5) is the defamation category.
    //
    // Signifies responses that contain, describe, enable, encourage, or
    // endorse defamation.
    CategoryDefamation HarmfulCategory = "defamation"
    // CategorySpecializedAdvice (S6) is the specialized advice category.
    //
    // Signifies responses contain, describe, enable, encourage, or endorse specialized advice.
    CategorySpecializedAdvice HarmfulCategory = "specialized_advice"
    // CategoryPrivacy (S7) is the privacy category.
    //
    // Responses contain, describe, enable, encourage, or endorse privacy.
    CategoryPrivacy HarmfulCategory = "privacy"
    // CategoryIntellectualProperty (S8) is the intellectual property
    // category. Responses that contain, describe, enable, encourage, or
    // endorse intellectual property.
    CategoryIntellectualProperty HarmfulCategory = "intellectual_property"
    // CategoryIndiscriminateWeapons (S9) is the indiscriminate weapons
    // category.
    //
    // Responses that contain, describe, enable, encourage, or endorse
    // indiscriminate weapons.
    CategoryIndiscriminateWeapons HarmfulCategory = "indiscriminate_weapons"
    // CategoryHate (S10) is the hate category.
    //
    // Responses contain, describe, enable, encourage, or endorse hate.
    CategoryHate HarmfulCategory = "hate"
    // CategorySuicideAndSelfHarm (S11) is the suicide/self-harm category.
    //
    // Responses contain, describe, enable, encourage, or endorse suicide or self-harm.
    CategorySuicideAndSelfHarm HarmfulCategory = "suicide_and_self_harm"
    // CategorySexualContent (S12) is the sexual content category.
    //
    // Responses contain, describe, enable, encourage, or endorse
    // sexual content.
    CategorySexualContent HarmfulCategory = "sexual_content"
    // CategoryElections (S13) is the elections category.
    //
    // Responses contain factually incorrect information about electoral
    // systems and processes, including in the time, place, or manner of
    // voting in civic elections.
    CategoryElections HarmfulCategory = "elections"
    // CategoryCodeInterpreterAbuse (S14) is the code interpreter abuse
    // category.
    //
    // Responses that contain, describe, enable, encourage, or
    // endorse code interpreter abuse.
    CategoryCodeInterpreterAbuse HarmfulCategory = "code_interpreter_abuse"
)

type ImageURLDetail

ImageURLDetail is the image url detail.

string

type ImageURLDetail string

type LogProbs

LogProbs is the top-level structure containing the log probability information.

type LogProbs struct {
    Content []struct {
        Token       string        `json:"token"`           // Token is the token of the log prob.
        LogProb     float64       `json:"logprob"`         // LogProb is the log prob of the log prob.
        Bytes       []byte        `json:"bytes,omitempty"` // Omitting the field if it is null
        TopLogProbs []TopLogProbs `json:"top_logprobs"`    // TopLogProbs is a list of the most likely tokens and their log probability, at this token position. In rare cases, there may be fewer than the number of requested top_logprobs returned.
    } `json:"content"` // Content is a list of message content tokens with log probability information.
}

type Moderation

Moderation represents one of possible moderation results.

type Moderation struct {
    Categories []HarmfulCategory `json:"categories"` // Categories is the categories of the result.
    Flagged    bool              `json:"flagged"`    // Flagged is the flagged of the result.
}

type ModerationModel

ModerationModel is the type for moderation models present on the groq api.

type ModerationModel model

type ModerationRequest

ModerationRequest represents a request structure for moderation API.

type ModerationRequest struct {
    // Input string `json:"input,omitempty"` // Input is the input text to be moderated.
    Messages []ChatCompletionMessage `json:"messages"`        // Messages is the messages of the chat completion request. These act as the prompt for the model.
    Model    ModerationModel         `json:"model,omitempty"` // Model is the model to use for the moderation.
}

type Opts

Opts is a function that sets options for a Groq client.

type Opts func(*Client)

func WithBaseURL
func WithBaseURL(baseURL string) Opts

WithBaseURL sets the base URL for the Groq client.

func WithClient
func WithClient(client *http.Client) Opts

WithClient sets the client for the Groq client.

func WithLogger
func WithLogger(logger *slog.Logger) Opts

WithLogger sets the logger for the Groq client.

type PromptAnnotation

PromptAnnotation represents the prompt annotation.

type PromptAnnotation struct {
    PromptIndex int `json:"prompt_index,omitempty"`
}

type RateLimitHeaders

RateLimitHeaders struct represents Groq rate limits headers.

type RateLimitHeaders struct {
    LimitRequests     int       `json:"x-ratelimit-limit-requests"`     // LimitRequests is the limit requests of the rate limit headers.
    LimitTokens       int       `json:"x-ratelimit-limit-tokens"`       // LimitTokens is the limit tokens of the rate limit headers.
    RemainingRequests int       `json:"x-ratelimit-remaining-requests"` // RemainingRequests is the remaining requests of the rate limit headers.
    RemainingTokens   int       `json:"x-ratelimit-remaining-tokens"`   // RemainingTokens is the remaining tokens of the rate limit headers.
    ResetRequests     ResetTime `json:"x-ratelimit-reset-requests"`     // ResetRequests is the reset requests of the rate limit headers.
    ResetTokens       ResetTime `json:"x-ratelimit-reset-tokens"`       // ResetTokens is the reset tokens of the rate limit headers.
}

type ResetTime

ResetTime is a time.Time wrapper for the rate limit reset time. string

type ResetTime string

func (ResetTime) String
func (r ResetTime) String() string

String returns the string representation of the ResetTime.

func (ResetTime) Time
func (r ResetTime) Time() time.Time

Time returns the time.Time representation of the ResetTime.

type Role

Role is the role of the chat completion message.

string

type Role string

type Segments

Segments is the segments of the response.

type Segments []struct {
    ID               int     `json:"id"`                // ID is the ID of the segment.
    Seek             int     `json:"seek"`              // Seek is the seek of the segment.
    Start            float64 `json:"start"`             // Start is the start of the segment.
    End              float64 `json:"end"`               // End is the end of the segment.
    Text             string  `json:"text"`              // Text is the text of the segment.
    Tokens           []int   `json:"tokens"`            // Tokens is the tokens of the segment.
    Temperature      float64 `json:"temperature"`       // Temperature is the temperature of the segment.
    AvgLogprob       float64 `json:"avg_logprob"`       // AvgLogprob is the avg log prob of the segment.
    CompressionRatio float64 `json:"compression_ratio"` // CompressionRatio is the compression ratio of the segment.
    NoSpeechProb     float64 `json:"no_speech_prob"`    // NoSpeechProb is the no speech prob of the segment.
    Transient        bool    `json:"transient"`         // Transient is the transient of the segment.
}

type StreamOptions

StreamOptions represents the stream options.

type StreamOptions struct {
    // If set, an additional chunk will be streamed before the data: [DONE] message.
    // The usage field on this chunk shows the token usage statistics for the entire request,
    // and the choices field will always be an empty array.
    // All other chunks will also include a usage field, but with a null value.
    IncludeUsage bool `json:"include_usage,omitempty"`
}

type TopLogProbs

TopLogProbs represents the top log probs.

type TopLogProbs struct {
    Token   string  `json:"token"`           // Token is the token of the top log probs.
    LogProb float64 `json:"logprob"`         // LogProb is the log prob of the top log probs.
    Bytes   []byte  `json:"bytes,omitempty"` // Bytes is the bytes of the top log probs.
}

type TranscriptionTimestampGranularity

TranscriptionTimestampGranularity is the timestamp granularity for the transcription.

string

type TranscriptionTimestampGranularity string

type Usage

Usage Represents the total token usage per request to Groq.

type Usage struct {
    PromptTokens     int `json:"prompt_tokens"`
    CompletionTokens int `json:"completion_tokens"`
    TotalTokens      int `json:"total_tokens"`
}

type Words

Words is the words of the response.

type Words []struct {
    Word  string  `json:"word"`  // Word is the word of the words.
    Start float64 `json:"start"` // Start is the start of the words.
    End   float64 `json:"end"`   // End is the end of the words.
}

Generated by gomarkdoc

Documentation

Overview

Package groq provides a unofficial client for the Groq API.

With specially designed hardware, the Groq API is a super fast way to query open source llms.

API Documentation: https://console.groq.com/docs/quickstart

Code generated by groq-modeler DO NOT EDIT.

Created at: 2024-10-26 10:01:35

groq-modeler Version 1.1.2

Index

Constants

View Source
const (
	AudioResponseFormatJSON        AudioResponseFormat = "json"         // AudioResponseFormatJSON is the JSON response format of some audio.
	AudioResponseFormatText        AudioResponseFormat = "text"         // AudioResponseFormatText is the text response format of some audio.
	AudioResponseFormatSRT         AudioResponseFormat = "srt"          // AudioResponseFormatSRT is the SRT response format of some audio.
	AudioResponseFormatVerboseJSON AudioResponseFormat = "verbose_json" // AudioResponseFormatVerboseJSON is the verbose JSON response format of some audio.
	AudioResponseFormatVTT         AudioResponseFormat = "vtt"          // AudioResponseFormatVTT is the VTT response format of some audio.

	TranscriptionTimestampGranularityWord    TranscriptionTimestampGranularity = "word"    // TranscriptionTimestampGranularityWord is the word timestamp granularity.
	TranscriptionTimestampGranularitySegment TranscriptionTimestampGranularity = "segment" // TranscriptionTimestampGranularitySegment is the segment timestamp granularity.
)
View Source
const (
	ChatMessageRoleSystem                      Role                             = "system"         // ChatMessageRoleSystem is the system chat message role.
	ChatMessageRoleUser                        Role                             = "user"           // ChatMessageRoleUser is the user chat message role.
	ChatMessageRoleAssistant                   Role                             = "assistant"      // ChatMessageRoleAssistant is the assistant chat message role.
	ChatMessageRoleFunction                    Role                             = "function"       // ChatMessageRoleFunction is the function chat message role.
	ChatMessageRoleTool                        Role                             = "tool"           // ChatMessageRoleTool is the tool chat message role.
	ImageURLDetailHigh                         ImageURLDetail                   = "high"           // ImageURLDetailHigh is the high image url detail.
	ImageURLDetailLow                          ImageURLDetail                   = "low"            // ImageURLDetailLow is the low image url detail.
	ImageURLDetailAuto                         ImageURLDetail                   = "auto"           // ImageURLDetailAuto is the auto image url detail.
	ChatMessagePartTypeText                    ChatMessagePartType              = "text"           // ChatMessagePartTypeText is the text chat message part type.
	ChatMessagePartTypeImageURL                ChatMessagePartType              = "image_url"      // ChatMessagePartTypeImageURL is the image url chat message part type.
	ChatCompletionResponseFormatTypeJSONObject ChatCompletionResponseFormatType = "json_object"    // ChatCompletionResponseFormatTypeJSONObject is the json object chat completion response format type.
	ChatCompletionResponseFormatTypeJSONSchema ChatCompletionResponseFormatType = "json_schema"    // ChatCompletionResponseFormatTypeJSONSchema is the json schema chat completion response format type.
	ChatCompletionResponseFormatTypeText       ChatCompletionResponseFormatType = "text"           // ChatCompletionResponseFormatTypeText is the text chat completion response format type.
	FinishReasonStop                           FinishReason                     = "stop"           // FinishReasonStop is the stop finish reason.
	FinishReasonLength                         FinishReason                     = "length"         // FinishReasonLength is the length finish reason.
	FinishReasonFunctionCall                   FinishReason                     = "function_call"  // FinishReasonFunctionCall is the function call finish reason.
	FinishReasonToolCalls                      FinishReason                     = "tool_calls"     // FinishReasonToolCalls is the tool calls finish reason.
	FinishReasonContentFilter                  FinishReason                     = "content_filter" // FinishReasonContentFilter is the content filter finish reason.
	FinishReasonNull                           FinishReason                     = "null"           // FinishReasonNull is the null finish reason.
)

Variables

View Source
var (
	// ModelGemma29BIt is an AI text chat model.
	//
	// It is created/provided by Google.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelGemma29BIt ChatModel = "gemma2-9b-it"
	// ModelGemma7BIt is an AI text chat model.
	//
	// It is created/provided by Google.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelGemma7BIt ChatModel = "gemma-7b-it"
	// ModelLlama3170BVersatile is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 32768 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama3170BVersatile ChatModel = "llama-3.1-70b-versatile"
	// ModelLlama318BInstant is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 131072 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama318BInstant ChatModel = "llama-3.1-8b-instant"
	// ModelLlama3211BTextPreview is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama3211BTextPreview ChatModel = "llama-3.2-11b-text-preview"
	// ModelLlama3211BVisionPreview is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama3211BVisionPreview ChatModel = "llama-3.2-11b-vision-preview"
	// ModelLlama321BPreview is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama321BPreview ChatModel = "llama-3.2-1b-preview"
	// ModelLlama323BPreview is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama323BPreview ChatModel = "llama-3.2-3b-preview"
	// ModelLlama3290BTextPreview is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama3290BTextPreview ChatModel = "llama-3.2-90b-text-preview"
	// ModelLlama3290BVisionPreview is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama3290BVisionPreview ChatModel = "llama-3.2-90b-vision-preview"
	// ModelLlama370B8192 is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama370B8192 ChatModel = "llama3-70b-8192"
	// ModelLlama38B8192 is an AI text chat model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama38B8192 ChatModel = "llama3-8b-8192"
	// ModelLlama3Groq70B8192ToolUsePreview is an AI text chat model.
	//
	// It is created/provided by Groq.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama3Groq70B8192ToolUsePreview ChatModel = "llama3-groq-70b-8192-tool-use-preview"
	// ModelLlama3Groq8B8192ToolUsePreview is an AI text chat model.
	//
	// It is created/provided by Groq.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlama3Groq8B8192ToolUsePreview ChatModel = "llama3-groq-8b-8192-tool-use-preview"
	// ModelLlavaV157B4096Preview is an AI text chat model.
	//
	// It is created/provided by Other.
	//
	// It has 4096 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelLlavaV157B4096Preview ChatModel = "llava-v1.5-7b-4096-preview"
	// ModelMixtral8X7B32768 is an AI text chat model.
	//
	// It is created/provided by Mistral AI.
	//
	// It has 32768 context window.
	//
	// It can be used with the following client methods:
	//	- CreateChatCompletion
	// 	- CreateChatCompletionStream
	// 	- CreateChatCompletionJSON
	ModelMixtral8X7B32768 ChatModel = "mixtral-8x7b-32768"
	// ModelWhisperLargeV3 is an AI audio transcription model.
	//
	// It is created/provided by OpenAI.
	//
	// It has 448 context window.
	//
	// It can be used with the following client methods:
	//	- CreateTranscription
	// 	- CreateTranslation
	ModelWhisperLargeV3 AudioModel = "whisper-large-v3"
	// ModelLlamaGuard38B is an AI moderation model.
	//
	// It is created/provided by Meta.
	//
	// It has 8192 context window.
	//
	// It can be used with the following client methods:
	//	- Moderate
	ModelLlamaGuard38B ModerationModel = "llama-guard-3-8b"
)

Functions

func AudioMultipartForm added in v0.9.3

func AudioMultipartForm(request AudioRequest, b builders.FormBuilder) error

AudioMultipartForm creates a form with audio file contents and the name of the model to use for audio processing.

func CreateFileField added in v0.9.3

func CreateFileField(
	request AudioRequest,
	b builders.FormBuilder,
) (err error)

CreateFileField creates the "file" form field from either an existing file or by using the reader.

Types

type APIError added in v0.9.0

type APIError struct {
	Code           any     `json:"code,omitempty"`  // Code is the code of the error.
	Message        string  `json:"message"`         // Message is the message of the error.
	Param          *string `json:"param,omitempty"` // Param is the param of the error.
	Type           string  `json:"type"`            // Type is the type of the error.
	HTTPStatusCode int     `json:"-"`               // HTTPStatusCode is the status code of the error.
}

APIError provides error information returned by the Groq API.

func (*APIError) Error added in v0.9.0

func (e *APIError) Error() string

Error method implements the error interface on APIError.

func (*APIError) UnmarshalJSON added in v0.9.0

func (e *APIError) UnmarshalJSON(data []byte) (err error)

UnmarshalJSON implements the json.Unmarshaler interface.

type AudioModel added in v0.9.3

type AudioModel model

AudioModel is the type for audio models present on the groq api.

type AudioRequest added in v0.9.0

type AudioRequest struct {
	Model       AudioModel          // Model is the model to use for the transcription.
	FilePath    string              // FilePath is either an existing file in your filesystem or a filename representing the contents of Reader.
	Reader      io.Reader           // Reader is an optional io.Reader when you do not want to use an existing file.
	Prompt      string              // Prompt is the prompt for the transcription.
	Temperature float32             // Temperature is the temperature for the transcription.
	Language    string              // Language is the language for the transcription. Only for transcription.
	Format      AudioResponseFormat // Format is the format for the response.
}

AudioRequest represents a request structure for audio API.

type AudioResponse added in v0.9.0

type AudioResponse struct {
	Task     string   `json:"task"`     // Task is the task of the response.
	Language string   `json:"language"` // Language is the language of the response.
	Duration float64  `json:"duration"` // Duration is the duration of the response.
	Segments Segments `json:"segments"` // Segments is the segments of the response.
	Words    Words    `json:"words"`    // Words is the words of the response.
	Text     string   `json:"text"`     // Text is the text of the response.

	Header http.Header // Header is the header of the response.
}

AudioResponse represents a response structure for audio API.

func (*AudioResponse) SetHeader added in v0.9.0

func (r *AudioResponse) SetHeader(header http.Header)

SetHeader sets the header of the response.

type AudioResponseFormat added in v0.9.0

type AudioResponseFormat string

AudioResponseFormat is the response format for the audio API.

Response formatted using AudioResponseFormatJSON by default.

string

type ChatCompletionChoice added in v0.9.0

type ChatCompletionChoice struct {
	Index int `json:"index"` // Index is the index of the choice.
	// Message is the chat completion message of the choice.
	Message ChatCompletionMessage `json:"message"` // Message is the chat completion message of the choice.
	// FinishReason is the finish reason of the choice.
	//
	// stop: API returned complete message,
	// or a message terminated by one of the stop sequences provided via the stop parameter
	// length: Incomplete model output due to max_tokens parameter or token limit
	// function_call: The model decided to call a function
	// content_filter: Omitted content due to a flag from our content filters
	// null: API response still in progress or incomplete
	FinishReason FinishReason `json:"finish_reason"` // FinishReason is the finish reason of the choice.
	// LogProbs is the log probs of the choice.
	//
	// This is basically the probability of the model choosing the token.
	LogProbs *LogProbs `json:"logprobs,omitempty"` // LogProbs is the log probs of the choice.
}

ChatCompletionChoice represents the chat completion choice.

type ChatCompletionMessage added in v0.9.0

type ChatCompletionMessage struct {
	Name         string              `json:"name"`                    // Name is the name of the chat completion message.
	Role         Role                `json:"role"`                    // Role is the role of the chat completion message.
	Content      string              `json:"content"`                 // Content is the content of the chat completion message.
	MultiContent []ChatMessagePart   `json:"-"`                       // MultiContent is the multi content of the chat completion message.
	FunctionCall *tools.FunctionCall `json:"function_call,omitempty"` // FunctionCall setting for Role=assistant prompts this may be set to the function call generated by the model.
	ToolCalls    []tools.ToolCall    `json:"tool_calls,omitempty"`    // ToolCalls setting for Role=assistant prompts this may be set to the tool calls generated by the model, such as function calls.
	ToolCallID   string              `json:"tool_call_id,omitempty"`  // ToolCallID is setting for Role=tool prompts this should be set to the ID given in the assistant's prior request to call a tool.
}

ChatCompletionMessage represents the chat completion message.

func (ChatCompletionMessage) MarshalJSON added in v0.9.0

func (m ChatCompletionMessage) MarshalJSON() ([]byte, error)

MarshalJSON method implements the json.Marshaler interface.

func (*ChatCompletionMessage) UnmarshalJSON added in v0.9.0

func (m *ChatCompletionMessage) UnmarshalJSON(bs []byte) (err error)

UnmarshalJSON method implements the json.Unmarshaler interface.

type ChatCompletionRequest added in v0.9.0

type ChatCompletionRequest struct {
	Model             ChatModel                     `json:"model"`                         // Model is the model of the chat completion request.
	Messages          []ChatCompletionMessage       `json:"messages"`                      // Messages is the messages of the chat completion request. These act as the prompt for the model.
	MaxTokens         int                           `json:"max_tokens,omitempty"`          // MaxTokens is the max tokens of the chat completion request.
	Temperature       float32                       `json:"temperature,omitempty"`         // Temperature is the temperature of the chat completion request.
	TopP              float32                       `json:"top_p,omitempty"`               // TopP is the top p of the chat completion request.
	N                 int                           `json:"n,omitempty"`                   // N is the n of the chat completion request.
	Stream            bool                          `json:"stream,omitempty"`              // Stream is the stream of the chat completion request.
	Stop              []string                      `json:"stop,omitempty"`                // Stop is the stop of the chat completion request.
	PresencePenalty   float32                       `json:"presence_penalty,omitempty"`    // PresencePenalty is the presence penalty of the chat completion request.
	ResponseFormat    *ChatCompletionResponseFormat `json:"response_format,omitempty"`     // ResponseFormat is the response format of the chat completion request.
	Seed              *int                          `json:"seed,omitempty"`                // Seed is the seed of the chat completion request.
	FrequencyPenalty  float32                       `json:"frequency_penalty,omitempty"`   // FrequencyPenalty is the frequency penalty of the chat completion request.
	LogitBias         map[string]int                `json:"logit_bias,omitempty"`          // LogitBias is must be a token id string (specified by their token ID in the tokenizer), not a word string. incorrect: `"logit_bias":{ "You": 6}`, correct: `"logit_bias":{"1639": 6}` refs: https://platform.openai.com/docs/api-reference/chat/create#chat/create-logit_bias
	LogProbs          bool                          `json:"logprobs,omitempty"`            // LogProbs indicates whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message. This option is currently not available on the gpt-4-vision-preview model.
	TopLogProbs       int                           `json:"top_logprobs,omitempty"`        // TopLogProbs is an integer between 0 and 5 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.
	User              string                        `json:"user,omitempty"`                // User is the user of the chat completion request.
	Tools             []tools.Tool                  `json:"tools,omitempty"`               // Tools is the tools of the chat completion request.
	ToolChoice        any                           `json:"tool_choice,omitempty"`         // This can be either a string or an ToolChoice object.
	StreamOptions     *StreamOptions                `json:"stream_options,omitempty"`      // Options for streaming response. Only set this when you set stream: true.
	ParallelToolCalls any                           `json:"parallel_tool_calls,omitempty"` // Disable the default behavior of parallel tool calls by setting it: false.
	RetryDelay        time.Duration                 `json:"-"`                             // RetryDelay is the delay between retries.
}

ChatCompletionRequest represents a request structure for the chat completion API.

type ChatCompletionResponse added in v0.9.0

type ChatCompletionResponse struct {
	ID                string                 `json:"id"`                 // ID is the id of the response.
	Object            string                 `json:"object"`             // Object is the object of the response.
	Created           int64                  `json:"created"`            // Created is the created time of the response.
	Model             ChatModel              `json:"model"`              // Model is the model of the response.
	Choices           []ChatCompletionChoice `json:"choices"`            // Choices is the choices of the response.
	Usage             Usage                  `json:"usage"`              // Usage is the usage of the response.
	SystemFingerprint string                 `json:"system_fingerprint"` // SystemFingerprint is the system fingerprint of the response.
	http.Header                              // Header is the header of the response.
}

ChatCompletionResponse represents a response structure for chat completion API.

func (*ChatCompletionResponse) SetHeader added in v0.9.0

func (r *ChatCompletionResponse) SetHeader(h http.Header)

SetHeader sets the header of the response.

type ChatCompletionResponseFormat added in v0.9.0

type ChatCompletionResponseFormat struct {
	// Type is the type of the chat completion response format.
	Type ChatCompletionResponseFormatType `json:"type,omitempty"`
	// JSONSchema is the json schema of the chat completion response format.
	JSONSchema *ChatCompletionResponseFormatJSONSchema `json:"json_schema,omitempty"`
}

ChatCompletionResponseFormat is the chat completion response format.

type ChatCompletionResponseFormatJSONSchema added in v0.9.0

type ChatCompletionResponseFormatJSONSchema struct {
	// Name is the name of the chat completion response format json
	// schema.
	//
	// it is used to further identify the schema in the response.
	Name string `json:"name"`
	// Description is the description of the chat completion response
	// format
	// json schema.
	Description string `json:"description,omitempty"`
	// description of the chat completion response format json schema.
	// Schema is the schema of the chat completion response format json schema.
	Schema schema.Schema `json:"schema"`
	// Strict determines whether to enforce the schema upon the generated
	// content.
	Strict bool `json:"strict"`
}

ChatCompletionResponseFormatJSONSchema is the chat completion response format json schema.

type ChatCompletionResponseFormatType added in v0.9.0

type ChatCompletionResponseFormatType string

ChatCompletionResponseFormatType is the chat completion response format type.

string

type ChatCompletionStream added in v0.9.0

type ChatCompletionStream struct {
	// contains filtered or unexported fields
}

ChatCompletionStream is a stream of ChatCompletionStreamResponse.

Note: Perhaps it is more elegant to abstract Stream using generics.

func (ChatCompletionStream) Close added in v0.9.0

func (stream ChatCompletionStream) Close() error

func (ChatCompletionStream) Recv added in v0.9.0

func (stream ChatCompletionStream) Recv() (response T, err error)

Recv receives a response from the stream.

type ChatCompletionStreamChoice added in v0.9.0

type ChatCompletionStreamChoice struct {
	Index        int                             `json:"index"`
	Delta        ChatCompletionStreamChoiceDelta `json:"delta"`
	FinishReason FinishReason                    `json:"finish_reason"`
}

ChatCompletionStreamChoice represents a response structure for chat completion API.

type ChatCompletionStreamChoiceDelta added in v0.9.0

type ChatCompletionStreamChoiceDelta struct {
	Content      string              `json:"content,omitempty"`
	Role         string              `json:"role,omitempty"`
	FunctionCall *tools.FunctionCall `json:"function_call,omitempty"`
	ToolCalls    []tools.ToolCall    `json:"tool_calls,omitempty"`
}

ChatCompletionStreamChoiceDelta represents a response structure for chat completion API.

type ChatCompletionStreamResponse added in v0.9.0

type ChatCompletionStreamResponse struct {
	ID                  string                       `json:"id"`                           // ID is the identifier for the chat completion stream response.
	Object              string                       `json:"object"`                       // Object is the object type of the chat completion stream response.
	Created             int64                        `json:"created"`                      // Created is the creation time of the chat completion stream response.
	Model               ChatModel                    `json:"model"`                        // Model is the model used for the chat completion stream response.
	Choices             []ChatCompletionStreamChoice `json:"choices"`                      // Choices is the choices for the chat completion stream response.
	SystemFingerprint   string                       `json:"system_fingerprint"`           // SystemFingerprint is the system fingerprint for the chat completion stream response.
	PromptAnnotations   []PromptAnnotation           `json:"prompt_annotations,omitempty"` // PromptAnnotations is the prompt annotations for the chat completion stream response.
	PromptFilterResults []struct {
		Index int `json:"index"`
	} `json:"prompt_filter_results,omitempty"` // PromptFilterResults is the prompt filter results for the chat completion stream response.
	// Usage is an optional field that will only be present when you set stream_options: {"include_usage": true} in your request.
	//
	// When present, it contains a null value except for the last chunk which contains the token usage statistics
	// for the entire request.
	Usage *Usage `json:"usage,omitempty"`
}

ChatCompletionStreamResponse represents a response structure for chat completion API.

type ChatMessageImageURL added in v0.9.0

type ChatMessageImageURL struct {
	URL    string         `json:"url,omitempty"`    // URL is the url of the image.
	Detail ImageURLDetail `json:"detail,omitempty"` // Detail is the detail of the image url.
}

ChatMessageImageURL represents the chat message image url.

type ChatMessagePart added in v0.9.0

type ChatMessagePart struct {
	Text     string               `json:"text,omitempty"`      // Text is the text of the chat message part.
	Type     ChatMessagePartType  `json:"type,omitempty"`      // Type is the type of the chat message part.
	ImageURL *ChatMessageImageURL `json:"image_url,omitempty"` // ImageURL is the image url of the chat message part.
}

ChatMessagePart represents the chat message part of a chat completion message.

type ChatMessagePartType added in v0.9.0

type ChatMessagePartType string

ChatMessagePartType is the chat message part type.

string

type ChatModel added in v0.9.3

type ChatModel model

ChatModel is the type for chat models present on the groq api.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is a Groq api client.

func NewClient

func NewClient(groqAPIKey string, opts ...Opts) (*Client, error)

NewClient creates a new Groq client.

func (*Client) CreateChatCompletion added in v0.9.0

func (c *Client) CreateChatCompletion(
	ctx context.Context,
	request ChatCompletionRequest,
) (response ChatCompletionResponse, err error)

CreateChatCompletion method is an API call to create a chat completion.

func (*Client) CreateChatCompletionJSON added in v0.9.1

func (c *Client) CreateChatCompletionJSON(
	ctx context.Context,
	request ChatCompletionRequest,
	output any,
) (err error)

CreateChatCompletionJSON method is an API call to create a chat completion w/ object output.

func (*Client) CreateChatCompletionStream added in v0.9.0

func (c *Client) CreateChatCompletionStream(
	ctx context.Context,
	request ChatCompletionRequest,
) (stream *ChatCompletionStream, err error)

CreateChatCompletionStream method is an API call to create a chat completion w/ streaming support.

If set, tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message.

func (*Client) CreateTranscription added in v0.9.0

func (c *Client) CreateTranscription(
	ctx context.Context,
	request AudioRequest,
) (AudioResponse, error)

CreateTranscription calls the transcriptions endpoint with the given request.

Returns transcribed text in the response_format specified in the request.

func (*Client) CreateTranslation added in v0.9.0

func (c *Client) CreateTranslation(
	ctx context.Context,
	request AudioRequest,
) (AudioResponse, error)

CreateTranslation calls the translations endpoint with the given request.

Returns the translated text in the response_format specified in the request.

func (*Client) Moderate added in v0.9.0

func (c *Client) Moderate(
	ctx context.Context,
	request ModerationRequest,
) (response Moderation, err error)

Moderate performs a moderation api call over a string. Input can be an array or slice but a string will reduce the complexity.

func (*Client) MustCreateChatCompletion added in v0.9.3

func (c *Client) MustCreateChatCompletion(
	ctx context.Context,
	request ChatCompletionRequest,
) (response ChatCompletionResponse)

MustCreateChatCompletion method is an API call to create a chat completion.

It panics if an error occurs.

type DefaultErrorAccumulator added in v0.9.0

type DefaultErrorAccumulator struct {
	Buffer errorBuffer
}

DefaultErrorAccumulator is a default implementation of ErrorAccumulator

func (*DefaultErrorAccumulator) Bytes added in v0.9.0

func (e *DefaultErrorAccumulator) Bytes() (errBytes []byte)

Bytes method returns the bytes of the error accumulator.

func (*DefaultErrorAccumulator) Write added in v0.9.0

func (e *DefaultErrorAccumulator) Write(p []byte) error

Write method writes bytes to the error accumulator.

type Endpoint added in v0.9.0

type Endpoint string

Endpoint is the endpoint for the groq api. string

type ErrContentFieldsMisused added in v0.9.0

type ErrContentFieldsMisused struct {
	// contains filtered or unexported fields
}

ErrContentFieldsMisused is an error that occurs when both Content and MultiContent properties are set.

func (ErrContentFieldsMisused) Error added in v0.9.0

func (e ErrContentFieldsMisused) Error() string

Error implements the error interface.

type ErrTooManyEmptyStreamMessages added in v0.9.0

type ErrTooManyEmptyStreamMessages struct{}

ErrTooManyEmptyStreamMessages is returned when the stream has sent too many empty messages.

func (ErrTooManyEmptyStreamMessages) Error added in v0.9.0

Error returns the error message.

type FinishReason added in v0.9.0

type FinishReason string

FinishReason is the finish reason. string

func (FinishReason) MarshalJSON added in v0.9.0

func (r FinishReason) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

type Format

type Format string

Format is the format of a response. string

const (
	// FormatText is the text format. It is the default format of a
	// response.
	FormatText Format = "text"
	// FormatJSON is the JSON format. There is no support for streaming with
	// JSON format selected.
	FormatJSON Format = "json"
)

type HarmfulCategory added in v0.9.0

type HarmfulCategory string

HarmfulCategory is a category of harmful content that can be used to categorize a chat history using Llama Guard 3.

[Llama Guard 3](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama-guard-3/)

Llama Guard 3 provides content safety support for the following languages: English, French, German, Hindi, Italian, Portuguese, Spanish, and Thai.

string

const (
	// CategoryViolentCrimes (S1) is the violent crimes category.
	//
	// Responses that enable, encourage, or endorse the commission of
	// violent crimes, including: (1) unlawful violence toward people
	// (ex: terrorism, genocide, murder, hate-motivated violent crimes,
	// child abuse, assault, battery, kidnapping) and (2) unlawful violence
	// toward animals (ex: animal abuse)
	CategoryViolentCrimes HarmfulCategory = "violent_crimes"
	// CategoryNonviolentCrimes (S2) is the non-violent crimes category.
	CategoryNonviolentCrimes HarmfulCategory = "non_violent_crimes"
	// CategorySexRelatedCrimes (S3) is the sex-related crimes category.
	CategorySexRelatedCrimes HarmfulCategory = "sex_related_crimes"
	// CategoryChildSexualExploitation (S4) is the child sexual exploitation
	// category.
	//
	// Responses that contain, describe, enable, encourage, or endorse the
	// sexual abuse of children.
	CategoryChildSexualExploitation HarmfulCategory = "child_sexual_exploitation"
	// CategoryDefamation (S5) is the defamation category.
	//
	// Signifies responses that contain, describe, enable, encourage, or
	// endorse defamation.
	CategoryDefamation HarmfulCategory = "defamation"
	// CategorySpecializedAdvice (S6) is the specialized advice category.
	//
	// Signifies responses contain, describe, enable, encourage, or endorse specialized advice.
	CategorySpecializedAdvice HarmfulCategory = "specialized_advice"
	// CategoryPrivacy (S7) is the privacy category.
	//
	// Responses contain, describe, enable, encourage, or endorse privacy.
	CategoryPrivacy HarmfulCategory = "privacy"
	// CategoryIntellectualProperty (S8) is the intellectual property
	// category. Responses that contain, describe, enable, encourage, or
	// endorse intellectual property.
	CategoryIntellectualProperty HarmfulCategory = "intellectual_property"
	// CategoryIndiscriminateWeapons (S9) is the indiscriminate weapons
	// category.
	//
	// Responses that contain, describe, enable, encourage, or endorse
	// indiscriminate weapons.
	CategoryIndiscriminateWeapons HarmfulCategory = "indiscriminate_weapons"
	// CategoryHate (S10) is the hate category.
	//
	// Responses contain, describe, enable, encourage, or endorse hate.
	CategoryHate HarmfulCategory = "hate"
	// CategorySuicideAndSelfHarm (S11) is the suicide/self-harm category.
	//
	// Responses contain, describe, enable, encourage, or endorse suicide or self-harm.
	CategorySuicideAndSelfHarm HarmfulCategory = "suicide_and_self_harm"
	// CategorySexualContent (S12) is the sexual content category.
	//
	// Responses contain, describe, enable, encourage, or endorse
	// sexual content.
	CategorySexualContent HarmfulCategory = "sexual_content"
	// CategoryElections (S13) is the elections category.
	//
	// Responses contain factually incorrect information about electoral
	// systems and processes, including in the time, place, or manner of
	// voting in civic elections.
	CategoryElections HarmfulCategory = "elections"
	// CategoryCodeInterpreterAbuse (S14) is the code interpreter abuse
	// category.
	//
	// Responses that contain, describe, enable, encourage, or
	// endorse code interpreter abuse.
	CategoryCodeInterpreterAbuse HarmfulCategory = "code_interpreter_abuse"
)

type ImageURLDetail added in v0.9.0

type ImageURLDetail string

ImageURLDetail is the image url detail.

string

type LogProbs added in v0.9.0

type LogProbs struct {
	Content []struct {
		Token       string        `json:"token"`           // Token is the token of the log prob.
		LogProb     float64       `json:"logprob"`         // LogProb is the log prob of the log prob.
		Bytes       []byte        `json:"bytes,omitempty"` // Omitting the field if it is null
		TopLogProbs []TopLogProbs `json:"top_logprobs"`    // TopLogProbs is a list of the most likely tokens and their log probability, at this token position. In rare cases, there may be fewer than the number of requested top_logprobs returned.
	} `json:"content"` // Content is a list of message content tokens with log probability information.
}

LogProbs is the top-level structure containing the log probability information.

type Moderation added in v0.9.0

type Moderation struct {
	Categories []HarmfulCategory `json:"categories"` // Categories is the categories of the result.
	Flagged    bool              `json:"flagged"`    // Flagged is the flagged of the result.
}

Moderation represents one of possible moderation results.

type ModerationModel added in v0.9.3

type ModerationModel model

ModerationModel is the type for moderation models present on the groq api.

type ModerationRequest added in v0.9.0

type ModerationRequest struct {
	// Input string `json:"input,omitempty"` // Input is the input text to be moderated.
	Messages []ChatCompletionMessage `json:"messages"`        // Messages is the messages of the chat completion request. These act as the prompt for the model.
	Model    ModerationModel         `json:"model,omitempty"` // Model is the model to use for the moderation.
}

ModerationRequest represents a request structure for moderation API.

type Opts added in v0.9.0

type Opts func(*Client)

Opts is a function that sets options for a Groq client.

func WithBaseURL added in v0.9.0

func WithBaseURL(baseURL string) Opts

WithBaseURL sets the base URL for the Groq client.

func WithClient

func WithClient(client *http.Client) Opts

WithClient sets the client for the Groq client.

func WithLogger added in v0.9.0

func WithLogger(logger *slog.Logger) Opts

WithLogger sets the logger for the Groq client.

type PromptAnnotation added in v0.9.0

type PromptAnnotation struct {
	PromptIndex int `json:"prompt_index,omitempty"`
}

PromptAnnotation represents the prompt annotation.

type RateLimitHeaders added in v0.9.0

type RateLimitHeaders struct {
	LimitRequests     int       `json:"x-ratelimit-limit-requests"`     // LimitRequests is the limit requests of the rate limit headers.
	LimitTokens       int       `json:"x-ratelimit-limit-tokens"`       // LimitTokens is the limit tokens of the rate limit headers.
	RemainingRequests int       `json:"x-ratelimit-remaining-requests"` // RemainingRequests is the remaining requests of the rate limit headers.
	RemainingTokens   int       `json:"x-ratelimit-remaining-tokens"`   // RemainingTokens is the remaining tokens of the rate limit headers.
	ResetRequests     ResetTime `json:"x-ratelimit-reset-requests"`     // ResetRequests is the reset requests of the rate limit headers.
	ResetTokens       ResetTime `json:"x-ratelimit-reset-tokens"`       // ResetTokens is the reset tokens of the rate limit headers.
}

RateLimitHeaders struct represents Groq rate limits headers.

type ResetTime added in v0.9.0

type ResetTime string

ResetTime is a time.Time wrapper for the rate limit reset time. string

func (ResetTime) String added in v0.9.0

func (r ResetTime) String() string

String returns the string representation of the ResetTime.

func (ResetTime) Time added in v0.9.0

func (r ResetTime) Time() time.Time

Time returns the time.Time representation of the ResetTime.

type Role added in v0.9.0

type Role string

Role is the role of the chat completion message.

string

type Segments added in v0.9.0

type Segments []struct {
	ID               int     `json:"id"`                // ID is the ID of the segment.
	Seek             int     `json:"seek"`              // Seek is the seek of the segment.
	Start            float64 `json:"start"`             // Start is the start of the segment.
	End              float64 `json:"end"`               // End is the end of the segment.
	Text             string  `json:"text"`              // Text is the text of the segment.
	Tokens           []int   `json:"tokens"`            // Tokens is the tokens of the segment.
	Temperature      float64 `json:"temperature"`       // Temperature is the temperature of the segment.
	AvgLogprob       float64 `json:"avg_logprob"`       // AvgLogprob is the avg log prob of the segment.
	CompressionRatio float64 `json:"compression_ratio"` // CompressionRatio is the compression ratio of the segment.
	NoSpeechProb     float64 `json:"no_speech_prob"`    // NoSpeechProb is the no speech prob of the segment.
	Transient        bool    `json:"transient"`         // Transient is the transient of the segment.
}

Segments is the segments of the response.

type StreamOptions added in v0.9.0

type StreamOptions struct {
	// If set, an additional chunk will be streamed before the data: [DONE] message.
	// The usage field on this chunk shows the token usage statistics for the entire request,
	// and the choices field will always be an empty array.
	// All other chunks will also include a usage field, but with a null value.
	IncludeUsage bool `json:"include_usage,omitempty"`
}

StreamOptions represents the stream options.

type TopLogProbs added in v0.9.0

type TopLogProbs struct {
	Token   string  `json:"token"`           // Token is the token of the top log probs.
	LogProb float64 `json:"logprob"`         // LogProb is the log prob of the top log probs.
	Bytes   []byte  `json:"bytes,omitempty"` // Bytes is the bytes of the top log probs.
}

TopLogProbs represents the top log probs.

type TranscriptionTimestampGranularity added in v0.9.0

type TranscriptionTimestampGranularity string

TranscriptionTimestampGranularity is the timestamp granularity for the transcription.

string

type Usage added in v0.9.0

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

Usage Represents the total token usage per request to Groq.

type Words added in v0.9.0

type Words []struct {
	Word  string  `json:"word"`  // Word is the word of the words.
	Start float64 `json:"start"` // Start is the start of the words.
	End   float64 `json:"end"`   // End is the end of the words.
}

Words is the words of the response.

Directories

Path Synopsis
examples
audio-lex-fridman command
Package main is an example of using groq-go to create a transcription using the whisper model.
Package main is an example of using groq-go to create a transcription using the whisper model.
chat-terminal command
Package main demonstrates how to use groq-go to create a chat application using the groq api accessable through the terminal.
Package main demonstrates how to use groq-go to create a chat application using the groq api accessable through the terminal.
e2b-go-project command
json-chat command
Package main demonstrates an example application of groq-go.
Package main demonstrates an example application of groq-go.
llava-blind command
Package main demonstrates an example application of groq-go.
Package main demonstrates an example application of groq-go.
moderation command
Package main is an example of using groq-go to create a chat moderation using the llama-3BGuard model.
Package main is an example of using groq-go to create a chat moderation using the llama-3BGuard model.
toolhouse-python-code-interpreter command
Package main shows an example of using the toolhouse go package.
Package main shows an example of using the toolhouse go package.
vhdl-documentor-json command
Package main is an example of using groq-go to create a chat completion using the llama-70B-tools-preview model to create headers for vhdl projects.
Package main is an example of using groq-go to create a chat completion using the llama-70B-tools-preview model to create headers for vhdl projects.
terminal-chat module
extensions
composio
Package composio provides a composio client for groq-go.
Package composio provides a composio client for groq-go.
e2b
Package e2b provides an e2b client for groq-go.
Package e2b provides an e2b client for groq-go.
toolhouse
Package toolhouse provides a Toolhouse extension for groq-go.
Package toolhouse provides a Toolhouse extension for groq-go.
pkg
builders
Package builders provides builders for HTTP requests and forms.
Package builders provides builders for HTTP requests and forms.
test
Package test contains test helpers.
Package test contains test helpers.
tools
Package tools contains the interfaces for groq-go tooling usable by llms.
Package tools contains the interfaces for groq-go tooling usable by llms.
scripts
models module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL