groq

Documentation

Overview

Package groq provides Groq-backed LLM, STT, and TTS via OpenAI-compatible API.

Index

• Constants
• func NewClient(apiKey string) *openai.Client
• type LLMService
  • func NewLLMService(apiKey, model string) *LLMService
  • func (s *LLMService) Chat(ctx context.Context, messages []map[string]any, ...) error

Constants

const DefaultLLMModel = "llama-3.1-8b-instant"

DefaultLLMModel is the default Groq chat model when none is specified.

Functions

func NewClient

func NewClient(apiKey string) *openai.Client

NewClient returns an OpenAI-compatible client configured for Groq. If apiKey is empty, config.GetEnv("GROQ_API_KEY", "") is used.

Types

type LLMService

type LLMService struct {
	// contains filtered or unexported fields
}

LLMService implements services.LLMService using Groq's OpenAI-compatible chat API.

func NewLLMService

func NewLLMService(apiKey, model string) *LLMService

NewLLMService creates a Groq LLM service. If apiKey is empty, config.GetEnv("GROQ_API_KEY", "") is used. If model is empty, DefaultLLMModel is used.

func (*LLMService) Chat

func (s *LLMService) Chat(ctx context.Context, messages []map[string]any, onToken func(*frames.LLMTextFrame)) error

Chat runs a completion and calls onToken for each streamed content delta (as LLMTextFrame).