genai

README

genai

The opinionated high performance professional-grade AI package for Go.

genai is intentional. Curious why it was created? See the release announcement at maruel.ca/post/genai-v0.1.0.

Features

• Full functionality: Full access to each backend-specific functionality. Access the raw API if needed with full message schema as Go structs.
• Tool calling via reflection: Tell the LLM to call a tool directly, described as a Go struct. No need to manually fiddle with JSON.
• Native JSON struct serialization: Pass a struct to tell the LLM what to generate, decode the reply into your struct. No need to manually fiddle with JSON. Supports required fields, enums, descriptions, etc. You can still fiddle if you want to. :)
• Decision inference: Ask typed questions about text or JSON state through Provider.SystemOne(). TypeSafe's Jev, Cloudflare's CLEF and CLEF-Flash, and local llama.cpp decision models share genai.Questions and genai.Answers maps with dynamic names and typed values. Document support depends on the provider and model. See the TypeSafe, Cloudflare, and llama.cpp examples.
• Streaming: Streams completion reply as the output is being generated, including thinking and tool calling, via go 1.23 iterators.
• Multi-modal: Process images, PDFs and videos (!) as input or output.
• Web Search: Search the web to answer your question and cite documents passed in.
• Smoke testing friendly: record and play back API calls at HTTP level to save 💰 and keep tests fast and reproducible, via the exposed HTTP transport. See example.
• Rate limits and usage: Parse the provider-specific HTTP headers and JSON response to get the tokens usage and remaining quota.
• Provide access to HTTP headers to enable beta features.

Design

• Safe and strict API implementation. All you love from a statically typed language. The library's smoke tests immediately fail on unknown RPC fields. Error code paths are properly implemented.
• Stateless. No global state, it is safe to use clients concurrently.
• Professional grade. smoke tested on live services with recorded traces located in testdata/ directories, e.g. providers/anthropic/testdata/TestClient/Scoreboard/.
• Trust, but verify. It generates a scoreboard based on actual behavior from each provider.
• Optimized for speed. Minimize memory allocations, compress data at the transport layer when possible. Groq, Mistral and OpenAI use brotli for HTTP compression instead of gzip, and POST's body to Google are gzip compressed.
• Lean: Few dependencies. No unnecessary abstraction layer.

Scoreboard

Provider	🌐	Mode	➛In	Out➛	Tool	JSON	Batch	File	Cite	Text	Probs	Limits	Usage	Finish
alibaba	🇨🇳	Sync, Stream🧠	💬📸	💬	✅🪨🔀🕸️	☁️	❌	❌	❌	🌱📏🛑	✅	❌	✅	✅
anthropic	🇺🇸	Sync, Stream🧠	💬📄📸	💬	✅🕸️🪨	📐[]	✅	❌	✅	📏🛑	❌	✅	✅	✅
antigravity	🇺🇸	Sync, Stream🧠	💬	💬	❌	📐	❌	❌	❌	❌	❌	❌	✅	✅
baseten	🇺🇸	Sync, Stream🧠	💬📸	💬	✅🪨🔀	☁️	❌	❌	❌	🌱📏🛑	❌	❌	✅	✅
bfl	🇩🇪	Sync	💬	📸	❌	❌	✅	❌	❌	🌱	❌	✅	❌	❌
cerebras	🇺🇸	Sync, Stream🧠	💬📸	💬	✅🪨	✅	❌	❌	❌	🌱📏🛑	✅	✅	✅	✅
claudecode	🇺🇸	Sync, Stream🧠	💬📸	💬	🕸️	❌	❌	❌	❌	❌	❌	❌	✅	✅
cloudflare	🇺🇸	Sync, Stream, SystemOne🧠	💬📸	🎯💬	💨	✅[]	❌	❌	❌	🌱📏	❌	❌	✅	💨
codex	🇺🇸	Sync, Stream🧠	💬📸	💬	❌	❌	❌	❌	❌	❌	❌	❌	✅	✅
cohere	🇨🇦	Sync, Stream🧠	💬	💬	✅🪨	✅	❌	❌	✅	🌱📏🛑	✅	❌	✅	✅
deepseek	🇨🇳	Sync, Stream🧠	💬	💬	✅🪨	☁️	❌	❌	❌	📏🛑	✅	❌	✅	✅
gemini	🇺🇸	Sync, Stream🧠, Embed	🎤🎥💬📄📸	💬🧬	✅🪨🕸️🔀	✅[]	❌	✅	❌	🌱🛑	❌	❌	✅	✅
groq	🇺🇸	Sync, Stream🧠	💬📸	💬	✅🪨🔀🕸️	☁️	❌	❌	❌	🌱📏🛑	❌	✅	✅	✅
huggingface	🇺🇸	Sync, Stream🧠	💬📸	💬	💨🪨	☁️	❌	❌	❌	🌱📏🛑	❌	✅	✅	✅
llamacpp	🏠	Sync, Stream, SystemOne, Embed🧠	🎤💬📸	🎯💬🧬	✅🪨🔀	✅[]	❌	❌	❌	🌱📏🛑	✅	❌	✅	✅
mistral	🇫🇷	Sync, Stream	🎤💬📄📸	💬	✅🪨	✅[]	❌	❌	❌	🌱📏🛑	❌	✅	✅	✅
ollama	🏠	Sync, Stream🧠, Embed, SystemOne	💬📸	🎯💬🧬	✅🔀	✅[]	❌	❌	❌	🌱📏🛑	✅	❌	✅	✅
openaichat	🇺🇸	Sync, Stream🧠, Embed	🎤💬📄📸	🎤💬📸🧬	✅🪨	✅	✅	✅	❌	🌱📏	❌	✅	✅	✅
openairesponses	🇺🇸	Sync, Stream🧠, Embed	💬📄📸	💬📸🧬	✅🪨🕸️	✅	✅	❌	❌	❌	❌	✅	✅	✅
opencode	🇺🇸	Sync, Stream🧠	💬📸	💬	❌	❌	❌	❌	❌	🌱	✅	❌	✅	✅
openrouter	🇺🇸	Sync, Stream🧠	💬📸	💬	✅🔀🪨	☁️[]	❌	❌	❌	🌱📏🛑	❌	❌	✅	✅
perplexity	🇺🇸	Sync, Stream🧠	💬📸	💬	🕸️	📐	❌	❌	✅	📏	❌	❌	✅	✅
pi	🇦🇹	Sync, Stream🧠	💬📸	💬	❌	❌	❌	❌	❌	🌱	✅	❌	✅	✅
pollinations	🇩🇪	Sync, Stream	💬📸	💬📸	✅🪨	☁️	❌	❌	❌	🌱📏	❌	❌	✅	✅
togetherai	🇺🇸	Sync, Stream🧠	💬📸	💬📸	✅🪨	✅[]	❌	❌	❌	🌱📏🛑	❌	❌	✅	✅
typesafe	🇺🇸	SystemOne	💬	🎯	❌	❌	❌	❌	❌	❌	❌	❌	✅	❌
xiaomi	🇨🇳	Sync, Stream🧠	🎤🎥💬📸	🎤💬	✅🪨🔀🕸️	☁️	❌	❌	❌	📏🛑	❌	❌	✅	✅
openaicompatible	N/A	Sync, Stream	💬	💬	❌	❌	❌	❌	❌	📏🛑	❌	❌	✅	✅

‼️ Click here for the legend of columns and symbols

• 🏠: Runs locally.
• Sync: Runs synchronously, the reply is only returned once completely generated
• Stream: Streams the reply as it is generated. Occasionally less features are supported in this mode
• 🧠: Has chain-of-thought thinking process
  • Both redacted (Anthropic, Gemini, OpenAI) and explicit (Deepseek R1, Qwen3, etc)
  • Many models can be used in both mode. In this case they will have two rows, one with thinking and one without. It is frequent that certain functionalities are limited in thinking mode, like tool calling.
• ✅: Implemented and works great
• ❌: Not supported by genai. The provider may support it, but genai does not (yet). Please send a PR to add it!
• 💬: Text
• 🎯: Decision: typed answers returned by SystemOne
• 🧬: Embedding: text vectors with qualified input order and semantic retrieval
• 📄: PDF: process a PDF as input, possibly with OCR
• 📸: Image: process an image as input; most providers support PNG, JPG, WEBP and non-animated GIF, or generate images
• 🎤: Audio: process an audio file (e.g. MP3, WAV, Flac, Opus) as input, or generate audio
• 🎥: Video: process a video (e.g. MP4) as input, or generate a video (e.g. Veo 3)
• 💨: Feature is flaky (Tool calling) or inconsistent (Usage or Finish reason is not always reported)
• 🌐: Country where the company is located
• Tool: Tool calling, using genai.ToolDef; best is ✅🪨🕸️🔀 - 🪨: Tool calling can be forced; aka you can force the model to call a tool. This is great. - 🕸️: Web search - 🔀: Supports messages and tool calls out of order
• JSON: ability to output JSON in free form, or with a forced schema specified as a Go struct
  • ✅: Supports both free form and with a schema
  • ☁️ :Supports only free form
    • 📐: Supports only a schema
    • []: Also supports an array at the root of the schema, most providers require an object
• Batch: Process asynchronously batches during off peak hours at a discounts
• Text: Text features
  • '🌱': Seed option for deterministic output
  • '📏': MaxTokens option to cap the amount of returned tokens
  • '🛑': Stop sequence to stop generation when a token is generated
• File: Upload and store large files via a separate API
• Cite: Citation generation from a provided document, specially useful for RAG
• Probs: Return logprobs to analyse each token probabilities
• Limits: Returns the rate limits, including the remaining quota

Examples

The following examples intentionally use a variety of providers to show the extent at which you can pick and chose. Close each provider client when finished.

Text Basic ✅

examples/txt_to_txt_sync/main.go: This selects a good default model based on Anthropic's currently published models, sends a prompt and prints the response as a string. 💡 Set ANTHROPIC_API_KEY.

func main() {
	ctx := context.Background()
	c, err := anthropic.New(ctx, genai.ModelGood)
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	msgs := genai.Messages{
		genai.NewTextMessage("Give me a life advice that sounds good but is a bad idea in practice. Answer succinctly."),
	}
	result, err := c.GenSync(ctx, msgs)
	fmt.Println(result.String())
}

This may print:

"Follow your passion and the money will follow."

This ignores market realities, financial responsibilities, and the fact that passion alone doesn't guarantee income or career viability.

Multiple Text Completions

examples/txt_to_txt_sync_multi/main.go: This shows how to do multiple message round trips adding additional follow-up messages from users. Set OPENAI_API_KEY.

func main() {
	ctx := context.Background()
	c, err := anthropic.New(ctx, genai.ModelGood)
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	msgs := genai.Messages{
		genai.NewTextMessage("Let's play a word association game. You pick a single word, then I pick the first word I think of, then you respond with a word, and so on.")
	}
	result, err := c.GenSync(ctx, msgs)
    if err != nil {
        panic(err)
    }
    // Show the message from ChatGPT
	fmt.Println(result.String())
    // Save the message in the collection of messages to build up context
    msgs = append(msgs, result.Message)
    // Add another user message
    msgs = append(msgs, genai.NewTextMessage("nightwish"))
    // Get another completion
    result, err := c.GenSync(ctx, msgs)
    // ...and so on.
}

Text Streaming 🏎

examples/txt_to_txt_stream/main.go: This is the same example as above, with the output streamed as it replies. This leverages go 1.23 iterators. Notice how little difference there is between both.

func main() {
	ctx := context.Background()
	c, err := anthropic.New(ctx, genai.ModelGood)
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	msgs := genai.Messages{
		genai.NewTextMessage("Give me a life advice that sounds good but is a bad idea in practice."),
	}
	fragments, finish := c.GenStream(ctx, msgs)
	for f := range fragments {
		os.Stdout.WriteString(f.Text)
	}
	_, err = finish()
}

Text Thinking 🧠

examples/txt_to_txt_thinking/main.go: genai supports for implicit reasoning (e.g. Anthropic) and explicit reasoning (e.g. Deepseek). The package adapters provide logic to automatically handle explicit Chain-of-Thoughts models, generally using <think> and </think> tokens. 💡 Set DEEPSEEK_API_KEY.

Snippet:

	c, err := deepseek.New(ctx, genai.ProviderOptionModel("deepseek-reasoner"))
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	msgs := genai.Messages{
		genai.NewTextMessage("Give me a life advice that sounds good but is a bad idea in practice."),
	}
	fragments, finish := c.GenStream(ctx, msgs)
	for f := range fragments {
		if f.Reasoning != "" {
			// ...
		} else if f.Text != "" {
			// ...
		}
	}

Text Citations ✍

examples/txt_to_txt_citations/main.go: Send entire documents and leverage providers which support automatic citations (Cohere, Anthropic) to leverage their functionality for a supercharged RAG. 💡 Set COHERE_API_KEY.

Snippet:

	const context = `...` // Introduction of On the Origin of Species by Charles Darwin...
	msgs := genai.Messages{{
		Requests: []genai.Request{
			{
				Doc: genai.Doc{
					Filename: "On-the-Origin-of-Species-by-Charles-Darwin.txt",
					Src:      strings.NewReader(context),
				},
			},
			{Text: "When did Darwin arrive home?"},
		},
	}}
	res, _ := c.GenSync(ctx, msgs)
	for _, r := range res.Replies {
		if !r.Citation.IsZero() {
			fmt.Printf("Citation:\n")
			for _, src := range r.Citation.Sources {
				fmt.Printf("- %q\n", src.Snippet)
			}
		}
	}
	fmt.Printf("\nAnswer: %s\n", res.String())

When asked When did Darwin arrive home? with the introduction of On the Origin of Species by Charles Darwin passed in as a document, this may print:

Citation:

• "excerpt from Charles Darwin's work 'On the Origin of Species'"
• "returned home in 1837."

Answer: 1837 was when Darwin returned home and began to reflect on the facts he had gathered during his time on H.M.S. Beagle.

Text Websearch 🕸️

examples/txt_to_txt_websearch-sync/main.go: Searches the web to answer your question. 💡 Set PERPLEXITY_API_KEY.

Snippet:

	c, err := perplexity.New(ctx, genai.ModelCheap)
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	msgs := genai.Messages{{
		Requests: []genai.Request{
			{Text: "Who holds ultimate power of Canada? Answer succinctly."},
		},
	}}

	// perplexity has websearch enabled by default so this is a no-op.
	//  It is needed to enable websearch for anthropic, gemini and openai.
	opts := genai.GenOptionWeb{Search: true, Fetch: true}
	res, _ := c.GenSync(ctx, msgs, &opts)
	for _, r := range res.Replies {
		if !r.Citation.IsZero() {
			fmt.Printf("Sources:\n")
			for _, src := range r.Citation.Sources {
				switch src.Type {
				case genai.CitationWeb:
					fmt.Printf("- %s / %s\n", src.Title, src.URL)
				case genai.CitationWebImage:
					fmt.Printf("- image: %s\n", src.URL)
				}
			}
		}
	}
	fmt.Printf("\nAnswer: %s\n", res.String())

Try it locally:

go run github.com/maruel/genai/examples/txt_to_txt_websearch-sync@latest

When asked Who holds ultimate power of Canada?, this may print:

Sources:

• Prime Minister of Canada / https://en.wikipedia.org/wiki/Prime_Minister_of_Canada

• Canadian Parliamentary System - Our Procedure / https://www.ourcommons.ca/procedure/our-procedure/parliamentaryFramework/c_g_parliamentaryframework-e.html

(...)

Image: https://learn.parl.ca/understanding-comprendre/images/articles/monarch-and-governor-general/house-of-commons.jpg

(...)

Answer: The ultimate power in Canada constitutionally resides with the monarch (King Charles III) as the head of state, with executive authority formally vested in him. However, (...)

Text Websearch (streaming) 🔍️

examples/txt_to_txt_websearch-stream/main.go: Searches the web to answer your question and streams the output to the console. 💡 Set PERPLEXITY_API_KEY.

go run github.com/maruel/genai/examples/txt_to_txt_websearch-stream@latest

Same as above, but streaming.

Log probabilities

examples/txt_to_txt_logprobs/main.go: List the alternative tokens that were considered during generation. This helps tune Temperature, TopP or TopK.

Try it locally:

go run github.com/maruel/genai/examples/txt_to_txt_logprobs@latest

When asked Tell a joke, this may print:

Provider huggingface
  Reply:
    Why don't scientists trust atoms?

    Because they make up everything!
  Logprobs:
    *    -0.000082: "Why"
         -9.625082: "Here"
        -11.250082: "What"
        -13.875082: "A"
        -14.500082: "How"
    *    -0.000003: " don"
        -14.125003: " do"
        -14.625003: " did"
        -14.625003: " dont"
        -14.875003: " didn"
    *    -0.000001: "'t"
        -14.000001: "’t"
        -18.062500: "'"
        -18.875000: "'T"
        -19.812500: "'s"
    *    -0.000002: " scientists"
        -14.250002: " Scientists"
        -14.250002: " eggs"
        -15.125002: " skeletons"
        -16.125002: " programmers"
    *    -0.000000: " trust"
        -16.250000: " trusts"
        -16.250000: " Trust"
        -17.250000: " like"
        -18.000000: " trusted"
    *    -0.000006: " atoms"
        -13.250006: "atoms"
        -13.500006: " stairs"
        -14.625006: " their"
        -15.000006: " electrons"
    *    -0.000011: "?\n\n"
        -12.125011: "?\n"
        -12.125011: "?"
        -14.750011: "？\n\n"
        -16.500011: " anymore"
(...)

Text Tools 🧰

examples/txt_to_txt_tool-sync/main.go: A LLM can both retrieve information and act on its environment through tool calling. This unblocks a whole realm of possibilities. Our design enables dense strongly typed code that favorably compares to python. 💡 Set CEREBRAS_API_KEY.

Snippet:

	type numbers struct {
		A int `json:"a"`
		B int `json:"b"`
	}
	msgs := genai.Messages{
		genai.NewTextMessage("What is 3214 + 5632? Call the tool \"add\" to tell me the answer. Do not explain. Be terse. Include only the answer."),
	}
	opts := genai.GenOptionTools{
		Tools: []genai.ToolDef{
			{
				Name:        "add",
				Description: "Add two numbers together and provides the result",
				Callback: func(ctx context.Context, input *numbers) (string, error) {
					return fmt.Sprintf("%d", input.A+input.B), nil
				},
			},
		},
		// Force the LLM to do a tool call.
		Force: genai.ToolCallRequired,
	}

	// Run the loop.
	res, _, _ := adapters.GenSyncWithToolCallLoop(ctx, c, msgs, &opts)
	// Print the answer which is the last message generated.
	fmt.Println(res[len(res)-1].String())

When asked What is 3214 + 5632?, this may print:

8846

Text Tools (streaming) 🐝

examples/txt_to_txt_tool-stream/main.go: Leverage a thinking model to see the thinking process while trying to use tool calls to answer the user's question. This enables keeping the user updated to see the progress. 💡 Set GROQ_API_KEY.

Snippet:

	fragments, finish := adapters.GenStreamWithToolCallLoop(ctx, p, msgs, &opts)
	for f := range fragments {
		if f.Reasoning != "" {
			// ...
		} else if f.Text != "" {
			// ...
		} else if !f.ToolCall.IsZero() {
			// ...
		}
	}

``

When asked What is 3214 + 5632?, this may print:

# Reasoning

User wants result of 3214+5632 using tool "add". Must be terse, only answer, no explanation. Need to call add function with a=3214, b=5632.

# Tool call

{fc_e9b9677b-898c-46df-9deb-39122bd6c69a add {"a":3214,"b":5632} map[] {}}

# Answer

8846

Text Tools (manual)

examples/txt_to_txt_tool-manual/main.go: Runs a manual loop and runs tool calls directly. 💡 Set CEREBRAS_API_KEY.

Snippet:

	res, _ := c.GenSync(ctx, msgs, &opts)
	// Add the assistant's message to the messages list.
	msgs = append(msgs, res.Message)
	// Process the tool call from the assistant.
	msg, _ := res.DoToolCalls(ctx, opts.Tools)
	// Add the tool call response to the messages list.
	msgs = append(msgs, msg)
	// Follow up so the LLM can interpret the tool call response.
	res, _ = c.GenSync(ctx, msgs, &opts)

Text Decode reply as a struct ⚙

examples/txt_to_txt_decode-json/main.go: Tell the LLM to use a specific Go struct to determine the JSON schema to generate the response. This is much more lightweight than tool calling!

It is very useful when we want the LLM to make a choice between values, to return a number or a boolean (true/false). Enums are supported. 💡 Set OPENAI_API_KEY.

Snippet:

	msgs := genai.Messages{
		genai.NewTextMessage("Is a circle round? Reply as JSON."),
	}
	var circle struct {
		Round bool `json:"round"`
	}
	opts := genai.GenOptionText{DecodeAs: &circle}
	res, _ := c.GenSync(ctx, msgs, &opts)
	res.Decode(&circle)
	fmt.Printf("Round: %v\n", circle.Round)

This will print:

Round: true

Text to Typed Decisions ❓

Ask yes/no, choice, or score questions about text or JSON state with Provider.SystemOne(). Set named questions in genai.Questions. Read typed answers under the same names from genai.Answers.

	questions := genai.Questions{
		"billing": {Type: genai.QuestionNoul, Instructions: genai.Text("Is this about billing?")},
	}
	res, err := c.SystemOne(ctx, &genai.SystemOneRequest{
		State: genai.Text("I was charged twice."), Questions: questions,
	})
	if err != nil {
		return err
	}
	fmt.Println(res.Answers["billing"].Noul)

A single inline JSON Doc supplies the state when State is unset. See the TypeSafe, Cloudflare, and llama.cpp for setup and constraints.

Complete examples: hosted text decisions, local text decisions, and local image and text decisions.

Text to Image 📸

examples/txt_to_img/main.go: Use Together.AI's free (!) image generation albeit with low rate limit.

Some providers return an URL that must be fetched manually within a few minutes or hours, some return the data inline. This example handles both cases. 💡 Set TOGETHER_API_KEY.

Snippet:

	msgs := genai.Messages{
		genai.NewTextMessage("Carton drawing of a husky playing on the beach."),
	}
	result, _ := c.GenSync(ctx, msgs)
	for _, r := range result.Replies {
		if r.Doc.IsZero() {
			continue
		}
		// The image can be returned as an URL or inline, depending on the provider.
		var src io.Reader
		if r.Doc.URL != "" {
			req, _ := c.HTTPClient().Get(r.Doc.URL)
			src = req.Body
			defer req.Body.Close()
		} else {
			src = r.Doc.Src
		}
		b, _ := io.ReadAll(src)
		os.WriteFile(r.Doc.GetFilename(), b, 0o644)
	}

Try it locally:

go run github.com/maruel/genai/examples/txt_to_img@latest

This may generate:

This generated picture shows a fake signature. I decided to keep this example as a reminder that the result comes from the data harvested that was created by real humans.

Image-Text to Video 🎥

examples/img-txt_to_vid/main.go: Leverage the content.jpg file generated in txt_to_img example to ask Veo 3 from Google to generate a video based on the image. 💡 Set GEMINI_API_KEY.

Snippet:

	// Warning: this is expensive.
	c, err := gemini.New(ctx, genai.ProviderOptionModel("veo-3.0-fast-generate-preview"))
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	f, _ := os.Open("content.jpg")
	defer f.Close()
	msgs := genai.Messages{
		genai.Message{Requests: []genai.Request{
			{Text: "Carton drawing of a husky playing on the beach."},
			{Doc: genai.Doc{Src: f}},
		}},
	}
	res, _ := c.GenSync(ctx, msgs)
	// Save the file in Replies like in the previous example ...

Try it locally:

go run github.com/maruel/genai/examples/img-txt_to_vid@latest

This may generate:

⚠ The MP4 has been recompressed to AVIF via compress.sh so GitHub can render it. The drawback is that audio is lost. View the original MP4 with audio (!) at content.mp4. May not work on Safari.

This is very impressive, but also very expensive.

Image-Text to Image 🖌

examples/img-txt_to_img/main.go: Edit an image with a prompt. Leverage the content.jpg file generated in txt_to_img example. 💡 Set BFL_API_KEY.

go run github.com/maruel/genai/examples/img-txt_to_img@latest

This may generate:

Image-Text to Image-Text 🍌

examples/img-txt_to_img-txt/main.go: Leverage the content.jpg file generated in txt_to_img example to ask gemini-2.5-flash-image-preview to change the image with a prompt and ask the model to explain what it did. 💡 Set GEMINI_API_KEY.

Snippet:

	// Warning: This is a bit expensive.
	c, err := gemini.New(ctx,
		genai.ProviderOptionModel("gemini-2.5-flash-image-preview"),
		genai.ProviderOptionModalities{genai.ModalityImage, genai.ModalityText},
	)
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	// ...
	res, _ := c.GenSync(ctx, msgs, &gemini.GenOption{ReasoningBudget: 0})

Try it locally:

go run github.com/maruel/genai/examples/img-txt_to_img-txt@latest

This may generate:

Of course! Here's an updated image with more animals. I added a playful dolphin jumping out of the water and a flock of seagulls flying overhead. I chose these animals to enhance the beach scene and create a more dynamic and lively atmosphere.

Wrote: content.png

This is quite impressive, but also quite expensive.

Image-Text to Text 👁

examples/img-txt_to_txt/main.go: Run vision to analyze a picture provided as an URL (source: wikipedia). The response is streamed out the console as the reply is generated. 💡 Set MISTRAL_API_KEY.

go run github.com/maruel/genai/examples/img-txt_to_txt@latest

This may generate:

The image depicts a single ripe banana. It has a bright yellow peel with a few small brown spots, indicating ripeness. The banana is curved, which is typical of its natural shape, and it has a stem at the top. The overall appearance suggests that it is ready to be eaten.

Image-Text to Text (local) 🏠

examples/img-txt_to_txt_local/main.go: is very similar to the previous example!

Use cmd/llama-serve to run a LLM locally, including tool calling and vision!

Start llama-server locally either by yourself or with this utility:

go run github.com/maruel/genai/cmd/llama-serve@latest \
  -model ggml-org/gemma-3-4b-it-GGUF/gemma-3-4b-it-Q8_0.gguf -- \
  --temp 1.0 --top-p 0.95 --top-k 64 \
  --jinja -fa -c 0 --no-warmup

Run vision 100% locally on CPU with only 8GB of RAM. No GPU required!

go run github.com/maruel/genai/examples/img-txt_to_txt_local@latest

Video-Text to Text 🎞️

examples/vid-txt_to_txt/main.go: Run vision to analyze a video. 💡 Set GEMINI_API_KEY.

Using this video:

Try it locally:

go run github.com/maruel/genai/examples/vid-txt_to_txt@latest

When asked What is the word, this generates:

Banana

Audio-Text to Text 🎤

examples/aud-txt_to_txt/main.go: Analyze an audio file. 💡 Set OPENAI_API_KEY.

Try it locally:

go run github.com/maruel/genai/examples/vid-txt_to_txt@latest

When asked What was the word?, this generates:

The word was "orange."

Usage and Quota 🍟🧀🥣

examples/txt_to_txt_quota/main.go: Prints the tokens processed and generated for the request and the remaining quota if the provider supports it. 💡 Set GROQ_API_KEY.

Snippet:

	msgs := genai.Messages{
		genai.NewTextMessage("Describe poutine as a French person who just arrived in Québec"),
	}
	res, _ := c.GenSync(ctx, msgs)
	fmt.Println(res.String())
	fmt.Printf("\nTokens usage: %s\n", res.Usage.String())

This may generate:

« Je viens tout juste d’arriver au Québec et, pour être honnête, je n’avais jamais entendu parler du fameux « poutine » avant de mettre le pied dans un petit resto du coin. »

(...)

Tokens usage: in: 83 (cached 0), reasoning: 0, out: 818, total: 901, requests/2025-08-29 15:58:13: 499999/500000, tokens/2025-08-29 15:58:12: 249916/250000

In addition to the token usage, remaining quota is printed.

Text with any provider ⁉

examples/txt_to_txt_any/main.go: Let the user chose the provider by name.

The relevant environment variable (e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY, etc) is used automatically for authentication.

Automatically selects a models on behalf of the user. Wraps the explicit thinking tokens if needed.

Supports ollama and llama-server even if they run on a remote host or non-default port.

Snippet:

	names := strings.Join(slices.Sorted(maps.Keys(providers.Available(ctx))), ", ")
	provider := flag.String("provider", "", "provider to use, "+names)
	flag.Parse()

	cfg := providers.All[*provider]
	c, err := cfg.Factory(ctx, genai.ModelGood)
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	p := adapters.WrapReasoning(c)
	res, _ := p.GenSync(...)

Try it locally:

go run github.com/maruel/genai/examples/txt_to_txt_any@latest \
    -provider cerebras \
    "Tell a good sounding advice that is a bad idea in practice."

OAuth2 Authentication 🔑

Two examples demonstrate full OAuth2 browser login flows with PKCE:

examples/txt_to_txt_oauth2_codex/main.go: Authenticate via OpenAI's auth server (same flow as Codex CLI) and talk to the ChatGPT backend API. Tokens are cached and refreshed automatically. No configuration needed.

go run github.com/maruel/genai/examples/txt_to_txt_oauth2_codex@latest

examples/txt_to_txt_oauth2_gemini/main.go: Authenticate via Google OAuth2 and talk to the Gemini API. Requires a Google Cloud project with OAuth2 Desktop credentials.

go run github.com/maruel/genai/examples/txt_to_txt_oauth2_gemini@latest -client-id=ID -client-secret=SECRET

Embeddings

genai.Provider.Embed supports Gemini, llama.cpp, Ollama, OpenAI Chat and OpenAI Responses clients. Other providers return base.ErrNotSupported. Configure an explicit embedding model with genai.ProviderOptionModel; automatic generation model selection does not choose an embedding model.

c, err := gemini.New(ctx, genai.ProviderOptionModel("gemini-embedding-2"))
if err != nil {
    return err
}
// Close c when finished.
out, err := c.Embed(ctx, &genai.EmbeddingRequest{
    Inputs: []genai.Request{{Text: "A kitten plays with yarn."}, {Text: "A cat plays with string."}},
})
if err != nil {
    return err
}
// out.Embeddings[i] corresponds to the ith input request.

See the text, image and audio embedding example.

Model Selection

For automatic model selection, pass one of the marker constants directly:

// Use genai.ModelCheap for the cheapest model, genai.ModelGood for a good
// everyday model, or genai.ModelSOTA for the state-of-the-art model.
c, err := anthropic.New(ctx, genai.ModelGood)
if c != nil {
	defer func() {
		if err := c.Close(); err != nil {
			log.Printf("Close: %v", err)
		}
	}()
}
if err != nil {
	log.Print(err)
	return
}

For a specific model, wrap the model ID with ProviderOptionModel:

// Specific model selection
c, err := anthropic.New(ctx, genai.ProviderOptionModel("claude-sonnet-4-20250514"))
if c != nil {
	defer func() {
		if err := c.Close(); err != nil {
			log.Printf("Close: %v", err)
		}
	}()
}
if err != nil {
	log.Print(err)
	return
}

For Gemini, the same option selects a Gemini model:

c, err := gemini.New(ctx, genai.ProviderOptionModel("gemini-2.5-flash"))
if c != nil {
	defer func() {
		if err := c.Close(); err != nil {
			log.Printf("Close: %v", err)
		}
	}()
}
if err != nil {
	log.Print(err)
	return
}

Models 🗒

Snapshot of all the supported models at docs/MODELS.md is updated weekly.

Try it locally:

go install github.com/maruel/genai/cmd/...@latest

list-models -provider huggingface

Providers with free tier 💸

As of August 2025, the following services offer a free tier (other limits apply):

• Cerebras has unspecified "generous" free tier
• Cloudflare Workers AI about 10k tokens/day
• Cohere (1000 RPCs/month)
• Google's Gemini 0.25qps, 1m tokens/month
• Groq 0.5qps, 500k tokens/day
• HuggingFace 10¢/month
• Mistral 1qps, 1B tokens/month
• Pollinations.ai provides many models for free, including image generation
• Together.AI provides many models for free at 1qps, including image generation
• Running Ollama or llama.cpp locally is free. :)

TODO

PRs are appreciated for any of the following. No need to ask! Just send a PR and make it pass CI checks. ❤️

Features

• Server-side MCP Client: OpenAI
  • Anthropic raw API is implemented and smoke tested but there's no abstraction layer yet
• Real-time / Live: Gemini, OpenAI, TogetherAI, ...
• More comprehensive file/cache abstraction
• Tokens counting: Anthropic, Cohere, Gemini, ...
• Image to 3D, e.g. github.com/Tencent-Hunyuan/Hunyuan3D-2

Providers

I'd be delighted if you want to contribute any missing provider being added, I'm particularly looking forward to these:

• AWS Bedrock
• Azure AI
• Fireworks responses
• Google's Vertex AI: It supports much more features than Gemini API.
• Groq
  • Speech to Text (STT)
  • Text to Speech (TTS)
  • Batch
• LM Studio: Easier way to run local models.
• [ ] Mistral
  • [ ] Batch
  • Fill in the Middle (FIM)
  • Transcription
• Novita: Supports lots of modalities.
• OpenAI
  • Audio
  • Batch Responses API
  • [ ] Files
  • Image streaming
• Runway: Specialized in images and videos.
• Synexai: It's very cheap.
• vLLM: The fastest way to run local models.

I'm also looking to disconnect more the scoreboard from the Go code. I believe the scoreboard is useful in itself and is not Go specific. I appreciate ideas towards achieving this, send them my way!

Thanks in advance! 🙏

Made with ❤️ by Marc-Antoine Ruel

Documentation

Overview

Package genai is the opiniated high performance professional-grade AI package for Go.

It provides a generic interface to interact with various LLM providers, while allowing full access to each provider's full capabilities.

Check out the examples for a quick start.

Example (GenSyncWithToolCallLoop_with_custom_HTTP_Header)

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"time"

	"github.com/maruel/roundtrippers"

	"github.com/maruel/genai"
	"github.com/maruel/genai/adapters"
	"github.com/maruel/genai/providers/anthropic"
)

func main() {
	// Modified version of the example in package adapters, with a custom header.
	//
	// As of June 2025, interleaved thinking can be enabled with a custom header.
	// https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#interleaved-thinking
	wrapper := func(h http.RoundTripper) http.RoundTripper {
		return &roundtrippers.Header{
			Transport: h,
			Header:    http.Header{"anthropic-beta": []string{"interleaved-thinking-2025-05-14"}},
		}
	}
	ctx := context.Background()
	c, err := anthropic.New(ctx, genai.ProviderOptionModel("claude-sonnet-4-20250514"), genai.ProviderOptionTransportWrapper(wrapper))
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	msgs := genai.Messages{genai.NewTextMessage("What season is Montréal currently in?")}
	opts := genai.GenOptionTools{
		Tools: []genai.ToolDef{locationClockTime},
		// Force the LLM to do a tool call first.
		Force: genai.ToolCallRequired,
	}
	newMsgs, _, err := adapters.GenSyncWithToolCallLoop(ctx, c, msgs, &opts)
	if err != nil {
		log.Print(err)
		return
	}
	fmt.Printf("%s\n", newMsgs[len(newMsgs)-1].String())
}

var locationClockTime = genai.ToolDef{
	Name:        "get_today_date_current_clock_time",
	Description: "Get the current clock time and today's date.",
	Callback: func(ctx context.Context, e *location) (string, error) {
		if e.Location != "Montréal" {
			return "ask again with Montréal", nil
		}
		return time.Now().Format("Monday 2006-01-02 15:04:05"), nil
	},
}

type location struct {
	Location string `json:"location" json_description:"Location to ask the current time in"`
}

Index

• Constants
• type Answer
  • func (a *Answer) MarshalJSON() ([]byte, error)
  • func (a *Answer) UnmarshalJSON(b []byte) error
• type Answers
• type Array
  • func (a Array) Validate() error
• type CacheEntry
• type Citation
  • func (c *Citation) IsZero() bool
  • func (c *Citation) Validate() error
• type CitationSource
  • func (cs *CitationSource) IsZero() bool
  • func (cs *CitationSource) Validate() error
• type CitationType
• type DecisionContent
  • func ParseDecisionContent(b []byte) (DecisionContent, error)
• type DecisionUsage
  • func (u *DecisionUsage) Validate() error
• type Doc
  • func (d *Doc) GetFilename() string
  • func (d *Doc) IsZero() bool
  • func (d *Doc) MarshalJSON() ([]byte, error)
  • func (d *Doc) Read(maxSize int64) (string, []byte, error)
  • func (d *Doc) UnmarshalJSON(b []byte) error
  • func (d *Doc) Validate() error
• type EmbeddingRequest
  • func (r *EmbeddingRequest) Validate() error
• type EmbeddingResponse
  • func (r *EmbeddingResponse) Validate() error
• type FinishReason
• type GenOption
• type GenOptionAudio
  • func (o *GenOptionAudio) Validate() error
• type GenOptionImage
  • func (o *GenOptionImage) Validate() error
• type GenOptionPollInterval
  • func (p GenOptionPollInterval) Validate() error
• type GenOptionSeed
  • func (s GenOptionSeed) Validate() error
• type GenOptionText
  • func (o *GenOptionText) DecodeSchema() (JSONSchema, error)
  • func (o *GenOptionText) Validate() error
• type GenOptionTools
  • func (o *GenOptionTools) Validate() error
• type GenOptionVideo
  • func (o *GenOptionVideo) Validate() error
• type GenOptionWeb
  • func (o *GenOptionWeb) Validate() error
• type JSONSchema
• type Job
• type Logprob
  • func (l *Logprob) GoString() string
  • func (l *Logprob) Validate() error
• type Message
  • func NewTextMessage(text string) Message
  • func (m *Message) Accumulate(mf *Reply) error
  • func (m *Message) Decode(x any) error
  • func (m *Message) DoToolCalls(ctx context.Context, tools []ToolDef) (Message, error)
  • func (m *Message) GoString() string
  • func (m *Message) IsZero() bool
  • func (m *Message) Reasoning() string
  • func (m *Message) Role() string
  • func (m *Message) String() string
  • func (m *Message) UnmarshalJSON(b []byte) error
  • func (m *Message) Validate() error
• type Messages
  • func (m Messages) Validate() error
• type Modalities
  • func (m Modalities) String() string
  • func (m Modalities) Validate() error
• type Modality
• type Model
• type NoulCriteria
  • func (c *NoulCriteria) Validate() error
• type Object
  • func (o Object) Validate() error
• type Provider
• type ProviderCapabilities
• type ProviderOption
• type ProviderOptionAPIKey
  • func (p ProviderOptionAPIKey) Validate() error
• type ProviderOptionModalities
  • func (p ProviderOptionModalities) Validate() error
• type ProviderOptionModel
  • func (p ProviderOptionModel) Validate() error
• type ProviderOptionPreloadedModels
  • func (p ProviderOptionPreloadedModels) Validate() error
• type ProviderOptionRemote
  • func (p ProviderOptionRemote) Validate() error
• type ProviderOptionStarterWrapper
  • func (p ProviderOptionStarterWrapper) Validate() error
• type ProviderOptionTransportWrapper
  • func (p ProviderOptionTransportWrapper) Validate() error
• type ProviderPing
• type ProviderScoreboardVariants
• type ProviderUnwrap
• type Question
  • func (q *Question) MarshalJSON() ([]byte, error)
  • func (q *Question) Validate() error
• type QuestionType
• type Questions
  • func (q Questions) Validate() error
• type RateLimit
  • func (r *RateLimit) String() string
  • func (r *RateLimit) Validate() error
• type RateLimitPeriod
• type RateLimitType
• type Reply
  • func (r *Reply) GoString() string
  • func (r *Reply) IsZero() bool
  • func (r *Reply) UnmarshalJSON(b []byte) error
  • func (r *Reply) Validate() error
• type Request
  • func (r *Request) UnmarshalJSON(b []byte) error
  • func (r *Request) Validate() error
• type Result
  • func (r *Result) Validate() error
• type ScoreLegend
  • func (l *ScoreLegend) UnmarshalJSON(b []byte) error
• type ScoreboardVariant
• type Starter
• type SystemOneRequest
  • func (r *SystemOneRequest) ReadState(maxDocBytes int64) (DecisionContent, []Doc, error)
  • func (r *SystemOneRequest) Validate() error
• type SystemOneResponse
  • func (r *SystemOneResponse) ValidateQuestions(qs *Questions) error
• type Text
  • func (Text) Validate() error
• type ToolCall
  • func (t *ToolCall) Call(ctx context.Context, tools []ToolDef) (string, error)
  • func (t *ToolCall) IsZero() bool
  • func (t *ToolCall) UnmarshalJSON(b []byte) error
  • func (t *ToolCall) Validate() error
• type ToolCallRequest
• type ToolCallResult
  • func (t *ToolCallResult) UnmarshalJSON(b []byte) error
  • func (t *ToolCallResult) Validate() error
• type ToolDef
  • func (t *ToolDef) GetInputSchema() (JSONSchema, error)
  • func (t *ToolDef) Validate() error
• type Usage
  • func (u *Usage) Add(r *Usage)
  • func (u *Usage) String() string

Examples

• Package (GenSyncWithToolCallLoop_with_custom_HTTP_Header)
• Provider (Embed)
• Provider (GenSync_audio)
• Provider (GenSync_pdf)
• Provider (GenSync_video)
• Provider (GenSync_vision)
• Questions

Constants

const (
	// ModalityAudio is support for audio formats like MP3, WAV, Opus, Flac, etc.
	ModalityAudio = scoreboard.ModalityAudio
	// ModalityDecision is typed answers returned by SystemOne.
	ModalityDecision = scoreboard.ModalityDecision
	// ModalityDocument is support for PDF with multi-modal comprehension, both images and text. This includes
	// code blocks.
	ModalityDocument = scoreboard.ModalityDocument
	// ModalityEmbedding is a vector returned by Embed.
	ModalityEmbedding = scoreboard.ModalityEmbedding
	// ModalityImage is support for image formats like PNG, JPEG, often single frame GIF, and WEBP.
	ModalityImage = scoreboard.ModalityImage
	// ModalityText is for raw text.
	ModalityText = scoreboard.ModalityText
	// ModalityVideo is support for video formats like MP4 or MKV.
	ModalityVideo = scoreboard.ModalityVideo
)

Types

type Answer

type Answer struct {
	// Type is the type of the question this is the answer to.
	Type QuestionType `json:"type"`
	// Noul is the probability that the answer is yes, from 0 to 1. It is set for QuestionNoul.
	Noul float64 `json:"noul,omitzero"`
	// Choice is the highest probability option. It is set for QuestionChoice.
	Choice string `json:"choice,omitzero"`
	// Score is the probability weighted value across the levels. It can land between levels. It is set for
	// QuestionScore.
	Score float64 `json:"score,omitzero"`
	// Confidence is how certain the model is, derived from Probabilities. It is set for QuestionChoice and
	// QuestionScore.
	Confidence float64 `json:"confidence,omitzero"`
	// Probabilities maps each option, or each level as a string key, to its probability. It is set for
	// QuestionChoice and QuestionScore.
	Probabilities map[string]float64 `json:"probabilities,omitzero"`
	// Legend maps each level as a string key to the description passed in Question.Score for that level.
	// It is set for QuestionScore.
	Legend ScoreLegend `json:"legend,omitzero"`
	// contains filtered or unexported fields
}

Answer is the answer to a Question.

It is a union discriminated by Type. Only the fields that match Type are set, and MarshalJSON writes only those fields, with the shape the API uses, so a probability or a score of 0 is not dropped.

An answer type the server adds later is kept as is instead of making the whole reply unusable.

func (*Answer) MarshalJSON

func (a *Answer) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler.

The fields of the union that do not match Type are omitted, even when they are zero, so that a score of 0 or a probability of 0 is preserved.

func (*Answer) UnmarshalJSON

func (a *Answer) UnmarshalJSON(b []byte) error

UnmarshalJSON implements json.Unmarshaler.

type Answers

type Answers map[string]*Answer

Answers holds the answer to each question, keyed by the question name. Entries must not be nil.

type Array

type Array []any

Array is DecisionContent that is a JSON array.

The values are any, like Object: they must be JSON encodable. They are not restricted to DecisionContent items.

func (Array) Validate

func (a Array) Validate() error

Validate implements internal.Validatable.

Every item is validated so that the error names the offending index.

type CacheEntry

type CacheEntry interface {
	GetID() string
	GetDisplayName() string
	GetExpiry() time.Time
}

CacheEntry is one file (or GenSync request) cached on the provider for reuse.

type Citation

type Citation struct {
	// CitedText is the text that was cited.
	CitedText string `json:"cited_text,omitzero"`
	// StartIndex is the starting character position of the citation in the answer (0-based).
	StartIndex int64 `json:"start_index,omitzero"`
	// EndIndex is the ending character position of the citation in the answer (0-based, exclusive).
	EndIndex int64 `json:"end_index,omitzero"`

	// Sources contains information about the source documents or tools that support this citation.
	Sources []CitationSource `json:"sources,omitzero"`
	// contains filtered or unexported fields
}

Citation represents a reference to source material that supports content. It provides a unified interface for different provider citation formats.

Normally one of CitedText or StartIndex/EndIndex is set.

func (*Citation) IsZero

func (c *Citation) IsZero() bool

IsZero returns true if the citation is empty.

func (*Citation) Validate

func (c *Citation) Validate() error

Validate ensures the citation is valid.

type CitationSource

type CitationSource struct {
	// Type indicates the source type.
	Type CitationType `json:"type,omitzero"`
	// ID is a unique identifier for the source (e.g., document ID, tool call ID).
	ID string `json:"id,omitzero"`
	// Title is the human-readable title of the source.
	Title string `json:"title,omitzero"`
	// URL is the web URL for the source, if applicable.
	URL string `json:"url,omitzero"`
	// Snippet is a snippet from the source, if applicable. It is the web search query for CitationWebQuery.
	Snippet string `json:"snippet,omitzero"`
	// StartCharIndex is the starting character position of the citation in the sourced document (0-based).
	StartCharIndex int64 `json:"start_index,omitzero"`
	// EndCharIndex is the ending character position of the citation in the sourced document (0-based, exclusive).
	EndCharIndex int64 `json:"end_index,omitzero"`
	// StartPageNumber is the starting page number of the citation in the sourced document (1-based).
	StartPageNumber int64 `json:"start_page_number,omitzero"`
	EndPageNumber   int64 `json:"end_page_number,omitzero"`
	// StartBlockIndex is the starting block index of the citation in the sourced document (0-based).
	StartBlockIndex int64 `json:"start_block_index,omitzero"`
	EndBlockIndex   int64 `json:"end_block_index,omitzero"`

	// Date is the date of the source, if applicable.
	Date string `json:"date,omitzero"`
	// Metadata contains additional source-specific information.
	// For document sources: document index, page numbers, etc.
	// For tool sources: tool output, function name, etc.
	// For web sources: encrypted index, search result info, etc.
	Metadata map[string]any `json:"metadata,omitzero"`
	// contains filtered or unexported fields
}

CitationSource represents a source that supports a citation.

func (*CitationSource) IsZero

func (cs *CitationSource) IsZero() bool

IsZero returns true if the citation source is empty.

func (*CitationSource) Validate

func (cs *CitationSource) Validate() error

Validate ensures the citation source is valid.

type CitationType

type CitationType int32

CitationType is a citation that a model returned as part of its reply.

const (
	// CitationWebQuery is an query used as part of a web search.
	CitationWebQuery CitationType = iota + 1
	// CitationWeb is an URL from a web search.
	CitationWeb
	// CitationWebImage is an URL to an image from a web search.
	CitationWebImage
	// CitationDocument is from a document provided as input or explicitly referenced.
	CitationDocument
	// CitationTool is when the provider refers to the result of a tool call in its answer.
	CitationTool
)

type DecisionContent

type DecisionContent interface {
	// Validate ensures the content is valid.
	Validate() error
	// contains filtered or unexported methods
}

DecisionContent is text, a JSON object, or a JSON array.

It is implemented by Text, Object and Array, and by JSON state read from a Doc. It marshals to the value itself, not to an object with fields. Use nil when the field is optional and is left unset.

It is used for state, instructions and criteria descriptions. It is meant to be sent; the API only returns it in Answer.Legend, decoded back by ScoreLegend, since an interface cannot be decoded on its own.

func ParseDecisionContent

func ParseDecisionContent(b []byte) (DecisionContent, error)

ParseDecisionContent decodes one JSON value into its DecisionContent variant.

The variant is picked from the JSON token, not from the Go value a decoder would produce for an any, so what it accepts is unambiguous.

type DecisionUsage

type DecisionUsage struct {
	// InputTokens counts the prompt tokens evaluated for all questions.
	InputTokens int64 `json:"input_tokens"`
	// OutputTokens counts output tokens reported by the provider, often zero for decision models.
	OutputTokens int64 `json:"output_tokens"`
	// ReasoningTokens counts reasoning tokens within OutputTokens, when reported by the provider.
	ReasoningTokens int64 `json:"reasoning_tokens,omitzero"`
}

DecisionUsage reports the token usage of a request.

func (*DecisionUsage) Validate

func (u *DecisionUsage) Validate() error

Validate rejects negative token counts.

type Doc

type Doc struct {
	// Filename is the name of the file. For many providers, only the extension
	// is relevant. They only use mime-type, which is derived from the filename's
	// extension. When an URL is provided or when the object provided to Document
	// implements a method with the signature `Name() string`, like an
	// `*os.File`, Filename is optional.
	Filename string `json:"filename,omitzero"`
	// Src is raw document data. It is perfectly fine to use a bytes.NewReader() or *os.File.
	Src io.ReadSeeker `json:"bytes,omitzero"`
	// URL is the reference to the raw data. When set, the mime-type is derived from the URL.
	URL string `json:"url,omitzero"`
	// contains filtered or unexported fields
}

Doc is a document.

func (*Doc) GetFilename

func (d *Doc) GetFilename() string

GetFilename returns the filename to use for the document, querying the Document's name if available.

func (*Doc) IsZero

func (d *Doc) IsZero() bool

IsZero returns true if the document is empty.

func (*Doc) MarshalJSON

func (d *Doc) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

func (*Doc) Read

func (d *Doc) Read(maxSize int64) (string, []byte, error)

Read reads the document content into memory.

maxSize must be positive. Oversized content is rejected, including unseekable sources and content that grows after its size is checked. It returns the mime type, the raw bytes and an error if any.

func (*Doc) UnmarshalJSON

func (d *Doc) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

func (*Doc) Validate

func (d *Doc) Validate() error

Validate ensures the block is valid.

type EmbeddingRequest

type EmbeddingRequest struct {
	// Inputs contains one text or document block per output vector.
	Inputs []Request `json:"inputs"`
	// Dimensions is the requested vector size, or zero for the model default.
	// Support depends on the provider and model.
	Dimensions int `json:"dimensions,omitzero"`
}

EmbeddingRequest embeds a batch of content with the client's configured model. Include any task prefixes required by that model in each request's Text.

func (*EmbeddingRequest) Validate

func (r *EmbeddingRequest) Validate() error

Validate implements Validatable.

type EmbeddingResponse

type EmbeddingResponse struct {
	Embeddings [][]float32 `json:"embeddings"`
	// Usage is zero when the provider doesn't report token counts.
	Usage Usage `json:"usage,omitzero"`
}

EmbeddingResponse contains one vector per input request, in input order. Values are preserved as returned by the provider. Normalization depends on the provider and model; callers comparing vectors must account for that contract.

func (*EmbeddingResponse) Validate

func (r *EmbeddingResponse) Validate() error

Validate implements Validatable. It checks uniform dimensions and finite, nonzero vectors without normalizing their values.

type FinishReason

type FinishReason string

FinishReason is the reason why the model stopped generating tokens.

It can be one of the well known below or a custom value.

const (
	// FinishedStop means the LLM was done for the turn. Some providers confuse it with
	// FinishedStopSequence.
	FinishedStop FinishReason = "stop"
	// FinishedLength means the model reached the maximum number of tokens allowed as set in
	// GenOptionText.MaxTokens or as limited by the provider.
	FinishedLength FinishReason = "length"
	// FinishedToolCalls means the model called one or multiple tools and needs the replies to continue the turn.
	FinishedToolCalls FinishReason = "tool_calls"
	// FinishedStopSequence means the model stopped because it saw a stop word as listed in GenOptionText.Stop.
	FinishedStopSequence FinishReason = "stop"
	// FinishedContentFilter means the model stopped because the reply got caught by a content filter.
	FinishedContentFilter FinishReason = "content_filter"
	// Pending means that it's not finished yet. For use with ProviderGenAsync.
	Pending FinishReason = "pending"
)

type GenOption

type GenOption interface {
	// Validate ensures the options object is valid.
	Validate() error
}

GenOption is options that can be provided to Provider.GenSync and Provider.GenStream.

type GenOptionAudio

type GenOptionAudio struct {
	// contains filtered or unexported fields
}

GenOptionAudio specifies audio generation options.

func (*GenOptionAudio) Validate

func (o *GenOptionAudio) Validate() error

Validate implements Validatable.

type GenOptionImage

type GenOptionImage struct {
	Width  int
	Height int
	// contains filtered or unexported fields
}

GenOptionImage is a list of frequent options supported by most ProviderDoc. Each provider is free to support more options through a specialized struct.

func (*GenOptionImage) Validate

func (o *GenOptionImage) Validate() error

Validate ensures the completion options are valid.

type GenOptionPollInterval

type GenOptionPollInterval time.Duration

GenOptionPollInterval is the time interval to poll generation progress when using GenSync.

func (GenOptionPollInterval) Validate

func (p GenOptionPollInterval) Validate() error

Validate ensures the poll interval is valid.

type GenOptionSeed

type GenOptionSeed int64

GenOptionSeed is the seed for the random number generator.

func (GenOptionSeed) Validate

func (s GenOptionSeed) Validate() error

Validate ensures the seed is valid.

type GenOptionText

type GenOptionText struct {
	// Temperature adjust the creativity of the sampling. Generally between 0 and 2.
	Temperature float64
	// TopP adjusts correctness sampling between 0 and 1. The higher the more diverse the output.
	TopP float64
	// MaxTokens is the maximum number of tokens to generate. Used to limit it
	// lower than the default maximum, for budget reasons.
	MaxTokens int64
	// TopLogprobs requests to return the top logprobs in the reply.
	TopLogprobs int64
	// SystemPrompt is the prompt to use for the system role.
	SystemPrompt string

	// TopK adjusts sampling where only the N first candidates are considered.
	TopK int64
	// Stop is the list of tokens to stop generation.
	Stop []string

	// ReplyAsJSON enforces the output to be valid JSON, any JSON. It is
	// important to tell the model to reply in JSON in the prompt itself.
	ReplyAsJSON bool
	// DecodeAs enforces a reply with a specific JSON structure. It must be either a JSON object or array
	// that can be decoded by encoding/json and can have jsonschema tags, usually a pointer to a struct, or
	// a JSONSchema.
	//
	// Not every provider accepts an array at the root of the reply, e.g. OpenAI and Anthropic require an
	// object. Wrap the slice in a struct with a single field when the provider rejects it.
	//
	// It is important to request the model to "reply in JSON" in the prompt itself.
	//
	// If you use a JSONSchema, it will be used to validate the reply.
	//
	// If you use a pointer to a struct, it is recommended to use jsonschema_description tags to describe each
	// field or argument. Use jsonschema:"enum=..." to enforce a specific value within a set. Use omitempty to
	// make the field optional. See https://github.com/invopop/jsonschema#example for more examples.
	DecodeAs any
	// contains filtered or unexported fields
}

GenOptionText is a list of frequent options supported by most Provider with text output modality. Each provider is free to support more options through a specialized struct.

The first group are options supported by (nearly) all providers.

The second group are options supported only by some providers. Using them may cause the chat operation to return a base.ErrNotsupported.

The third group are options supported by a few providers and a few models on each, that will slow down generation (increase latency) and will increase token use (cost).

func (*GenOptionText) DecodeSchema

func (o *GenOptionText) DecodeSchema() (JSONSchema, error)

DecodeSchema returns the JSONSchema for the DecodeAs field.

If DecodeAs is a JSONSchema instance, it is returned directly. Otherwise it uses invopop/jsonschema to construct the schema from the type's fields and tags.

func (*GenOptionText) Validate

func (o *GenOptionText) Validate() error

Validate ensures the completion options are valid.

type GenOptionTools

type GenOptionTools struct {
	// Tools is the list of tools that the LLM can request to call.
	Tools []ToolDef
	// Force tells the LLM a tool call must be done, or not.
	Force ToolCallRequest
}

GenOptionTools specifies tool calling options.

func (*GenOptionTools) Validate

func (o *GenOptionTools) Validate() error

Validate ensures the completion options are valid.

type GenOptionVideo

type GenOptionVideo struct {
	// Duration of the video to generate, if supported.
	//
	// Veo 2 supports only between 5 and 8 seconds and Veo 3 only supports 8 seconds.
	Duration time.Duration
	// contains filtered or unexported fields
}

GenOptionVideo specifies video generation options.

func (*GenOptionVideo) Validate

func (o *GenOptionVideo) Validate() error

Validate implements Validatable.

type GenOptionWeb

type GenOptionWeb struct {
	// Search specifies if websearch should be enabled. It is generally disabled by default except for
	// perplexity.
	//
	// # Warning
	//
	// This will become a structure to provide information about included and excluded domains, and the user's
	// location.
	Search bool
	// Fetch specifies if web fetch should be enabled. When enabled, the LLM can fetch content from URLs.
	//
	// Currently supported by Anthropic and Gemini.
	Fetch bool
}

GenOptionWeb specifies web access options.

func (*GenOptionWeb) Validate

func (o *GenOptionWeb) Validate() error

Validate implements GenOption.

type JSONSchema

type JSONSchema = json.RawMessage

JSONSchema is a JSON Schema document encoded as raw JSON bytes.

type Job

type Job string

Job is a pending job.

type Logprob

type Logprob struct {
	ID      int64   `json:"id,omitzero"`   // Input token ID.
	Text    string  `json:"text,omitzero"` // Text in UTF-8.
	Logprob float64 `json:"logprob"`       // Log probability of the token. It should normally be non-zero but sometimes it is.
}

Logprob represents a single log probability information for a token.

One of ID or Text must be set.

func (*Logprob) GoString

func (l *Logprob) GoString() string

GoString returns a JSON representation of the reply for debugging purposes.

func (*Logprob) Validate

func (l *Logprob) Validate() error

Validate ensures the logprob is valid.

type Message

type Message struct {
	// Requests is the content from the user.
	//
	// It is normally a single message. It is more frequently multiple items when using multi-modal content.
	Requests []Request `json:"request,omitzero"`
	// User must only be used when sent by the user. Only some provider (e.g. OpenAI, Groq, DeepSeek) support it.
	User string `json:"user,omitzero"`

	// Replies is the message from the LLM.
	//
	// It is generally a single reply with text or a tool call. Some models can emit multiple content blocks,
	// either multi modal or multiple text blocks: a code block and a different block with an explanantion. Some
	// models can emit multiple tool calls at once.
	Replies []Reply `json:"reply,omitzero"`

	// ToolCallResult is the result for a tool call that the LLM requested to make.
	//
	// These messages are generated by the "computer".
	ToolCallResults []ToolCallResult `json:"tool_call_results,omitzero"`
	// contains filtered or unexported fields
}

Message is a part of an exchange with a LLM.

It is effectively a union, with the exception of the User field that can be set with In.

func NewTextMessage

func NewTextMessage(text string) Message

NewTextMessage is a shorthand function to create a Message with a single text block.

func (*Message) Accumulate

func (m *Message) Accumulate(mf *Reply) error

Accumulate adds a Reply to the message being streamed.

It is used by GenStream. There's generally no need to call it by end users.

func (*Message) Decode

func (m *Message) Decode(x any) error

Decode decodes the JSON message into the struct.

For generation replies, request JSON with ReplyAsJSON or DecodeAs in GenOptionText.

Note: this doesn't verify the type is the same as specified in GenOptionText.DecodeAs.

func (*Message) DoToolCalls

func (m *Message) DoToolCalls(ctx context.Context, tools []ToolDef) (Message, error)

DoToolCalls processes all the ToolCall in the Reply if any.

Returns a Message to be added back to the list of messages, only if msg.IsZero() is true.

func (*Message) GoString

func (m *Message) GoString() string

GoString returns a JSON representation of the reply for debugging purposes.

func (*Message) IsZero

func (m *Message) IsZero() bool

IsZero returns true if the message is empty.

func (*Message) Reasoning

func (m *Message) Reasoning() string

Reasoning returns all the reasoning concatenated, if any.

func (*Message) Role

func (m *Message) Role() string

Role returns one of "user", "assistant" or "computer".

func (*Message) String

func (m *Message) String() string

String is a short hand to get the request or reply content as text.

It ignores reasoning or multi-modal content.

func (*Message) UnmarshalJSON

func (m *Message) UnmarshalJSON(b []byte) error

UnmarshalJSON adds validation during decoding.

func (*Message) Validate

func (m *Message) Validate() error

Validate ensures the message is valid.

type Messages

type Messages []Message

Messages is a chronological list of messages in an exchange with a LLM.

Adjacent messages may have the same role. Tool calls may remain unresolved, and user messages and other tool results may intervene before their results arrive. Providers may impose additional constraints. Conversion must preserve event order without merging messages or inventing tool results.

func (Messages) Validate

func (m Messages) Validate() error

Validate ensures the messages are valid.

type Modalities

type Modalities []scoreboard.Modality

Modalities represents the modality supported by the provider in a specific scenario. It can be multiple modalities in multi-modals scenarios.

func (Modalities) String

func (m Modalities) String() string

func (Modalities) Validate

func (m Modalities) Validate() error

Validate implements Validatable.

type Modality

type Modality = scoreboard.Modality

Modality is a modality supported by the provider.

It is aliased from scoreboard.Modality to avoid a circular dependency.

type Model

type Model interface {
	GetID() string
	String() string
	// Context returns the number of tokens the model can process as input.
	Context() int64
}

Model represents a served model by the provider.

Use Provider.ListModels() to get a list of models.

type NoulCriteria

type NoulCriteria struct {
	// True describes what a yes (value near 1) means.
	True DecisionContent `json:"true,omitzero"`
	// False describes what a no (value near 0) means.
	False DecisionContent `json:"false,omitzero"`
}

NoulCriteria describes what the yes and the no answers of a noul question mean.

func (*NoulCriteria) Validate

func (c *NoulCriteria) Validate() error

Validate ensures the criteria are valid.

type Object

type Object map[string]any

Object is DecisionContent that is a JSON object.

The values are any, like Reply.Opaque: they must be JSON encodable. Go has no recursive JSON value type that does not require wrapping every scalar, so a nested map[string]any, []any or a struct with JSON tags is passed as is. It is the equivalent of the SDKs' Mapping[str, JSONValue | None].

func (Object) Validate

func (o Object) Validate() error

Validate implements internal.Validatable.

Every value is validated so that the error names the offending key.

type Provider

type Provider interface {
	// Close releases resources held by the provider.
	io.Closer

	// Name returns the name of the provider.
	Name() string
	// ModelID returns the model currently used by the provider. It can be an empty string.
	ModelID() string
	// OutputModalities returns the output modalities supported by this specific client configuration.
	//
	// This states what kind of output the model will generate (text, audio, image, video, decision). It varies per
	// provider and models. The vast majority of providers and models support only output modality like
	// text-only, image-only, etc.
	OutputModalities() Modalities
	// Capabilities returns the optional capabilities this provider supports.
	Capabilities() ProviderCapabilities
	// Scoreboard returns what the provider supports.
	//
	// Some models have more features than others, e.g. some models may be text-only while others have vision or
	// audio support.
	//
	// The client code may be the limiting factor for some models, and not the provider itself.
	//
	// The values returned here should have gone through a smoke test to make sure they are valid.
	Scoreboard() scoreboard.Score
	// HTTPClient returns the underlying http client. It may be necessary to use it to fetch the results from
	// the provider. An example is retrieving Veo 3 generated videos from Gemini requires the authentication
	// headers to be set.
	HTTPClient() *http.Client

	// GenSync runs generation synchronously.
	//
	// Multiple options can be mixed together, both standard ones like *GenOptionImage, *GenOptionText,
	// *GenOptionTools and provider-specialized options struct, e.g. *anthropic.GenOptionText, *gemini.GenOption.
	GenSync(ctx context.Context, msgs Messages, opts ...GenOption) (Result, error)
	// GenStream runs generation synchronously, yielding the fragments of replies as the server sends them.
	//
	// No need to accumulate the fragments into a Message since the Result contains the accumulated message.
	GenStream(ctx context.Context, msgs Messages, opts ...GenOption) (iter.Seq[Reply], func() (Result, error))
	// SystemOne answers typed questions about a state without text generation or streaming.
	// It uses the client's configured model. The request is not modified.
	// Requires ProviderCapabilities.SystemOne. Returns base.ErrNotSupported otherwise.
	SystemOne(ctx context.Context, req *SystemOneRequest) (*SystemOneResponse, error)
	// Embed returns one vector per input request in order using the configured model.
	// Configure an explicit embedding model; automatic generation model selection
	// does not choose one. Content blocks and options are not modified; document
	// readers may be read and sought. Input size limits depend
	// on the provider and model. Returns base.ErrNotSupported when unsupported.
	// Provider-specific raw APIs expose additional embedding controls.
	// Capabilities().Embed describes endpoint implementation; the scoreboard
	// Embedding lookup describes tested support for the configured model.
	Embed(ctx context.Context, in *EmbeddingRequest) (*EmbeddingResponse, error)
	// ListModels returns the list of models the provider supports. Not all providers support it, some will
	// return an ErrorNotSupported. For local providers like llamacpp and ollama, they may return only the
	// model currently loaded.
	ListModels(ctx context.Context) ([]Model, error)

	// GenAsync requests a generation and returns a pending job that can be polled.
	//
	// Requires ProviderCapabilities.GenAsync to be set. Returns base.ErrNotSupported otherwise.
	GenAsync(ctx context.Context, msgs Messages, opts ...GenOption) (Job, error)
	// PokeResult requests the state of the job.
	//
	// When the job is still pending, Result.Usage.FinishReason is Pending.
	//
	// Requires ProviderCapabilities.GenAsync to be set. Returns base.ErrNotSupported otherwise.
	PokeResult(ctx context.Context, job Job) (Result, error)
	// CacheAddRequest caches a request.
	//
	// Requires ProviderCapabilities.Caching to be set. Returns base.ErrNotSupported otherwise.
	//
	// # Warning
	//
	// May be changed in the future.
	CacheAddRequest(ctx context.Context, msgs Messages, name, displayName string, ttl time.Duration, opts ...GenOption) (string, error)
	// CacheList lists the caches entries.
	//
	// Requires ProviderCapabilities.Caching to be set. Returns base.ErrNotSupported otherwise.
	//
	// # Warning
	//
	// May be changed in the future.
	CacheList(ctx context.Context) ([]CacheEntry, error)
	// CacheDelete deletes a cache entry.
	//
	// Requires ProviderCapabilities.Caching to be set. Returns base.ErrNotSupported otherwise.
	//
	// # Warning
	//
	// May be changed in the future.
	CacheDelete(ctx context.Context, name string) error
}

Provider is the base interface that all provider interfaces embed.

The first group contains local methods. Calling these methods will not make an HTTP request.

The second group is supported by the majority of providers.

The rest is supported by a limited number of providers.

Example (Embed)

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/maruel/genai"
	"github.com/maruel/genai/providers/gemini"
)

func main() {
	ctx := context.Background()
	c, err := gemini.New(ctx, genai.ProviderOptionModel("gemini-embedding-2"))
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Fatal(err)
	}
	if !c.Capabilities().Embed {
		log.Fatal("embedding endpoint is unavailable")
	}
	sb := c.Scoreboard()
	if f := sb.Embedding(c.ModelID()); f != nil {
		fmt.Printf("Qualified default size: %d\n", f.Dimensions)
	} else {
		fmt.Println("Embedding model support is unmeasured")
	}
	out, err := c.Embed(ctx, &genai.EmbeddingRequest{
		Inputs:     []genai.Request{{Text: "A kitten plays with yarn."}, {Text: "A cat plays with string."}},
		Dimensions: 768,
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Returned %d vectors with %d dimensions\n", len(out.Embeddings), len(out.Embeddings[0]))
}

Example (GenSync_audio)

package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"strings"

	"github.com/maruel/genai"
	"github.com/maruel/genai/providers/gemini"
)

func main() {
	// Supported by Gemini, OpenAI.

	// Using a free small model for testing.
	// See https://ai.google.dev/gemini-api/docs/models/gemini?hl=en
	ctx := context.Background()
	c, err := gemini.New(ctx, genai.ProviderOptionModel("gemini-2.5-flash-lite"))
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	f, err := os.Open("internal/internaltest/testdata/mystery_word.mp3")
	if err != nil {
		log.Print(err)
		return
	}
	defer func() { _ = f.Close() }()
	msgs := genai.Messages{
		{
			Requests: []genai.Request{
				{Text: "What is the word said? Reply with only the word."},
				{Doc: genai.Doc{Src: f}},
			},
		},
	}
	resp, err := c.GenSync(ctx, msgs)
	if err != nil {
		log.Print(err)
		return
	}
	fmt.Printf("Heard: %v\n", strings.TrimRight(strings.ToLower(resp.String()), "."))
	// This would Output: Heard: orange
}

Example (GenSync_pdf)

package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"strings"

	"github.com/maruel/genai"
	"github.com/maruel/genai/providers/gemini"
)

func main() {
	// Supported by Anthropic, Gemini, Mistral, OpenAI.

	// Using a free small model for testing.
	// See https://ai.google.dev/gemini-api/docs/models/gemini?hl=en
	ctx := context.Background()
	c, err := gemini.New(ctx, genai.ProviderOptionModel("gemini-2.5-flash-lite"))
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	f, err := os.Open("internal/internaltest/testdata/hidden_word.pdf")
	if err != nil {
		log.Print(err)
		return
	}
	defer func() { _ = f.Close() }()
	msgs := genai.Messages{
		{
			Requests: []genai.Request{
				{Text: "What is the word? Reply with only the word."},
				{Doc: genai.Doc{Src: f}},
			},
		},
	}
	resp, err := c.GenSync(ctx, msgs)
	if err != nil {
		log.Print(err)
		return
	}
	fmt.Printf("Hidden word in PDF: %v\n", strings.ToLower(resp.String()))
	// This would Output: Hidden word in PDF: orange
}

Example (GenSync_video)

package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"strings"

	"github.com/maruel/genai"
	"github.com/maruel/genai/providers/gemini"
)

func main() {
	// Supported by Gemini, TogetherAI.

	// Using a free small model for testing.
	// See https://ai.google.dev/gemini-api/docs/models/gemini?hl=en
	ctx := context.Background()
	c, err := gemini.New(ctx, genai.ProviderOptionModel("gemini-2.5-flash"))
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	f, err := os.Open("internal/internaltest/testdata/animation.mp4")
	if err != nil {
		log.Print(err)
		return
	}
	defer func() { _ = f.Close() }()
	// TogetherAI seems to require separate messages for text and images.
	msgs := genai.Messages{
		genai.NewTextMessage("What is the word? Reply with exactly and only one word."),
		{Requests: []genai.Request{{Doc: genai.Doc{Src: f}}}},
	}
	resp, err := c.GenSync(ctx, msgs)
	if err != nil {
		log.Print(err)
		return
	}
	fmt.Printf("Saw: %v\n", strings.ToLower(resp.String()))
	// This would Output: Saw: banana
}

Example (GenSync_vision)

package main

import (
	"bytes"
	"context"
	"fmt"
	"log"
	"os"

	"github.com/maruel/genai"
	"github.com/maruel/genai/providers/gemini"
)

func main() {
	// Supported by Anthropic, Gemini, Groq, Mistral, Ollama, OpenAI, TogetherAI.

	// Using a free small model for testing.
	// See https://ai.google.dev/gemini-api/docs/models/gemini?hl=en
	ctx := context.Background()
	c, err := gemini.New(ctx, genai.ProviderOptionModel("gemini-2.5-flash-lite"))
	if c != nil {
		defer func() {
			if err := c.Close(); err != nil {
				log.Printf("Close: %v", err)
			}
		}()
	}
	if err != nil {
		log.Print(err)
		return
	}
	bananaJpg, err := os.ReadFile("internal/internaltest/testdata/banana.jpg")
	if err != nil {
		log.Print(err)
		return
	}
	msgs := genai.Messages{
		{
			Requests: []genai.Request{
				{Text: "Is it a banana? Reply with only the word yes or no."},
				{Doc: genai.Doc{Filename: "banana.jpg", Src: bytes.NewReader(bananaJpg)}},
			},
		},
	}
	resp, err := c.GenSync(ctx, msgs)
	if err != nil {
		log.Print(err)
		return
	}
	fmt.Printf("Banana: %v\n", resp.String())
	// This would Output: Banana: yes
}

type ProviderCapabilities

type ProviderCapabilities struct {
	// Embed indicates that the client implements the embedding endpoint, independent
	// of its configured model. scoreboard.Score.Embedding reports qualified support;
	// a missing measurement means unknown, not unsupported.
	Embed bool
	// SystemOne indicates the provider implements typed decision inference.
	SystemOne bool
	// GenAsync indicates the provider supports GenAsync and PokeResult for batch operations.
	GenAsync bool
	// Caching indicates the provider supports CacheAddRequest, CacheList, and CacheDelete.
	Caching bool
	// contains filtered or unexported fields
}

ProviderCapabilities describes optional capabilities implemented by a client.

type ProviderOption

type ProviderOption interface {
	Validate() error
}

ProviderOption is an option for provider constructors.

type ProviderOptionAPIKey

type ProviderOptionAPIKey string

ProviderOptionAPIKey provides an API key to authenticate to the server.

Most providers require an API key, and the client will look at an environment variable "<PROVIDER>_API_KEY" to use as a default value if unspecified.

func (ProviderOptionAPIKey) Validate

func (p ProviderOptionAPIKey) Validate() error

Validate implements Validatable.

type ProviderOptionModalities

type ProviderOptionModalities Modalities

ProviderOptionModalities is the list of output modalities you request the model to support.

Most provider support text output only. Most models support output of only one modality, either text, image, audio or video. But a few models do support both text and images.

When unspecified, the provider defaults to all modalities supported by the provider and the selected model.

Even when Model is set to a specific model ID, a ListModels call may be made to discover its supported output modalities for providers that support multiple output modalities.

ProviderOptionModalities can be set without a Model to test if a provider supports a modality without causing a ListModels call.

func (ProviderOptionModalities) Validate

func (p ProviderOptionModalities) Validate() error

Validate implements Validatable.

type ProviderOptionModel

type ProviderOptionModel string

ProviderOptionModel specifies which model to use.

For automatic model selection, use the predefined constants ModelCheap, ModelGood, or ModelSOTA directly as provider options. The provider internally calls ListModels() to discover models and select based on its heuristics. Providers that do not support ListModels (e.g. bfl or perplexity) use a hardcoded list.

When unspecified, no model is selected. This is useful when only calling ListModels(). Generation calls will fail.

Keep in mind that as providers cycle through new models, it's possible a specific model ID is not available anymore or that the default model changes.

const (
	// ModelCheap requests the provider to automatically select the cheapest model it can find.
	ModelCheap ProviderOptionModel = "CHEAP"
	// ModelGood requests the provider to automatically select a good every day model that has a good
	// performance/cost trade-off.
	ModelGood ProviderOptionModel = "GOOD"
	// ModelSOTA requests the provider to automatically select the best state-of-the-art model
	// it can find.
	ModelSOTA ProviderOptionModel = "SOTA"
)

Model markers for automatic model selection. These are ProviderOptionModel values that can be passed directly to provider constructors.

func (ProviderOptionModel) Validate

func (p ProviderOptionModel) Validate() error

Validate implements Validatable.

type ProviderOptionPreloadedModels

type ProviderOptionPreloadedModels []Model

ProviderOptionPreloadedModels is a list of models that are preloaded into the provider, to replace the call to ListModels, for example with automatic model selection and modality detection.

This is mostly used for unit tests or repeated client creation to save on HTTP requests.

func (ProviderOptionPreloadedModels) Validate

func (p ProviderOptionPreloadedModels) Validate() error

Validate implements Validatable.

type ProviderOptionRemote

type ProviderOptionRemote string

ProviderOptionRemote is the remote address to access the service.

It is mostly used by locally hosted services (llamacpp, ollama) or for generic client (openaicompatible).

func (ProviderOptionRemote) Validate

func (p ProviderOptionRemote) Validate() error

Validate implements Validatable.

type ProviderOptionStarterWrapper

type ProviderOptionStarterWrapper func(Starter) Starter

ProviderOptionStarterWrapper wraps the subprocess starter used by CLI-based providers.

This is the subprocess analog of ProviderOptionTransportWrapper: it lets callers intercept process creation for recording, replay, or testing.

func (ProviderOptionStarterWrapper) Validate

func (p ProviderOptionStarterWrapper) Validate() error

Validate implements Validatable.

type ProviderOptionTransportWrapper

type ProviderOptionTransportWrapper func(http.RoundTripper) http.RoundTripper

ProviderOptionTransportWrapper wraps the HTTP transport used by the provider.

This is useful for adding middleware like logging, tracing, or HTTP recording for tests.

func (ProviderOptionTransportWrapper) Validate

func (p ProviderOptionTransportWrapper) Validate() error

Validate implements Validatable.

type ProviderPing

type ProviderPing interface {
	Provider
	// Ping enables confirming that the provider is accessible, without incurring cost. This is useful for local
	// providers to detect if they are accessible or not.
	Ping(ctx context.Context) error
}

ProviderPing represents a provider that you can ping.

type ProviderScoreboardVariants

type ProviderScoreboardVariants interface {
	ScoreboardVariants() []ScoreboardVariant
}

ProviderScoreboardVariants is optionally implemented by providers that have multiple regional backends with different model catalogs.

When implemented, documentation tools generate a separate table per variant instead of a single table.

type ProviderUnwrap

type ProviderUnwrap interface {
	Unwrap() Provider
}

ProviderUnwrap is exposed when the Provider is actually a wrapper around another one, like adapters.ProviderReasoning or ProviderUsage. This is useful when looking for other interfaces.

type Question

type Question struct {
	// Type is how the state is evaluated. It selects which of Noul, Choice or Score must be set.
	Type QuestionType
	// Instructions is what the model should decide or rate. Providers may require it.
	Instructions DecisionContent
	// Noul describes what the yes and the no outcomes mean. It can only be set for QuestionNoul, where it
	// is optional.
	//
	// Recommended reading: https://docs.typesafe.ai/primitives/noul
	Noul *NoulCriteria
	// Choice maps the options of a choice question to their descriptions. It can only be set for
	// QuestionChoice, where at least one option is required. Use nil for an option that needs no extra
	// detail.
	//
	// Recommended reading: https://docs.typesafe.ai/primitives/choice
	Choice map[string]DecisionContent
	// Score lists the levels of a score question rubric, in order, starting at level 0. It can only be set
	// for QuestionScore, where at least one level is required. Use nil for an undescribed level.
	// Providers may impose stricter limits.
	//
	// Recommended reading: https://docs.typesafe.ai/primitives/score
	Score []DecisionContent
}

Question is a typed question about a state.

Exactly one of Noul, Choice or Score can be set, the one matching Type. Instructions is shared by the three types.

Type is explicit instead of inferred from the field that is set: a choice or score question whose criteria are nil at runtime, which append and conditional map building produce, would otherwise be silently asked as a noul question instead of being rejected.

Recommended reading: https://docs.typesafe.ai/primitives/advanced

func (*Question) MarshalJSON

func (q *Question) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler.

func (*Question) Validate

func (q *Question) Validate() error

Validate checks shared content and union consistency, not provider-specific limits.

type QuestionType

type QuestionType string

QuestionType is the type of a Question.

const (
	// QuestionNoul is a yes/no question. The answer is the probability that the answer is yes.
	QuestionNoul QuestionType = "noul"
	// QuestionChoice picks one option from a set defined by the question.
	QuestionChoice QuestionType = "choice"
	// QuestionScore rates the state along an ordered rubric.
	QuestionScore QuestionType = "score"
)

Question types.

type Questions

type Questions map[string]*Question

Questions is a set of questions to ask about one state, keyed by the name to report each answer under.

The names are chosen by the caller. They are not sent to the model, they are only used to key the answers.

Set SystemOneRequest.Questions directly. SystemOneResponse.Answers holds the typed answers under the same dynamic names. Entries must not be nil.

Example

ExampleQuestions declares typed questions under caller-selected names.

package main

import (
	"encoding/json"
	"fmt"
	"log"

	"github.com/maruel/genai"
)

func main() {
	questions := genai.Questions{
		"billing": {Type: genai.QuestionNoul, Instructions: genai.Text("Is this request about billing?")},
		"tone":    {Type: genai.QuestionChoice, Instructions: genai.Text("What is the tone of the customer?"), Choice: map[string]genai.DecisionContent{"calm": nil, "angry": nil}},
	}

	raw, err := json.MarshalIndent(questions, "", "  ")
	if err != nil {
		log.Print(err)
		return
	}
	fmt.Printf("%s\n", raw)
}

Output:
{
  "billing": {
    "type": "noul",
    "instructions": "Is this request about billing?"
  },
  "tone": {
    "type": "choice",
    "instructions": "What is the tone of the customer?",
    "criteria": {
      "angry": null,
      "calm": null
    }
  }
}

func (Questions) Validate

func (q Questions) Validate() error

Validate ensures the questions are valid.

type RateLimit

type RateLimit struct {
	Type      RateLimitType
	Period    RateLimitPeriod
	Limit     int64
	Remaining int64
	Reset     time.Time
}

RateLimit contains the limit, remaining, and reset values for a metric.

func (*RateLimit) String

func (r *RateLimit) String() string

func (*RateLimit) Validate

func (r *RateLimit) Validate() error

Validate ensures the rate limit is valid.

type RateLimitPeriod

type RateLimitPeriod int32

RateLimitPeriod defines the time period for a rate limit.

const (
	PerOther RateLimitPeriod = iota // For non-standard periods
	PerMinute
	PerDay
	PerMonth
)

RateLimitPeriod values.

type RateLimitType

type RateLimitType int32

RateLimitType defines the type of rate limit.

const (
	Requests RateLimitType = iota + 1
	Tokens
)

RateLimitType values.

type Reply

type Reply struct {
	// Text is the content of the text message.
	Text string `json:"text,omitzero"`

	// Doc can be audio, video, image, PDF or any other format, including reference text.
	Doc Doc `json:"doc,omitzero"`

	// Citation contains references to source material that support the content.
	Citation Citation `json:"citation,omitzero"`

	// Reasoning is the reasoning done by the LLM.
	Reasoning string `json:"reasoning,omitzero"`

	// ToolCall is a tool call that the LLM requested to make.
	ToolCall ToolCall `json:"tool_call,omitzero"`

	// Opaque is added to keep continuity on the processing. A good example is Anthropic's extended thinking, or
	// server-side tool calling. It must be kept during an exchange.
	//
	// A message with only Opaque set is valid. It can be used in combination with other fields. This field is
	// specific to both the provider and the model.
	//
	// The data must be JSON-serializable.
	Opaque map[string]any `json:"opaque,omitzero"`
	// contains filtered or unexported fields
}

Reply is a block of information returned by the provider.

Normally only one of the field must be set. The exception is the Opaque field.

Reply generally represents content returned by the provider, like a block of text or a document returned by the model. It can be a silent tool call request. It can also be an opaque block. A good example is traces of server side tool calling like Web search or MCP tool calling.

func (*Reply) GoString

func (r *Reply) GoString() string

GoString returns a JSON representation of the reply for debugging purposes.

func (*Reply) IsZero

func (r *Reply) IsZero() bool

IsZero returns true if the Reply is empty.

An empty reply is not valid.

func (*Reply) UnmarshalJSON

func (r *Reply) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

func (*Reply) Validate

func (r *Reply) Validate() error

Validate ensures the block is valid.

type Request

type Request struct {
	// Text is the content of the text message.
	Text string `json:"text,omitzero"`

	// Doc can be audio, video, image, PDF or any other format, including reference text.
	Doc Doc `json:"doc,omitzero"`
	// contains filtered or unexported fields
}

Request is an input content block for generation or embedding.

It is effectively a union, only one of the 2 related field groups can be set.

func (*Request) UnmarshalJSON

func (r *Request) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

func (*Request) Validate

func (r *Request) Validate() error

Validate ensures the block is valid.

type Result

type Result struct {
	Message
	Usage Usage
	// Logprobs is a list of multiple log probabilities, each for a token.
	//
	// The first item of each subslice is the chosen token. The next items are the candidates not chosen.
	//
	// Some providers only return the probability for the chosen tokens and not for the candidates.
	Logprobs [][]Logprob
}

Result is the result of a completion.

It is a Message along with Usage metadata about the operation. It optionally include Logprobs if requested and the provider support it.

func (*Result) Validate

func (r *Result) Validate() error

Validate ensures the result is valid.

type ScoreLegend

type ScoreLegend map[string]DecisionContent

ScoreLegend is the description of each level of a score answer, keyed by the level.

It is the Question.Score criteria of the question, echoed back by the API. The values are decoded as Text, Object or Array; a nil value is a level the caller left undescribed.

func (*ScoreLegend) UnmarshalJSON

func (l *ScoreLegend) UnmarshalJSON(b []byte) error

UnmarshalJSON implements json.Unmarshaler.

DecisionContent is an interface, so encoding/json cannot decode it on its own; each level is decoded into the variant matching its JSON token here.

type ScoreboardVariant

type ScoreboardVariant struct {
	// Name is the display name for this variant, e.g. "Intl", "US".
	Name string
	// Score is the scoreboard data for this variant.
	Score scoreboard.Score
}

ScoreboardVariant is a named scoreboard for a specific backend or region of a provider.

type Starter

type Starter func(ctx context.Context, args []string) (stdin io.WriteCloser, stdout io.ReadCloser, wait func() error, err error)

Starter launches a subprocess and returns its stdin, stdout, a wait function, and any error.

It is the subprocess equivalent of http.RoundTripper: the lowest-level abstraction that CLI-based providers (claudecode, codex, opencode) use to spawn a process.

type SystemOneRequest

type SystemOneRequest struct {
	// State is required unless Docs contains one application/json document.
	State     DecisionContent `json:"state"`
	Questions Questions       `json:"questions"`
	// Docs contains inline document attachments. When State is unset, one application/json document
	// supplies the state instead. Supported attachment types depend on the provider and model.
	// Remote URLs are not supported.
	Docs []Doc `json:"docs,omitzero"`
}

SystemOneRequest is a System One decision request. Providers use their client's configured model and own transport envelopes and stricter protocol limits. Docs are inline attachments. Providers select supported document types, enforce read limits, and encode document bytes in their native wire format.

func (*SystemOneRequest) ReadState

func (r *SystemOneRequest) ReadState(maxDocBytes int64) (DecisionContent, []Doc, error)

ReadState validates the request and returns the resolved state and remaining attachments. It reads a JSON state document with maxDocBytes as the byte limit, but does not read attachments. It leaves request fields unchanged. Returned attachments share the request's Docs slice.

func (*SystemOneRequest) Validate

func (r *SystemOneRequest) Validate() error

Validate checks the shared state, questions and inline documents without reading attachment bytes. Providers validate document types.

type SystemOneResponse

type SystemOneResponse struct {
	// Answers holds one answer per question, keyed by the question name.
	Answers Answers `json:"answers"`
	// Usage is the token usage of the request.
	Usage DecisionUsage `json:"usage"`
}

SystemOneResponse is the response of a POST /v1/systemone request.

func (*SystemOneResponse) ValidateQuestions

func (r *SystemOneResponse) ValidateQuestions(qs *Questions) error

ValidateQuestions checks that the response answers each requested question with the right type.

type Text

type Text string

Text is DecisionContent that is a JSON string.

func (Text) Validate

func (Text) Validate() error

Validate implements internal.Validatable.

Any text is valid, including the empty string, the API is the one that decides whether a specific field may be empty.

type ToolCall

type ToolCall struct {
	ID        string `json:"id,omitzero"`        // Unique identifier for the tool call. Necessary for parallel tool calling.
	Name      string `json:"name,omitzero"`      // Tool being called.
	Arguments string `json:"arguments,omitzero"` // encoded as JSON

	// Opaque is added to keep continuity on the processing. A good example is Anthropic's extended thinking. It
	// must be kept during an exchange.
	//
	// A message with only Opaque set is valid.
	Opaque map[string]any `json:"opaque,omitzero"`
	// contains filtered or unexported fields
}

ToolCall is a tool call that the LLM requested to make.

func (*ToolCall) Call

func (t *ToolCall) Call(ctx context.Context, tools []ToolDef) (string, error)

Call invokes the ToolDef.Callback with arguments from the ToolCall, returning the result string.

It decodes the ToolCall.Arguments and passes it to the ToolDef.Callback.

func (*ToolCall) IsZero

func (t *ToolCall) IsZero() bool

IsZero returns true if the tool call is empty.

func (*ToolCall) UnmarshalJSON

func (t *ToolCall) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

func (*ToolCall) Validate

func (t *ToolCall) Validate() error

Validate ensures the tool call request from the LLM is valid.

type ToolCallRequest

type ToolCallRequest int

ToolCallRequest determines if we want the LLM to request a tool call.

const (
	// ToolCallAny is the default, the model is free to choose if a tool is called or not. For some models (like
	// llama family), it may be a bit too "tool call happy".
	ToolCallAny ToolCallRequest = iota
	// ToolCallRequired means a tool call is required. Don't forget to change the value after sending the
	// response!
	ToolCallRequired
	// ToolCallNone means that while tools are described, they should not be called. It is useful when a LLM did
	// tool calls, got the response and now it's time to generate some text to present to the end user.
	ToolCallNone
)

type ToolCallResult

type ToolCallResult struct {
	ID     string `json:"id,omitzero"`
	Name   string `json:"name,omitzero"`
	Result string `json:"result,omitzero"`
	// contains filtered or unexported fields
}

ToolCallResult is the result for a tool call that the LLM requested to make.

func (*ToolCallResult) UnmarshalJSON

func (t *ToolCallResult) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

func (*ToolCallResult) Validate

func (t *ToolCallResult) Validate() error

Validate ensures the tool result is valid.

type ToolDef

type ToolDef struct {
	// Name must be unique among all tools.
	Name string
	// Description must be a LLM-friendly short description of the tool.
	Description string
	// Callback is the function to call with the inputs.
	// It must accept a context.Context one struct pointer as input: (ctx context.Context, input *struct{}). The
	// struct must use json_schema to be serializable as JSON.
	// It must return the result and an error: (string, error).
	Callback any
	// InputSchemaOverride overrides the schema deduced from the Callback's second argument. It's meant to be
	// used when an enum or a description is set dynamically, or with complex if/then/else that would be tedious
	// to describe as struct tags.
	//
	// It is okay to initialize Callback, then take the return value of GetInputSchema() to initialize InputSchemaOverride, then mutate it.
	InputSchemaOverride JSONSchema
	// contains filtered or unexported fields
}

ToolDef describes a tool that the LLM can request to use.

func (*ToolDef) GetInputSchema

func (t *ToolDef) GetInputSchema() (JSONSchema, error)

GetInputSchema returns the json schema for the input argument of the callback.

func (*ToolDef) Validate

func (t *ToolDef) Validate() error

Validate ensures the tool definition is valid.

For the Name field, it uses the rule according to https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/implement-tool-use#example-simple-tool-definition

type Usage

type Usage struct {
	// Token usage for the current request.
	InputTokens       int64
	InputCachedTokens int64
	ReasoningTokens   int64
	OutputTokens      int64
	TotalTokens       int64
	// FinishReason indicates why the model stopped generating tokens.
	FinishReason FinishReason
	// ServiceTier is the quality of service tier used to process the request,
	// as reported by the provider. Common values include "standard", "batch",
	// "flex", "default", "auto", etc. Empty when not reported.
	ServiceTier string
	// Limits contains a list of rate limit details from the provider.
	Limits []RateLimit
}

Usage from the LLM provider.

func (*Usage) Add

func (u *Usage) Add(r *Usage)

Add accumulates the usage from another result.

func (*Usage) String

func (u *Usage) String() string