cloudflare

package
v0.7.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 23, 2026 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package cloudflare implements a client for the Cloudflare AI API.

It is described at https://developers.cloudflare.com/api/resources/ai/

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func ProcessStream 

func ProcessStream(chunks iter.Seq[ChatStreamChunkResponse]) (iter.Seq[genai.Reply], func() (genai.Usage, [][]genai.Logprob, error))

ProcessStream converts the raw packets from the streaming API into Reply fragments.

func Scoreboard 

func Scoreboard() scoreboard.Score

Scoreboard for Cloudflare.

Types

type AccountID 

type AccountID string

AccountID provides an account ID for Cloudflare Workers AI.

Get your account ID at https://dash.cloudflare.com/profile/api-tokens

func (AccountID) Validate 

func (a AccountID) Validate() error

Validate implements genai.Validatable.

type ChatRequest 

type ChatRequest struct {
	Messages          []Message `json:"messages"`
	FrequencyPenalty  float64   `json:"frequency_penalty,omitzero"` // [0, 2.0]
	MaxTokens         int64     `json:"max_tokens,omitzero"`
	PresencePenalty   float64   `json:"presence_penalty,omitzero"`   // [0, 2.0]
	RepetitionPenalty float64   `json:"repetition_penalty,omitzero"` // [0, 2.0]
	ResponseFormat    struct {
		Type       string           `json:"type,omitzero"` // json_object, json_schema
		JSONSchema genai.JSONSchema `json:"json_schema,omitzero"`
	} `json:"response_format,omitzero"`
	GuidedJSON  genai.JSONSchema `json:"guided_json,omitzero"`
	Seed        int64            `json:"seed,omitzero"`
	Stream      bool             `json:"stream,omitzero"`
	Temperature float64          `json:"temperature,omitzero"` // [0, 5]
	Tools       []Tool           `json:"tools,omitzero"`
	TopK        int64            `json:"top_k,omitzero"` // [1, 50]
	TopP        float64          `json:"top_p,omitzero"` // [0, 2.0]
}

ChatRequest structure depends on the model used.

The general description is at https://developers.cloudflare.com/api/resources/ai/methods/run/

A specific one is https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/

func (*ChatRequest) Init 

func (c *ChatRequest) Init(msgs genai.Messages, model string, opts ...genai.GenOption) error

Init initializes the provider specific completion request with the generic completion request.

func (*ChatRequest) SetStream 

func (c *ChatRequest) SetStream(stream bool)

SetStream sets the streaming mode.

type ChatResponse 

type ChatResponse struct {
	Result struct {
		MessageResponse
		Usage Usage `json:"usage"`
	} `json:"result"`
	Success  bool       `json:"success"`
	Errors   []struct{} `json:"errors"`   // Annoyingly, it's included all the time
	Messages []struct{} `json:"messages"` // Annoyingly, it's included all the time
}

ChatResponse is somewhat documented at https://developers.cloudflare.com/api/resources/ai/methods/run/ See UnionMember7.

func (*ChatResponse) ToResult 

func (c *ChatResponse) ToResult() (genai.Result, error)

ToResult converts the response to a genai.Result.

type ChatStreamChunkResponse 

type ChatStreamChunkResponse struct {
	Response  Response   `json:"response"`
	P         string     `json:"p"`
	ToolCalls []ToolCall `json:"tool_calls"`
	Usage     Usage      `json:"usage"`
}

ChatStreamChunkResponse is not documented. If you find the documentation for this please tell me!

type Client 

type Client struct {
	base.NotImplemented
	// contains filtered or unexported fields
}

Client implements genai.Provider.

func New 

func New(ctx context.Context, opts ...genai.ProviderOption) (*Client, error)

New creates a new client to talk to the Cloudflare Workers AI platform API.

If AccountID is not provided, it tries to load it from the CLOUDFLARE_ACCOUNT_ID environment variable. If ProviderOptionAPIKey is not provided, it tries to load it from the CLOUDFLARE_API_KEY environment variable. If none is found, it will still return a client coupled with an base.ErrAPIKeyRequired error. Get your account ID and API key at https://dash.cloudflare.com/profile/api-tokens

To use multiple models, create multiple clients. Use one of the model from https://developers.cloudflare.com/workers-ai/models/

Example (HTTP_record) 
package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"os"

	"gopkg.in/dnaeon/go-vcr.v4/pkg/recorder"

	"github.com/maruel/genai"
	"github.com/maruel/genai/httprecord"
	"github.com/maruel/genai/providers/cloudflare"
)

func main() {
	// Example to do HTTP recording and playback for smoke testing.
	// The example recording is in testdata/example.yaml.
	var rr *recorder.Recorder
	defer func() {
		// In a smoke test, use t.Cleanup().
		if rr != nil {
			if err := rr.Stop(); err != nil {
				log.Printf("Failed saving recordings: %v", err)
			}
		}
	}()

	// Simple trick to force recording via an environment variable.
	mode := recorder.ModeRecordOnce
	if os.Getenv("RECORD") == "all" {
		mode = recorder.ModeRecordOnly
	}
	wrapper := func(h http.RoundTripper) http.RoundTripper {
		var err error
		rr, err = httprecord.New("testdata/example", h, recorder.WithMode(mode))
		if err != nil {
			log.Fatal(err)
		}
		return rr
	}
	// When playing back the smoke test, no API key is needed. Insert a fake API key.
	var opts []genai.ProviderOption
	if os.Getenv("CLOUDFLARE_ACCOUNT_ID") == "" {
		opts = append(opts, cloudflare.AccountID("ACCOUNT_ID"))
	}
	if os.Getenv("CLOUDFLARE_API_KEY") == "" {
		opts = append(opts, genai.ProviderOptionAPIKey("<insert_api_key_here>"))
	}
	ctx := context.Background()
	c, err := cloudflare.New(ctx, append([]genai.ProviderOption{genai.ProviderOptionTransportWrapper(wrapper)}, opts...)...)
	if err != nil {
		log.Fatal(err)
	}
	models, err := c.ListModels(ctx)
	if err != nil {
		log.Fatal(err)
	}
	if len(models) > 1 {
		fmt.Println("Found multiple models")
	}
}

Output:
Found multiple models

func (*Client) GenStream 

func (c *Client) GenStream(ctx context.Context, msgs genai.Messages, opts ...genai.GenOption) (iter.Seq[genai.Reply], func() (genai.Result, error))

GenStream implements genai.Provider.

func (*Client) GenStreamRaw 

func (c *Client) GenStreamRaw(ctx context.Context, in *ChatRequest) (iter.Seq[ChatStreamChunkResponse], func() error)

GenStreamRaw provides access to the raw API.

func (*Client) GenSync 

func (c *Client) GenSync(ctx context.Context, msgs genai.Messages, opts ...genai.GenOption) (genai.Result, error)

GenSync implements genai.Provider.

func (*Client) GenSyncRaw 

func (c *Client) GenSyncRaw(ctx context.Context, in *ChatRequest, out *ChatResponse) error

GenSyncRaw provides access to the raw API.

func (*Client) HTTPClient 

func (c *Client) HTTPClient() *http.Client

HTTPClient returns the HTTP client to fetch results (e.g. videos) generated by the provider.

func (*Client) ListModels 

func (c *Client) ListModels(ctx context.Context) ([]genai.Model, error)

ListModels implements genai.Provider.

func (*Client) ModelID 

func (c *Client) ModelID() string

ModelID implements genai.Provider.

It returns the selected model ID.

func (*Client) Name 

func (c *Client) Name() string

Name implements genai.Provider.

It returns the name of the provider.

func (*Client) OutputModalities 

func (c *Client) OutputModalities() genai.Modalities

OutputModalities implements genai.Provider.

It returns the output modalities, i.e. what kind of output the model will generate (text, audio, image, video, etc).

func (*Client) Scoreboard 

func (c *Client) Scoreboard() scoreboard.Score

Scoreboard implements genai.Provider.

type ErrorResponse 

type ErrorResponse struct {
	Errors []struct {
		Message string `json:"message"`
		Code    int    `json:"code"`
	} `json:"errors"`
	Success  bool       `json:"success"`
	Result   struct{}   `json:"result"`
	Messages []struct{} `json:"messages"` // Annoyingly, it's included all the time
}

ErrorResponse is the provider-specific error response.

func (*ErrorResponse) Error 

func (er *ErrorResponse) Error() string

func (*ErrorResponse) IsAPIError 

func (er *ErrorResponse) IsAPIError() bool

IsAPIError implements base.ErrorResponseI.

type Message 

type Message struct {
	Role       string `json:"role"` // "system", "assistant", "user", "tool"
	Content    string `json:"content,omitzero"`
	ToolCallID string `json:"tool_call_id,omitzero"`
}

Message is not well specified in the API documentation. https://developers.cloudflare.com/api/resources/ai/methods/run/

func (*Message) From 

func (m *Message) From(in *genai.Message) error

From must be called with at most one Request, one Reply or one ToolCallResults.

type MessageResponse 

type MessageResponse struct {
	// Normally a string, or an object if response_format.type == "json_schema".
	Response  json.RawMessage `json:"response"`
	ToolCalls []ToolCall      `json:"tool_calls"`
}

MessageResponse is a message in a provider-specific response.

func (*MessageResponse) To 

func (msg *MessageResponse) To(out *genai.Message) error

To converts to the genai equivalent.

type Model 

type Model struct {
	ID          string `json:"id"`
	Source      int64  `json:"source"`
	Name        string `json:"name"`
	Description string `json:"description"`
	CreatedAt   Time   `json:"created_at"`
	Task        struct {
		ID          string `json:"id"`
		Name        string `json:"name"`
		Description string `json:"description"`
	} `json:"task"`
	Tags       []string `json:"tags"`
	Properties []struct {
		PropertyID string          `json:"property_id"`
		Value      json.RawMessage `json:"value"` // sometimes a string, sometimes an array
	} `json:"properties"`
}

Model is the provider-specific model metadata.

func (*Model) Context 

func (m *Model) Context() int64

Context implements genai.Model.

func (*Model) GetID 

func (m *Model) GetID() string

GetID implements genai.Model.

func (*Model) Price 

func (m *Model) Price() (float64, float64)

Price returns the input and output price per token.

func (*Model) String 

func (m *Model) String() string

type ModelPricing 

type ModelPricing struct {
	Currency string  `json:"currency"`
	Price    float64 `json:"price"`
	Unit     string  `json:"unit"` // "per M input tokens", "per M output tokens"
}

ModelPricing is the pricing information for a model.

type ModelsResponse 

type ModelsResponse struct {
	Result     []Model `json:"result"`
	ResultInfo struct {
		Count      int64 `json:"count"`
		Page       int64 `json:"page"`
		PerPage    int64 `json:"per_page"`
		TotalCount int64 `json:"total_count"`
	} `json:"result_info"`
	Success  bool       `json:"success"`
	Errors   []struct{} `json:"errors"`   // Annoyingly, it's included all the time
	Messages []struct{} `json:"messages"` // Annoyingly, it's included all the time
}

ModelsResponse represents the response structure for Cloudflare models listing.

type Response 

type Response string

Response is normally the response but it can be true (bool) sometimes?

func (*Response) UnmarshalJSON 

func (r *Response) UnmarshalJSON(b []byte) error

UnmarshalJSON implements json.Unmarshaler.

type Time 

type Time time.Time

Time is a wrapper around time.Time to support unmarshalling for cloudflare non-standard encoding.

func (*Time) UnmarshalJSON 

func (t *Time) UnmarshalJSON(b []byte) error

UnmarshalJSON implements json.Unmarshaler.

type Tool 

type Tool struct {
	Type     string `json:"type"` // "function"
	Function struct {
		Description string           `json:"description"`
		Name        string           `json:"name"`
		Parameters  genai.JSONSchema `json:"parameters"`
	} `json:"function"`
}

Tool is a provider-specific tool definition.

type ToolCall 

type ToolCall struct {
	Type     string `json:"type,omitzero"` // "function"
	ID       string `json:"id,omitzero"`
	Index    int64  `json:"index,omitzero"`
	Function struct {
		Name      string `json:"name,omitzero"`
		Arguments string `json:"arguments"`
	} `json:"function,omitzero"`

	Arguments json.RawMessage `json:"arguments"`
	Name      string          `json:"name"`
}

ToolCall can be populated differently depending on the model used.

func (*ToolCall) To 

func (c *ToolCall) To(out *genai.ToolCall) error

To converts to the genai equivalent.

type Usage 

type Usage struct {
	CompletionTokens   int64 `json:"completion_tokens"`
	PromptTokens       int64 `json:"prompt_tokens"`
	TotalTokens        int64 `json:"total_tokens"`
	PromptTokensDetail struct {
		CachedTokens int64 `json:"cached_tokens"`
	} `json:"prompt_tokens_details,omitzero"`
}

Usage is the provider-specific token usage.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.