websearch

Documentation

Overview

Package websearch is the contract behind txco://web/search: the request a rule may make, the answer it gets back, and the seam a search provider implements. It does no networking. The handler in chassis/server fetches through chassis/fetch; a provider (websearch/brave) only builds the request and reads the body.

A search is one question to a public search engine, under the tenant's own key, answered with titles, addresses and snippets, never the provider's raw body. Its kind is web (pages) or news (articles, from the engine's news index). Results are external text: the stack that reads them decides what they may do.

Index

• Constants
• func Clean(s string) string
• func Parse(meta []byte) (Request, *Error)
• func Register(name string, c Constructor)
• func Registered() []string
• type Answer
• type Config
• type Constructor
• type Error
  • func Errorf(code, format string, args ...any) *Error
  • func (e *Error) Error() string
• type Failure
• type Provider
  • func Open(name string, cfg Config) (Provider, error)
• type Request
• type Result
  • func Normalize(in []Result) (out []Result, truncated bool)

Constants

const (
	CodeInvalidRequest   = "invalid_request"
	CodeMissingSecret    = "missing_secret"
	CodeDisabled         = "disabled"
	CodeRateLimited      = "rate_limited"
	CodeBusy             = "busy"
	CodeTimeout          = "timeout"
	CodeProviderAuth     = "provider_auth"
	CodeProviderError    = "provider_error"
	CodeResponseTooLarge = "response_too_large"
	CodeInvalidResponse  = "invalid_response"
	CodeNoTenant         = "no_tenant"
	CodeInternal         = "internal_error"
)

Error codes. The handler prefixes them txco_web_. The fetcher's own codes (destination_denied, dns_failed, connection_failed, timeout, response_too_large) pass through under the same prefix.

const (
	MaxQueryChars = 500
	MaxQueryWords = 75
	MaxCount      = 20
	DefaultCount  = 10

	MaxTitleBytes       = 1 << 10
	MaxDescriptionBytes = 4 << 10
	MaxURLBytes         = 2 << 10
	MaxAnswerBytes      = 128 << 10
)

The contract's bounds. Brave allows 600 characters and 75 words; the contract fits inside them.

const DefaultInto = "_websearch"

DefaultInto is where the answer lands when a rule names no `into`.

Functions

func Clean

func Clean(s string) string

Clean makes provider text plain: tags stripped, entities decoded, whitespace collapsed.

func Parse

func Parse(meta []byte) (Request, *Error)

Parse reads and checks the op's WITH object. An *Error names the first field at fault; it is the whole of what a rule author sees.

func Register

func Register(name string, c Constructor)

Register adds a provider under its name. A provider package calls it from init(); the chassis links one with a blank import, the way the chat backends are linked.

func Registered

func Registered() []string

Registered lists the linked providers, sorted.

Types

type Answer

type Answer struct {
	OK        bool     `json:"ok"`
	Query     string   `json:"query"`
	Kind      string   `json:"kind"`
	Provider  string   `json:"provider"`
	Results   []Result `json:"results"`
	Count     int      `json:"count"`
	Truncated bool     `json:"truncated"`
}

Answer is the op's result object on success.

type Config

type Config struct {
	// BraveURL is the Brave API root (--web-search-brave-url); "" is the
	// public one.
	BraveURL string
}

Config carries what a provider needs from the chassis configuration.

type Constructor

type Constructor func(Config) (Provider, error)

Constructor builds a provider from the configuration.

type Error

type Error struct {
	Code       string `json:"code"`
	Message    string `json:"message"`
	Status     int    `json:"status,omitempty"`
	RetryAfter int    `json:"retry_after,omitempty"`
}

Error is a failed search, as the rule sees it at `into`. Message is fixed text for the rule author: it never carries the provider's body, a header or an address. Status is the provider's, when it answered. RetryAfter is seconds, on rate_limited from the provider.

func Errorf

func Errorf(code, format string, args ...any) *Error

Errorf builds an Error.

func (*Error) Error

func (e *Error) Error() string

type Failure

type Failure struct {
	OK       bool   `json:"ok"`
	Query    string `json:"query,omitempty"`
	Provider string `json:"provider,omitempty"`
	Error    *Error `json:"error"`
}

Failure is the op's result object when the search did not happen or did not answer. The provider's body is never in it.

type Provider

type Provider interface {
	// Name is the registered name, and the `provider` field of the answer.
	Name() string
	// RequiredSecrets names the tenant secrets the provider needs, as the
	// ai:// backends do (BRAVE_SEARCH_KEY). The handler materializes them
	// from the tenant's store, with the env fallback the ai:// ops use.
	RequiredSecrets() []string
	// Build turns a validated request into the one fetch to make.
	Build(req Request, key []byte) (fetch.Request, error)
	// Parse reads a 2xx body into results, in the provider's order, raw:
	// Normalize cleans and caps them. An error means the body was not
	// what the provider sends (invalid_response).
	Parse(body []byte) ([]Result, error)
	// Failure maps a non-2xx answer to the op's error: provider_auth for
	// a refused key, rate_limited with retry_after, invalid_request when
	// the provider refused a parameter, provider_error otherwise.
	Failure(status int, header http.Header) *Error
}

Provider is one search engine behind txco://web/search. It builds the request and reads the answer; it never dials. The handler fetches through chassis/fetch with the request a provider built, so every provider gets the same egress guard, caps and deadline.

Build is the only place the key is handled. It goes into the header the provider reads it from and nowhere else: not the URL, not a log, not an error.

func Open

func Open(name string, cfg Config) (Provider, error)

Open builds the named provider. An unknown name lists what is linked.

type Request

type Request struct {
	Query string
	// Kind is web (pages, the default) or news (recent articles from the
	// engine's news index: an outlet's headlines without reading its pages).
	Kind  string
	Count int
	// Country is two upper-case letters, ALL, or "" for the provider's
	// default. Language is a lower-case ISO 639-1 code, or "" for the
	// provider's default. The provider refuses one it lacks.
	Country    string
	Language   string
	Freshness  string // any, day, week, month, year
	SafeSearch string // off, moderate, strict
}

Request is one validated search.

type Result

type Result struct {
	Title       string `json:"title"`
	URL         string `json:"url"`
	Description string `json:"description"`
	// PageDate is the page's date as the provider reports it, RFC 3339 in
	// UTC, or null. Brave calls it page_age: "published or last modified".
	PageDate *string `json:"page_date"`
	// Source is the URL's host, lower case.
	Source string `json:"source"`
}

Result is one hit, in the provider's order.

func Normalize

func Normalize(in []Result) (out []Result, truncated bool)

Normalize applies the answer's rules to a provider's results: a result with no title or an unusable URL is dropped; a URL over its cap drops its result (a cut URL points somewhere else); titles and descriptions are cut at their caps on a character boundary; and the answer as a whole stays under MaxAnswerBytes by dropping results from the end. Truncated reports any cut or drop for size.