searxng

Documentation

Overview

Package searxng provides a full-featured SearXNG search client for Go, mirroring the mcp-searxng TypeScript server (src/search.ts).

Key behaviours ported from the MCP:

• Full search parameter support (language, time_range, safesearch, categories, engines, pageno, num_results, min_score)
• HTML fallback: always active — when the instance returns 403/404 or a non-JSON body, the client retries without format=json and parses the HTML DOM
• Basic authentication via SEARXNG_AUTH_USERNAME / SEARXNG_AUTH_PASSWORD (or AUTH_USERNAME / AUTH_PASSWORD for MCP compatibility)
• Custom User-Agent via SEARXNG_USER_AGENT / USER_AGENT
• HTTP proxy via SEARXNG_HTTP_PROXY / HTTP_PROXY / HTTPS_PROXY
• Configurable timeout via SEARXNG_TIMEOUT_MS (default 10 000 ms)
• Result truncation via SEARXNG_MAX_RESULTS (operator cap 1–20)
• Per-result char limit via SEARXNG_MAX_RESULT_CHARS

Index

• Variables
• func FormatText(data *SearchResponse) string
• type Client
  • func NewClient() *Client
  • func NewClientForTest(rawURL string, transport http.RoundTripper) *Client
  • func NewClientWithURL(rawURL string) *Client
  • func NewClientWithURLAndAuth(rawURL, username, password string) *Client
  • func (c *Client) IsConfigured() bool
  • func (c *Client) Search(input SearchInput) (*SearchResponse, error)
• type Infobox
• type InfoboxURL
• type PaginationOptions
• type SearchInput
• type SearchResponse
• type URLCache
  • func NewURLCache() *URLCache
  • func (c *URLCache) Close()
  • func (c *URLCache) Get(url string) (string, bool)
  • func (c *URLCache) Set(url, htmlContent, markdownContent string)
• type URLReader
  • func NewURLReader() *URLReader
  • func (r *URLReader) FetchMarkdown(ctx context.Context, rawURL string, opts PaginationOptions) (string, error)
• type WebResult

Variables

var DefaultURLCache = NewURLCache()

DefaultURLCache is the process-wide singleton, matching the MCP's exported urlCache.

Functions

func FormatText

func FormatText(data *SearchResponse) string

FormatText formats a SearchResponse into human-readable text, matching the text output path of performWebSearch in search.ts.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is a stateless SearXNG search client. Create one with NewClient and reuse it across requests.

func NewClient

func NewClient() *Client

NewClient creates a Client configured from environment variables.

Environment variables (in priority order):

• SEARXNG_URL or SEARXNG_BASE_URL — base URL of the SearXNG instance (required)
• SEARXNG_TIMEOUT_MS — request timeout in milliseconds (default 10 000)
• SEARXNG_AUTH_USERNAME / AUTH_USERNAME — basic auth username
• SEARXNG_AUTH_PASSWORD / AUTH_PASSWORD — basic auth password
• SEARXNG_USER_AGENT / USER_AGENT — custom User-Agent header
• SEARXNG_HTTP_PROXY / HTTP_PROXY / HTTPS_PROXY — proxy URL

func NewClientForTest

func NewClientForTest(rawURL string, transport http.RoundTripper) *Client

NewClientForTest creates a minimal Client with a custom http.RoundTripper. Intended for unit tests only; skips env-var reading for transport/auth.

func NewClientWithURL

func NewClientWithURL(rawURL string) *Client

NewClientWithURL creates a Client pointing at a specific base URL. All other settings are read from environment variables.

func NewClientWithURLAndAuth

func NewClientWithURLAndAuth(rawURL, username, password string) *Client

NewClientWithURLAndAuth creates a Client with an explicit base URL and Basic Auth credentials. username and password are empty strings when no auth is needed.

func (*Client) IsConfigured

func (c *Client) IsConfigured() bool

IsConfigured reports whether a base URL has been set.

func (*Client) Search

func (c *Client) Search(input SearchInput) (*SearchResponse, error)

Search executes a search and returns the response. It transparently falls back to HTML parsing when the instance does not support the JSON format endpoint.

type Infobox

type Infobox struct {
	Infobox string       `json:"infobox"`
	Content string       `json:"content,omitempty"`
	URLs    []InfoboxURL `json:"urls,omitempty"`
}

Infobox mirrors SearXNGWebInfobox.

type InfoboxURL

type InfoboxURL struct {
	Title string `json:"title"`
	URL   string `json:"url"`
}

InfoboxURL is a link entry inside an infobox.

type PaginationOptions

type PaginationOptions struct {
	// StartChar is the character offset to start from (0-based).
	StartChar int
	// MaxLength caps the number of characters returned. 0 means no cap.
	MaxLength int
	// Section returns only the content under the heading matching this string.
	Section string
	// ParagraphRange selects specific paragraphs: "3", "1-5", "10-".
	ParagraphRange string
	// ReadHeadings returns only the heading lines instead of full content.
	ReadHeadings bool
}

PaginationOptions controls how extracted content is sliced. Mirrors the PaginationOptions interface in url-reader.ts.

type SearchInput

type SearchInput struct {
	// Query is the search string. Required.
	Query string
	// PageNo is the page number (default 1).
	PageNo int
	// TimeRange restricts results to a time window: "day", "week", "month", "year".
	TimeRange string
	// Language is the BCP-47 language code, e.g. "en", "fr". "all" means no restriction.
	Language string
	// Safesearch filter: 0 = none, 1 = moderate, 2 = strict.
	Safesearch int
	// MinScore filters results whose score is below this threshold (0.0–1.0).
	// Zero means no filter.
	MinScore float64
	// NumResults caps the number of returned results (1–20). Zero means server default.
	NumResults int
	// Categories is a comma-separated list of SearXNG categories, e.g. "general,news".
	Categories string
	// Engines is a comma-separated list of engine names, e.g. "google,bing,ddg".
	Engines string
}

SearchInput holds all parameters for a search request. Zero values are omitted from the query string.

type SearchResponse

type SearchResponse struct {
	Query           string      `json:"query"`
	NumberOfResults int         `json:"number_of_results"`
	Results         []WebResult `json:"results"`
	Suggestions     []string    `json:"suggestions,omitempty"`
	Corrections     []string    `json:"corrections,omitempty"`
	Answers         []string    `json:"answers,omitempty"`
	Infoboxes       []Infobox   `json:"infoboxes,omitempty"`

	// SourceFormat is set internally: "json" when parsed from JSON,
	// "html" when obtained via the HTML fallback path.
	SourceFormat string `json:"-"`
}

SearchResponse is the full JSON payload from SearXNG /search?format=json.

type URLCache

type URLCache struct {
	// contains filtered or unexported fields
}

URLCache is a TTL + LFU (least-frequently-used on tie) in-memory cache. Mirrors SimpleCache from cache.ts.

func NewURLCache

func NewURLCache() *URLCache

NewURLCache creates a cache with TTL and max size read from env vars, matching the MCP's CACHE_TTL_MS and CACHE_MAX_ENTRIES.

func (*URLCache) Close

func (c *URLCache) Close()

Close stops the background cleanup goroutine.

func (*URLCache) Get

func (c *URLCache) Get(url string) (string, bool)

Get retrieves cached markdown content for url. Returns ("", false) on miss or expiry.

func (*URLCache) Set

func (c *URLCache) Set(url, htmlContent, markdownContent string)

Set stores the HTML and its markdown conversion for url.

type URLReader

type URLReader struct {
	// contains filtered or unexported fields
}

URLReader fetches and converts URLs to readable text.

func NewURLReader

func NewURLReader() *URLReader

NewURLReader creates a URLReader sharing the same auth / proxy config as the searxng Client.

func (*URLReader) FetchMarkdown

func (r *URLReader) FetchMarkdown(ctx context.Context, rawURL string, opts PaginationOptions) (string, error)

FetchMarkdown fetches the URL and returns its content as readable text. The content is cached (DefaultURLCache) for subsequent calls.

type WebResult

type WebResult struct {
	Title         string   `json:"title"`
	URL           string   `json:"url"`
	Content       string   `json:"content"`
	Score         float64  `json:"score,omitempty"`
	Engine        string   `json:"engine,omitempty"`
	Engines       []string `json:"engines,omitempty"`
	Category      string   `json:"category,omitempty"`
	PublishedDate string   `json:"publishedDate,omitempty"`
	Thumbnail     string   `json:"thumbnail,omitempty"`
}

WebResult mirrors SearXNGWebResult from the MCP types.ts.