Directories
¶
| Path | Synopsis |
|---|---|
|
Package adminui serves the yago operator console as a server-rendered surface styled with IBM Carbon design tokens and progressively enhanced with htmx.
|
Package adminui serves the yago operator console as a server-rendered surface styled with IBM Carbon design tokens and progressively enhanced with htmx. |
|
Package boltvault is the bbolt implementation of the vault Engine.
|
Package boltvault is the bbolt implementation of the vault Engine. |
|
Package bootstrap fetches the configured YaCy seed lists and returns the seeds they advertise.
|
Package bootstrap fetches the configured YaCy seed lists and returns the seeds they advertise. |
|
Package cachedpage serves read-only cached copies of locally stored pages on the public listener: the text this node extracted at crawl time, rendered escaped, so a searcher can inspect a result without visiting the origin site.
|
Package cachedpage serves read-only cached copies of locally stored pages on the public listener: the text this node extracted at crawl time, rendered escaped, so a searcher can inspect a result without visiting the origin site. |
|
Package contentquality gates crawled text before it is stored and indexed, using the deterministic token-based subset of the Gopher (Rae et al., arXiv:2112.11446, Appendix A) and C4 (Raffel et al., arXiv:1910.10683) quality rules that FineWeb (arXiv:2406.17557) validated at web scale.
|
Package contentquality gates crawled text before it is stored and indexed, using the deterministic token-based subset of the Gopher (Rae et al., arXiv:2112.11446, Appendix A) and C4 (Raffel et al., arXiv:1910.10683) quality rules that FineWeb (arXiv:2406.17557) validated at web scale. |
|
Package crawlbroker is the node's gRPC edge to the crawl fleet.
|
Package crawlbroker is the node's gRPC edge to the crawl fleet. |
|
Package crawldispatch turns an operator's request to crawl seed URLs into a CrawlOrder and hands it to the crawl fleet.
|
Package crawldispatch turns an operator's request to crawl seed URLs into a CrawlOrder and hands it to the crawl fleet. |
|
Package crawlformats persists the operator's shared document-format toggles: which format families every crawl parses (YaCy TextParser parity).
|
Package crawlformats persists the operator's shared document-format toggles: which format families every crawl parses (YaCy TextParser parity). |
|
Package crawling owns the crawlReceipt endpoint, which rejects remote crawl receipts until an operator policy enables remote crawl work.
|
Package crawling owns the crawlReceipt endpoint, which rejects remote crawl receipts until an operator policy enables remote crawl work. |
|
Package crawlresults absorbs ingest batches returned by the crawl fleet, storing their URL metadata and then their postings through the node's existing receivers.
|
Package crawlresults absorbs ingest batches returned by the crawl fleet, storing their URL metadata and then their postings through the node's existing receivers. |
|
Package crawlruns is the node's in-memory registry of crawl runs.
|
Package crawlruns is the node's in-memory registry of crawl runs. |
|
Package crawlschedule stores recurring crawl definitions and decides when each is due — YaCy's Automation_p (recorded API calls with a recurring schedule) parity for the one action worth repeating: re-crawling a site (UI-19).
|
Package crawlschedule stores recurring crawl definitions and decides when each is due — YaCy's Automation_p (recorded API calls with a recurring schedule) parity for the one action worth repeating: re-crawling a site (UI-19). |
|
Package crossorigin applies a configurable, default-deny CORS policy to an HTTP handler.
|
Package crossorigin applies a configurable, default-deny CORS policy to an HTTP handler. |
|
Package dhtexchange builds outbound YaCy DHT exchange batches.
|
Package dhtexchange builds outbound YaCy DHT exchange batches. |
|
Package documentsearch finds documents containing wanted terms, orders them by relevance, and reports how many documents matched each term.
|
Package documentsearch finds documents containing wanted terms, orders them by relevance, and reports how many documents matched each term. |
|
Package events records recent structured node events in a bounded in-memory ring so operators and the admin UI can review important activity without scraping logs.
|
Package events records recent structured node events in a bounded in-memory ring so operators and the admin UI can review important activity without scraping logs. |
|
Package eventstore is a durable, bounded log of structured node events so the admin event log survives restarts.
|
Package eventstore is a durable, bounded log of structured node events so the admin event log survives restarts. |
|
Package eviction frees storage when the vault nears its quota.
|
Package eviction frees storage when the vault nears its quota. |
|
Package faviconproxy serves result favicons through this node, so the hosts behind search results never see the searcher's browser before a click (the privacy property YaCy gets from its ViewImage proxy).
|
Package faviconproxy serves result favicons through this node, so the hosts behind search results never see the searcher's browser before a click (the privacy property YaCy gets from its ViewImage proxy). |
|
Package filetypeclass classifies a crawled document's file type from its Content-Type and URL, so the filetype: search operator and the filetype navigation facet recognise a document by what it actually is rather than only by a file extension in its URL.
|
Package filetypeclass classifies a crawled document's file type from its Content-Type and URL, so the filetype: search operator and the filetype navigation facet recognise a document by what it actually is rather than only by a file extension in its URL. |
|
Package httpguard enforces the HTTP request invariants every module endpoint shares: method set, body limit, and request timeout.
|
Package httpguard enforces the HTTP request invariants every module endpoint shares: method set, body limit, and request timeout. |
|
Package indextransfer owns outbound YaCy index transfer HTTP calls.
|
Package indextransfer owns outbound YaCy index transfer HTTP calls. |
|
Package judgments persists operator-curated relevance judgments — a query paired with graded result URLs — as the human-labelled training and regression set the ranking learner (YagoRank, ADR-0035) fits weights against.
|
Package judgments persists operator-curated relevance judgments — a query paired with graded result URLs — as the human-labelled training and regression set the ranking learner (YagoRank, ADR-0035) fits weights against. |
|
Package landing serves the node's static landing page on GET and HEAD.
|
Package landing serves the node's static landing page on GET and HEAD. |
|
Package landiscovery finds peers on the local network the way Syncthing's local discovery does: a small UDP beacon broadcast on a fixed port announces this node's hash and peer port, and received announcements from other nodes hand a candidate endpoint to the greeter, which runs the ordinary hello exchange — the beacon itself is never trusted for anything but «someone at this address claims to be a peer», so a forged packet can at most trigger one verified greet.
|
Package landiscovery finds peers on the local network the way Syncthing's local discovery does: a small UDP beacon broadcast on a fixed port announces this node's hash and peer port, and received announcements from other nodes hand a candidate endpoint to the greeter, which runs the ordinary hello exchange — the beacon itself is never trusted for anything but «someone at this address claims to be a peer», so a forged packet can at most trigger one verified greet. |
|
Package memvault is the in-memory implementation of the vault Engine.
|
Package memvault is the in-memory implementation of the vault Engine. |
|
Package metrichistory samples a curated set of the node's Prometheus metrics into a bounded in-memory ring, so the admin Performance page can render a short operational history (request throughput, error rate, latency, DHT transfer rate, queue depths) without an external metrics stack.
|
Package metrichistory samples a curated set of the node's Prometheus metrics into a bounded in-memory ring, so the admin Performance page can render a short operational history (request throughput, error rate, latency, DHT transfer rate, queue depths) without an external metrics stack. |
|
Package metrics translates domain results into Prometheus series.
|
Package metrics translates domain results into Prometheus series. |
|
Package modifierhint supplies the guidance shown when a search carried a filter operator (filetype:, site:, tld:, inurl:) but no free-text term and so matched nothing.
|
Package modifierhint supplies the guidance shown when a search carried a filter operator (filetype:, site:, tld:, inurl:) but no free-text term and so matched nothing. |
|
Package nodeidentity holds this node's self-description: the hash, network, and seed attributes that identify it, and the rule for whether a peer request addresses it.
|
Package nodeidentity holds this node's self-description: the hash, network, and seed attributes that identify it, and the rule for whether a peer request addresses it. |
|
Package nodestatus owns the node's runtime status: its self-seed, the version/uptime header every endpoint echoes, and the query.html capacity answers.
|
Package nodestatus owns the node's runtime status: its self-seed, the version/uptime header every endpoint echoes, and the query.html capacity answers. |
|
Package peeradmission answers inbound hello requests: it classifies the calling peer by probing it back, and returns a bounded freshest set of reachable peers.
|
Package peeradmission answers inbound hello requests: it classifies the calling peer by probing it back, and returns a bounded freshest set of reachable peers. |
|
Package peerannouncement greets known peers on an interval: it announces this node to them and reports their reachability to the peer roster.
|
Package peerannouncement greets known peers on an interval: it announces this node to them and reports their reachability to the peer roster. |
|
Package peerbirth keeps this peer's birth date: the moment the node first started with its data directory.
|
Package peerbirth keeps this peer's birth date: the moment the node first started with its data directory. |
|
Package peerblock keeps a durable operator-managed blocklist of peer hashes.
|
Package peerblock keeps a durable operator-managed blocklist of peer hashes. |
|
Package peeridentity keeps this peer's stable identity — its hash and name — so a node bootstraps without the operator having to supply them.
|
Package peeridentity keeps this peer's stable identity — its hash and name — so a node bootstraps without the operator having to supply them. |
|
Package peernews keeps the YaCy peer news pool: news records this peer publishes to the network and news records that arrive attached to other peers' seeds.
|
Package peernews keeps the YaCy peer news pool: news records this peer publishes to the network and news records that arrive attached to other peers' seeds. |
|
Package peerroster owns the set of network peers this node knows.
|
Package peerroster owns the set of network peers this node knows. |
|
Package portaltheme stores and renders the operator-editable public portal theme (ADR-0033): Handlebars templates for the search and results pages plus a shared styles block, persisted in the durable vault and rendered server-side.
|
Package portaltheme stores and renders the operator-editable public portal theme (ADR-0033): Handlebars templates for the search and results pages plus a shared styles block, persisted in the durable vault and rendered server-side. |
|
Package publicportal serves the anonymous public search portal: a minimal, server-rendered search page on the node's public HTTP listener that works without JavaScript and in legacy browsers.
|
Package publicportal serves the anonymous public search portal: a minimal, server-rendered search page on the node's public HTTP listener that works without JavaScript and in legacy browsers. |
|
Package publicratelimit throttles the anonymous public search surfaces the way YaCy's search.public.max.access tiers do: per-client sliding windows at three second, one minute, and ten minute horizons, with raised limits for authenticated callers (a valid bearer key or the local operator).
|
Package publicratelimit throttles the anonymous public search surfaces the way YaCy's search.public.max.access tiers do: per-client sliding windows at three second, one minute, and ten minute horizons, with raised limits for authenticated callers (a valid bearer key or the local operator). |
|
Package publicrobots serves the node's own /robots.txt — YaCy's ConfigRobotsTxt_p parity.
|
Package publicrobots serves the node's own /robots.txt — YaCy's ConfigRobotsTxt_p parity. |
|
Package rankfit fits the ranking weight profile to measured relevance (YagoRank, ADR-0035).
|
Package rankfit fits the ranking weight profile to measured relevance (YagoRank, ADR-0035). |
|
Package rankingprofile persists the node's default search ranking weights and serves them live to the local searcher, so an operator can retune ranking without a restart and the setting survives one.
|
Package rankingprofile persists the node's default search ranking weights and serves them live to the local searcher, so an operator can retune ranking without a restart and the setting survives one. |
|
Package readpriority defers a bulk writer while interactive reads are in flight, so searches and admin pages reach the disk ahead of ingest writes and their fsyncs (IO-PRIO-01, made a configurable, observable gate for PERF-PRIO-02).
|
Package readpriority defers a bulk writer while interactive reads are in flight, so searches and admin pages reach the disk ahead of ingest writes and their fsyncs (IO-PRIO-01, made a configurable, observable gate for PERF-PRIO-02). |
|
Package recrawlfrontier keeps a durable, node-side recrawl schedule: for each crawled URL it records when the page was last fetched and when it is next due for a recrawl, so pages are refetched on their profile's cadence and the schedule survives node and crawler restarts.
|
Package recrawlfrontier keeps a durable, node-side recrawl schedule: for each crawled URL it records when the page was last fetched and when it is next due for a recrawl, so pages are refetched on their profile's cadence and the schedule survives node and crawler restarts. |
|
Package redirectpurge sweeps already-indexed search-engine click-tracking documents out of the live corpus (SEARCH-29).
|
Package redirectpurge sweeps already-indexed search-engine click-tracking documents out of the live corpus (SEARCH-29). |
|
Package remotecrawl owns opt-in YaCy remote crawl delegation.
|
Package remotecrawl owns opt-in YaCy remote crawl delegation. |
|
Package rwi owns RWI posting intake, storage, search, and eviction.
|
Package rwi owns RWI posting intake, storage, search, and eviction. |
|
Package searchactivity keeps a bounded in-memory journal of recent search requests for the admin console — YaCy's AccessTracker_p parity (UI-16).
|
Package searchactivity keeps a bounded in-memory journal of recent search requests for the admin console — YaCy's AccessTracker_p parity (UI-16). |
|
Package searcheval scores a searcher's ranking quality with NDCG@k so a test can gate ranking changes against regression, and turns query→relevant-URL labels (including corpus-derived pseudo-labels) into judgments.
|
Package searcheval scores a searcher's ranking quality with NDCG@k so a test can gate ranking changes against regression, and turns query→relevant-URL labels (including corpus-derived pseudo-labels) into judgments. |
|
Package searchsession keeps paging stable by caching assembled result prefixes and extending them in bounded windows when a deeper page needs more results.
|
Package searchsession keeps paging stable by caching assembled result prefixes and extending them in bounded windows when a deeper page needs more results. |
|
Package seedimport keeps a durable per-URL record of the last seed-list import so the admin console can show when each configured seed list was last refreshed and whether it succeeded.
|
Package seedimport keeps a durable per-URL record of the last seed-list import so the admin console can show when each configured seed list was last refreshed and whether it succeeded. |
|
Package settingsstore is a durable, admin-writable key/value store for the node's operator-overridable runtime settings.
|
Package settingsstore is a durable, admin-writable key/value store for the node's operator-overridable runtime settings. |
|
Package shardvault is the sharded, compressed bbolt implementation of the vault Engine (ADR-0025).
|
Package shardvault is the sharded, compressed bbolt implementation of the vault Engine (ADR-0025). |
|
Package snippetmark highlights query terms in result snippets for the human search surfaces.
|
Package snippetmark highlights query terms in result snippets for the human search surfaces. |
|
Package spellcheck corrects query spelling against the node's own indexed vocabulary with the Symmetric Delete algorithm (SymSpell, Garbe 2012): the dictionary is precomputed once so a lookup only deletes characters from the query term, never the dictionary, making correction fast enough to run on every zero-result query.
|
Package spellcheck corrects query spelling against the node's own indexed vocabulary with the Symmetric Delete algorithm (SymSpell, Garbe 2012): the dictionary is precomputed once so a lookup only deletes characters from the query term, never the dictionary, making correction fast enough to run on every zero-result query. |
|
Package stopwords carries a compact multilingual function-word list for query-time decisions: which query words may be required of documents, verified against peer results, anchor snippets, or drive highlighting.
|
Package stopwords carries a compact multilingual function-word list for query-time decisions: which query words may be required of documents, verified against peer results, anchor snippets, or drive highlighting. |
|
Package tracectx implements the W3C Trace Context header (traceparent) for the node's own request legs — enough to correlate one public search with the peer fan-out it caused and to attach exemplars to latency histograms (OPS-10).
|
Package tracectx implements the W3C Trace Context header (traceparent) for the node's own request legs — enough to correlate one public search with the peer fan-out it caused and to attach exemplars to latency histograms (OPS-10). |
|
Package transfertally keeps the running and persisted totals of this peer's YaCy index exchange for the advertised seed statistics.
|
Package transfertally keeps the running and persisted totals of this peer's YaCy index exchange for the advertised seed statistics. |
|
Package urldenylist keeps a durable, operator-managed denylist of URLs and whole domains.
|
Package urldenylist keeps a durable, operator-managed denylist of URLs and whole domains. |
|
Package urlmeta owns the transferURL endpoint, URL intake, and URL metadata storage and lookup.
|
Package urlmeta owns the transferURL endpoint, URL intake, and URL metadata storage and lookup. |
|
Package urlmetastaleness orders url metadata from stalest to freshest so eviction can name the stalest urls without decompressing a single row.
|
Package urlmetastaleness orders url metadata from stalest to freshest so eviction can name the stalest urls without decompressing a single row. |
|
Package urlreferences knows, for every url, the words that reference it, so eviction can drop a url's postings without scanning the word index.
|
Package urlreferences knows, for every url, the words that reference it, so eviction can drop a url's postings without scanning the word index. |
|
Package vault lends each module a transactionally-scoped, codec-typed collection over its own ordered byte buckets.
|
Package vault lends each module a transactionally-scoped, codec-typed collection over its own ordered byte buckets. |
|
Package vaulttest holds the storage-contract suite every vault Engine driver runs against itself.
|
Package vaulttest holds the storage-contract suite every vault Engine driver runs against itself. |
|
Package websearch provides optional, admin-gated web retrieval after a local and federated miss or alongside those sources in always mode.
|
Package websearch provides optional, admin-gated web retrieval after a local and federated miss or alongside those sources in always mode. |
|
Package wordforms expands a query word into the inflected surface forms that actually occur in the node's index.
|
Package wordforms expands a query word into the inflected surface forms that actually occur in the node's index. |
Click to show internal directories.
Click to hide internal directories.