ai

package
v0.80.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 4, 2026 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

Package ai is a small OpenAI-compatible LLM client with a configurable fallback chain, used to turn a messy torrent name into a clean title before the TMDB lookup. It's deliberately narrow: one task (title identification), no tool-calling loop. The chain is walked in order, a per-slot circuit breaker skips known-down providers, and the first usable answer wins.

Index

Constants

View Source 
const (
	OutcomeOK         = "ok"
	OutcomeIncomplete = "incomplete"
	OutcomeError      = "error"
)

Run outcome labels. A run is OK when it produced a complete, usable measurement; INCOMPLETE when transiently cut short (rate limit) — the "faltante" state; ERROR when it yielded no usable reply at all (hard failure). These drive the durable per-slot history so the UI can show success/error status, error persistence, and the date of the last success.

View Source 
const (
	TaskRename   = "rename"   // full metadata extraction (title+year+kind+season+episode)
	TaskIdentify = "identify" // title-only extraction (categoryFromAI / art fail-safe)
	TaskSchedule = "schedule" // natural-language schedule → {kind,weekday,hour,minute}
)

Task ids. TaskRename is the default (empty Task string normalizes to it) so the existing single-task dataset and any user-saved set keep working unchanged.

View Source 
const OriginDefault = "default"

OriginDefault tags a BenchmarkCase that ships with JackUI (vs a user-added one, whose Origin is empty). Informational only — scoring ignores it.

Variables

View Source 
var DefaultBenchmarkCases = markDefault([]BenchmarkCase{

	{Raw: "Inception.2010.1080p.BluRay.x264-SPARKS", Expect: "Inception - 2010"},
	{Raw: "The.Matrix.1999.2160p.UHD.BluRay.x265-TERMINAL", Expect: "The Matrix - 1999"},
	{Raw: "Dune.Part.Two.2024.1080p.WEB-DL.DDP5.1.Atmos.H.264-FLUX", Expect: "Dune Part Two - 2024"},
	{Raw: "Oppenheimer.2023.1080p.WEBRip.x264-RARBG", Expect: "Oppenheimer - 2023"},
	{Raw: "The.Shawshank.Redemption.1994.720p.BrRip.x264.YIFY", Expect: "The Shawshank Redemption - 1994"},
	{Raw: "Everything.Everywhere.All.at.Once.2022.1080p.AMZN.WEB-DL.DDP5.1.H.264-CMRG", Expect: "Everything Everywhere All at Once - 2022"},

	{Raw: "Spider-Man_Across_the_Spider-Verse_2023_1080p_WEBRip_x265", Expect: "Spider-Man Across the Spider-Verse - 2023"},
	{Raw: "john wick chapter 4 (2023) [1080p] [BluRay] [5.1] [YTS.MX]", Expect: "John Wick Chapter 4 - 2023"},
	{Raw: "The-Grand-Budapest-Hotel-2014-1080p-BluRay-x264", Expect: "The Grand Budapest Hotel - 2014"},

	{Raw: "1917.2019.1080p.BluRay.x264-AAA", Expect: "1917 - 2019"},
	{Raw: "Blade.Runner.2049.2017.2160p.UHD.BluRay.x265-IAMABLE", Expect: "Blade Runner 2049 - 2017"},
	{Raw: "Wonder.Woman.1984.2020.1080p.HMAX.WEB-DL.DDP5.1.Atmos.x264-EVO", Expect: "Wonder Woman 1984 - 2020"},
	{Raw: "2001.A.Space.Odyssey.1968.2160p.UHD.BluRay.x265-DEPTH", Expect: "2001 A Space Odyssey - 1968"},
	{Raw: "2012.2009.1080p.BluRay.x264-METiS", Expect: "2012 - 2009"},
	{Raw: "Death.Race.2000.1975.1080p.BluRay.x264-USURY", Expect: "Death Race 2000 - 1975"},

	{Raw: "Interstellar.2014.2160p.UHD.BluRay.REMUX.HDR.HEVC.Atmos-EPSiLON", Expect: "Interstellar - 2014"},
	{Raw: "Mad.Max.Fury.Road.2015.2160p.BluRay.x265.10bit.HDR10Plus.DTS-HD.MA.7.1-SWTYBLZ", Expect: "Mad Max Fury Road - 2015"},
	{Raw: "Avatar.The.Way.of.Water.2022.2160p.WEB-DL.DV.HDR.HEVC.DDP5.1.Atmos-FLUX", Expect: "Avatar The Way of Water - 2022"},

	{Raw: "Aliens.1986.Special.Edition.1080p.BluRay.x264-GECKOS", Expect: "Aliens - 1986"},
	{Raw: "The.Lord.of.the.Rings.The.Two.Towers.2002.EXTENDED.1080p.BluRay.x264-SiNNERS", Expect: "The Lord of the Rings The Two Towers - 2002"},
	{Raw: "Apocalypse.Now.1979.Final.Cut.PROPER.REPACK.1080p.BluRay.x264-CiNEFiLE", Expect: "Apocalypse Now - 1979"},

	{Raw: "Breaking.Bad.S03E07.720p.HDTV.x264-CTU", Expect: "Breaking Bad - S03E07"},
	{Raw: "Game.of.Thrones.S01E09.Baelor.1080p.BluRay.x264-DEMAND", Expect: "Game of Thrones - S01E09"},
	{Raw: "The.Last.of.Us.S02E03.1080p.WEB.H264-SuccessfulCrab", Expect: "The Last of Us - S02E03"},
	{Raw: "severance.s02e05.1080p.web.h264-successfulcrab", Expect: "Severance - S02E05"},
	{Raw: "The.Bear.S03E01.Tomorrow.1080p.HULU.WEB-DL.DDP5.1.H.264-FLUX", Expect: "The Bear - S03E01"},
	{Raw: "Stranger.Things.4x09.The.Piggyback.2160p.NF.WEB-DL", Expect: "Stranger Things - S04E09"},
	{Raw: "The.Office.US.S05E14.Stress.Relief.720p.HDTV.x264", Expect: "The Office US - S05E14"},

	{Raw: "The.Wire.S04.COMPLETE.1080p.BluRay.x264-AVCHD", Expect: "The Wire - S04"},
	{Raw: "Chernobyl.S01.COMPLETE.2160p.UHD.BluRay.x265-AJP69", Expect: "Chernobyl - S01"},
	{Raw: "Dark.S03.German.1080p.NF.WEB-DL.DD5.1.x264-TEPES", Expect: "Dark - S03"},
	{Raw: "Friends.Season.2.Complete.720p.BluRay.x264-PSYCHD", Expect: "Friends - S02"},

	{Raw: "[SubsPlease] Sousou no Frieren - 05 (1080p) [F02B9CEE].mkv", Expect: "Sousou no Frieren - E05"},
	{Raw: "[Erai-raws] Frieren - 01 [1080p][Multiple Subtitle]", Expect: "Frieren - E01"},
	{Raw: "[Judas] Jujutsu Kaisen S2 - 23 [1080p][HEVC x265 10bit][Multi-Subs]", Expect: "Jujutsu Kaisen - S02E23"},
	{Raw: "[HorribleSubs] One Punch Man - 12 [720p].mkv", Expect: "One Punch Man - E12"},
	{Raw: "Kimetsu.no.Yaiba.S04E08.1080p.CR.WEB-DL.AAC2.0.H.264-VARYG", Expect: "Kimetsu no Yaiba - S04E08"},

	{Raw: "O.Auto.da.Compadecida.2000.DUBLADO.1080p", Expect: "O Auto da Compadecida - 2000"},
	{Raw: "Cidade.de.Deus.2002.NACIONAL.1080p.BluRay.x264", Expect: "Cidade de Deus - 2002"},
	{Raw: "Ainda.Estou.Aqui.2024.1080p.AMZN.WEB-DL.DDP5.1.DUAL-DUBLADO", Expect: "Ainda Estou Aqui - 2024"},
	{Raw: "Tropa.de.Elite.2.2010.NACIONAL.720p.BluRay.x264-CiNEFiLHOS", Expect: "Tropa de Elite 2 - 2010"},
	{Raw: "Divertida.Mente.2.2024.DUBLADO.DUAL.AUDIO.1080p.WEB-DL", Expect: "Divertida Mente 2 - 2024"},
	{Raw: "Round.6.S03E01.DUAL.1080p.NF.WEB-DL.DDP5.1.Atmos-NTb", Expect: "Round 6 - S03E01"},

	{Raw: "Amelie.2001.FRENCH.1080p.BluRay.x264-LOST", Expect: "Amélie - 2001"},
	{Raw: "La.Casa.de.Papel.S02E03.SPANISH.1080p.NF.WEB-DL.DD5.1.x264-PECULATE", Expect: "La Casa de Papel - S02E03"},
	{Raw: "Sen.to.Chihiro.no.Kamikakushi.2001.JAPANESE.1080p.BluRay.x264-MOOVEE", Expect: "Sen to Chihiro no Kamikakushi - 2001"},
	{Raw: "Oldboy.2003.KOREAN.REMASTERED.1080p.BluRay.x264-GiMCHi", Expect: "Oldboy - 2003"},

	{Raw: "Planet.Earth.II.S01E01.Islands.2160p.UHD.BluRay.x265-WhiteRhino", Expect: "Planet Earth II - S01E01"},
	{Raw: "Free.Solo.2018.1080p.BluRay.x264-ROVERS", Expect: "Free Solo - 2018"},
	{Raw: "Senna.2010.DOCUMENTARY.1080p.BluRay.x264-HAGGiS", Expect: "Senna - 2010"},

	{Raw: "UFC.300.Pereira.vs.Hill.PPV.2024.04.13.1080p.WEB.h264-VERUM", Expect: "UFC 300 Pereira vs Hill - 2024"},
	{Raw: "Formula1.2024.Round22.Las.Vegas.Grand.Prix.Race.SkyF1HD.1080p", Expect: "Formula 1 Las Vegas Grand Prix Race - 2024"},
	{Raw: "WWE.WrestleMania.40.Night.1.PPV.2024.1080p.PCOK.WEB-DL.H264", Expect: "WWE WrestleMania 40 Night 1 - 2024"},

	{Raw: "Pink.Floyd.The.Dark.Side.Of.The.Moon.1973.FLAC.24bit-96kHz", Expect: "Pink Floyd - The Dark Side of the Moon - 1973"},
	{Raw: "Metallica - Discography (1983-2016) [320kbps]", Expect: "Metallica"},

	{Raw: "StudioA.24.03.15.Jane.Doe.Wet.And.Wild.XXX.1080p.MP4-XXX", Expect: "StudioA - Jane Doe - Wet And Wild"},
	{Raw: "ArtHouse.23.11.02.Mary.Major.Morning.Coffee.XXX.2160p.MP4-WRB", Expect: "ArtHouse - Mary Major - Morning Coffee"},
	{Raw: "GlamourPlus.22.07.19.Anna.Bloggs.Natural.Beauty.XXX.1080p", Expect: "GlamourPlus - Anna Bloggs - Natural Beauty"},

	{Raw: "www.Torrenting.com - The.Batman.2022.1080p.WEBRip.x264-YIFY", Expect: "The Batman - 2022"},
	{Raw: "www.UIndex.org    -    Gladiator.II.2024.1080p.WEB-DL.H264-ETHEL", Expect: "Gladiator II - 2024"},
	{Raw: "[ Torrent911.my ] Le.Comte.de.Monte-Cristo.2024.FRENCH.1080p.WEB.H264-FW", Expect: "Le Comte de Monte-Cristo - 2024"},

	{Raw: "Up.2009.1080p.BluRay.x264-CBGB", Expect: "Up - 2009"},
	{Raw: "It.2017.1080p.BluRay.x264-SPARKS", Expect: "It - 2017"},
	{Raw: "Her.2013.720p.BluRay.x264-GECKOS", Expect: "Her - 2013"},
	{Raw: "Us.2019.1080p.WEB-DL.DD5.1.H264-CMRG", Expect: "Us - 2019"},
})

DefaultBenchmarkCases seeds a fresh store. The set is deliberately broad — every category below exists because the tiny original 7-case set made models look better than they were: it had no season packs, no year-in-title traps, no BR/dubbed releases, no site-tag prefixes, no live events, no adult-scene naming — exactly the inputs weak models botch in production. Expects use the canonical label parsed by parseExpect: 

Movie:        "Inception - 2010"       (Título - Ano; year informational, not scored)
TV episode:   "Breaking Bad - S03E07"  (title 60% + season/episode 40%)
Episode only: "Frieren - E01"          (anime absolute numbering)
Season pack:  "The Wire - S04"         (title 60% + season 40%)
Plain title:  "Metallica"              (title only)

GUARD: none of these raws may appear as a few-shot example inside renameSystem/identifySystem — that would let models copy the answer from the prompt and inflate the benchmark (enforced by TestDefaultCasesNotInPrompts).

View Source 
var DefaultIdentifyCases = markDefaultTask([]BenchmarkCase{
	{Raw: "Sicario.2015.1080p.BluRay.x264-SPARKS", Expect: "Sicario"},
	{Raw: "Arrival.2016.2160p.UHD.BluRay.x265-TERMINAL", Expect: "Arrival"},
	{Raw: "The.Witcher.S02E01.1080p.NF.WEB-DL.DDP5.1.x264-NTb", Expect: "The Witcher"},
	{Raw: "Parasite.2019.KOREAN.1080p.BluRay.x264-REGRET", Expect: "Parasite"},
	{Raw: "[SubsPlease] Chainsaw Man - 04 (1080p) [9A8B7C6D].mkv", Expect: "Chainsaw Man"},
	{Raw: "Cidade.Baixa.2005.NACIONAL.1080p.WEB-DL", Expect: "Cidade Baixa"},
}, TaskIdentify)

DefaultIdentifyCases seed the IDENTIFY task — the title-ONLY extraction used by categoryFromAI and the art fail-safe chain. Scored on the title alone (no season/episode), so Expect is a bare title (the "- YYYY" tail is informational). Distinct raws from the rename set so the two tasks measure different inputs.

GUARD: not in identifySystem's prompt — enforced by TestIdentifyCasesNotInPrompt.

View Source 
var DefaultScheduleCases = markDefaultTask([]BenchmarkCase{

	{Raw: "checar toda segunda-feira às 06h45", Expect: "weekly:1:6:45"},
	{Raw: "rodar nas terças-feiras 19h", Expect: "weekly:2:19:0"},
	{Raw: "às quartas-feiras de manhã", Expect: "weekly:3:8:0"},
	{Raw: "atualizar quinta-feira às 23h30", Expect: "weekly:4:23:30"},
	{Raw: "todo sábado às 10h", Expect: "weekly:6:10:0"},
	{Raw: "aos domingos de tarde", Expect: "weekly:0:14:0"},
	{Raw: "check it every Friday at 8pm", Expect: "weekly:5:20:0"},

	{Raw: "uma vez por dia às 05h00", Expect: "daily:5:0"},
	{Raw: "diariamente às 23:15", Expect: "daily:23:15"},
	{Raw: "check once a day at 6pm", Expect: "daily:18:0"},

	{Raw: "a cada 2 horas", Expect: "interval:120"},
	{Raw: "três vezes por dia", Expect: "interval:480"},
	{Raw: "every 30 minutes", Expect: "interval:30"},
}, TaskSchedule)

DefaultScheduleCases seed the SCHEDULE task — the natural-language → schedule parse used by the watchlist. They exist because the chain used to be ranked on rename ALONE, so a model that's terrible at schedules could sit at the top and turn "Toda segunda-feira às 07h00" into a daily 07:00 (bug A). Expect uses the compact label parsed by parseScheduleExpect: "weekly:<weekday>:<HH>:<MM>", "daily:<HH>:<MM>", "interval:<minutes>" (weekday 0=Sun … 6=Sat). The weekday is scored heavily, so a wrong day (or a daily-instead-of-weekly) genuinely hurts.

GUARD: none of these raws may appear verbatim in scheduleSystem's few-shot set (would let a model copy the answer) — enforced by TestScheduleCasesNotInPrompt.

View Source 
var ErrInvalidSchedule = errors.New("ai: text is not a recognizable schedule")

ErrInvalidSchedule means the model understood the request but the text isn't a recognizable schedule (or no model produced usable JSON). Handlers map it to 422 so the UI can ask the user to rephrase.

Functions

func DefaultBenchmarkStorePath 

func DefaultBenchmarkStorePath(dataDir string) string

DefaultBenchmarkStorePath returns the standard location inside the data dir.

func NeedsRerun 

func NeedsRerun(s SlotScore) bool

AdoptBenchmark rebuilds the live chain from benchmark scores: every model that produced a usable reply (Samples>0), ordered best-first by composite. This is what the user wants — "use the best benchmark" — while keeping the free local models in the chain as low-ranked fallbacks (the breaker skips a rate-limited vendor at runtime, falling through to the next, ultimately the free local). RerunIncomplete re-benchmarks ONLY the models flagged Incomplete in prev (cases transiently skipped, typically rate-limited) and merges the fresh scores over prev. This backs the "Rodar faltantes" button: run it later — a day after, even — so the retry lands OUTSIDE the rate-limit window and the model finally gets a complete score. Paid models are filtered out (never spend on them); if nothing is incomplete, prev is returned unchanged. NeedsRerun reports whether a result is worth re-running via "Rodar faltantes": either the model was left Incomplete (some cases transiently skipped) OR it failed with a rate limit. The rate-limit check also catches results persisted BEFORE the Incomplete flag existed, so the button works on pre-existing data without forcing a full re-run first. A hard failure (bad output) is NOT re-runnable — re-trying won't change a model that genuinely can't comply.

func RankBefore 

func RankBefore(a, b SlotScore) bool

func RunOutcome 

func RunOutcome(s SlotScore) string

RunOutcome classifies a freshly-measured score into ok/incomplete/error. Order matters: a partially-measured run is "incomplete" (the re-runnable faltante state) even if it gathered some samples; only a run with zero usable replies and no transient cut is a hard "error".

Types

type BenchmarkCase 

type BenchmarkCase struct {
	Raw    string `json:"raw"`
	Expect string `json:"expect"`
	// Task selects which AI task this case measures: "rename" (default), "identify"
	// or "schedule" (see TaskRename/…). Empty normalizes to "rename" so legacy cases
	// (no Task column) and the UI's plain-title textarea keep scoring the rename task
	// exactly as before — the multi-task framework is additive, not a break.
	Task string `json:"task,omitempty"`
	// Origin is "default" for the built-in set (see OriginDefault) and empty for
	// user-added cases. Informational — scoring ignores it, and the UI's textarea
	// editor doesn't round-trip it, so any user-saved set simply becomes custom.
	Origin string `json:"origin,omitempty"`
}

BenchmarkCase is one labelled example: a raw torrent/release name and the canonical label we expect the model to extract. The set is user-editable (persisted in the benchmark store) — that's the "modifiable" part: tune it to the kind of releases you actually download and the chain re-ranks for them.

Expect carries the STRUCTURE inline, in the same canonical form the rename feature produces, so examples and results are coherent with séries/temporadas/ episódios (parsed by parseExpect at scoring time — no schema migration):

  • Movie: "Inception - 2010" (Título - Ano)
  • TV episode: "Breaking Bad - S03E07" (Série - Temporada/Episódio)
  • TV (no season): "Frieren - E01" (Série - Episódio)
  • Season pack: "The Wire - S04" (Série - Temporada, sem episódio)
  • Plain title: "Inception" (sem estrutura → só o título conta)

func AllDefaultBenchmarkCases 

func AllDefaultBenchmarkCases() []BenchmarkCase

AllDefaultBenchmarkCases is the full multi-task seed: rename + schedule + identify. The store seeds this on first use so the benchmark covers every AI task out of the box. DefaultBenchmarkCases (rename only) is kept for the prompt-leak guards and legacy-seed detection.

type BenchmarkStore 

type BenchmarkStore struct {
	// contains filtered or unexported fields
}

BenchmarkStore persists benchmark results, the (user-editable) case set, and the resulting chain order so a re-ranking survives restarts and the chain boots in its best-known order without re-running the benchmark.

func NewBenchmarkStore 

func NewBenchmarkStore(pool *sql.DB) (*BenchmarkStore, error)

NewBenchmarkStore wires the benchmark store onto the shared Postgres pool. Schema is applied centrally (internal/db migrations).

func (*BenchmarkStore) Cases 

func (s *BenchmarkStore) Cases() []BenchmarkCase

Cases returns the user-editable case set, seeding the defaults on first use. The seed is the FULL multi-task set (rename + schedule + identify), so a fresh install benchmarks every AI task out of the box.

func (*BenchmarkStore) Close 

func (s *BenchmarkStore) Close() error

Close is a no-op: the shared pool's lifecycle is owned by main.

func (*BenchmarkStore) LoadCostConfig 

func (s *BenchmarkStore) LoadCostConfig() (CostConfig, bool)

LoadCostConfig returns the persisted cost knobs and ok=true when any were saved (so boot can apply a UI override; otherwise the env/yaml config stands).

func (*BenchmarkStore) Order 

func (s *BenchmarkStore) Order() []string

Order returns the persisted chain order (slot ids, best-first), or nil if no benchmark has run — the chain then keeps its config order.

func (*BenchmarkStore) RecordOne 

func (s *BenchmarkStore) RecordOne(sc SlotScore) error

RecordOne is RecordRun for a single freshly-measured score, in its own transaction — used to update the durable history incrementally, one slot at a time, as a benchmark run progresses (see UpsertResult).

func (*BenchmarkStore) RecordRun 

func (s *BenchmarkStore) RecordRun(fresh []SlotScore) error

RecordRun updates the durable per-slot history from a set of FRESHLY-measured scores (the ones RunSlots/RerunIncomplete just produced — never carried-over rows, or the failure streak would bump for a run that never happened). It runs independently of SaveResults (different table) so the timeline survives the DELETE+INSERT there. Transitions:

  • ok → last_success_at = now, streak cleared
  • incomplete → last_success_at preserved, streak cleared (rate-limited, not a hard error)
  • error → last_success_at preserved, streak extended (first_failure_at kept; +1)

func (*BenchmarkStore) Results 

func (s *BenchmarkStore) Results() []SlotScore

Results returns the last benchmark, ordered best-first by chain_order, with each row's durable run history (outcome, success/run timestamps, failure streak) joined in. Legacy rows with no history row leave those fields empty/zero.

func (*BenchmarkStore) SaveCostConfig 

func (s *BenchmarkStore) SaveCostConfig(cc CostConfig) error

SaveCostConfig persists the runtime cost knobs (set from the Settings UI) so they survive a restart and override the env/yaml defaults on boot.

func (*BenchmarkStore) SaveResults 

func (s *BenchmarkStore) SaveResults(scores []SlotScore) error

SaveResults replaces the stored results with a fresh run. The slice is assumed sorted best-first (as Run returns it), so the index becomes chain_order.

func (*BenchmarkStore) SetCases 

func (s *BenchmarkStore) SetCases(cases []BenchmarkCase) error

SetCases replaces the entire case set (the UI sends the full edited list).

func (*BenchmarkStore) UpsertResult 

func (s *BenchmarkStore) UpsertResult(sc SlotScore) error

UpsertResult writes/updates a single slot's result row immediately, without touching the rest of the table — unlike SaveResults (delete-all + reinsert, meant for the FINAL, fully-ordered set). Used to persist progress AS EACH slot finishes during a run, so a benchmark that stops partway (timeout, restart) still has whatever was measured so far, instead of an all-or-nothing write at the end. chain_order is left at its current value (or 0 for a new row) — the closing SaveResults call re-derives the real best-first order once the whole run completes.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

func New 

func New(cfg config.AIConfig) *Client

New builds a Client from config. Returns nil when AI is disabled or no usable chain slot resolves — callers treat nil as "no AI, use the regex fallback".

func (*Client) AdoptBenchmark 

func (c *Client) AdoptBenchmark(scores []SlotScore)

func (*Client) AffordableSlots 

func (c *Client) AffordableSlots(slots []Slot) []Slot

FreeOnly drops paid models (a metered provider without a free marker) so the benchmark never spends credits on them — this also prunes paid leftovers a pre-filter run had adopted into the chain (e.g. a Zen "big-pickle"). AffordableSlots drops models the benchmark isn't allowed to PAY to test: those whose cost exceeds maxCostPer1M, and those with UNKNOWN cost (-1 — a metered provider with no pricing, never call it). With the default ceiling of 0 this is exactly "free only" (so we never spend); raising the ceiling lets cheap paid models in. This also prunes paid leftovers a pre-filter run adopted (e.g. Zen "big-pickle", which is unknown-cost).

func (*Client) ApplyChain 

func (c *Client) ApplyChain(defs []config.AIChainSlot)

ApplyChain replaces the live chain with the given (provider, model) defs in order — used to adopt a benchmark ranking as the working chain (best first, free local models retained as low-ranked fallbacks). Unresolvable defs are skipped; an empty result leaves the chain unchanged.

func (*Client) ApplyOrder 

func (c *Client) ApplyOrder(order []string)

ApplyOrder re-sorts the live chain to the given slot-id order (best first). Unknown ids are ignored; slots not named keep their relative order at the end.

func (*Client) CostConfig 

func (c *Client) CostConfig() CostConfig

CostConfig returns the live cost config (never nil after New).

func (*Client) DiscoverModels 

func (c *Client) DiscoverModels(ctx context.Context) []Slot

DiscoverModels lists candidate models across ALL providers so the benchmark can test every available model, not just one per provider from the chain:

  • ollama: every locally-installed model (/api/tags)
  • free tier: the whole /v1/models catalog (Groq) — nothing is billed
  • metered: ONLY models we can prove are free (a :free/-free id, or an explicit 0 price). Paid frontier models on OpenRouter / OpenCode Zen are ignored so the benchmark can't burn credits. (Zen returns no pricing at all, so the suffix is the only signal there.)

Skips models already in the chain. Caps at 100 per provider so the benchmark doesn't take hours on OpenRouter's 300+ model catalog.

func (*Client) DiscoverModelsForProvider 

func (c *Client) DiscoverModelsForProvider(ctx context.Context, provider string) []Slot

DiscoverModelsForProvider lists candidate models for a specific provider.

func (*Client) DiscoverOllamaModels 

func (c *Client) DiscoverOllamaModels(ctx context.Context) []Slot

DiscoverOllamaModels queries the local Ollama (/api/tags) for models it serves and returns a Slot per model that isn't already in the chain — so the benchmark can test EVERY available model, not just the one wired into the chain. This includes Ollama CLOUD models (the "-cloud" suffix): recent Ollama registers them locally and lists them in /api/tags, so they're discovered here too (ollamaCanComplete keeps them — /api/show for a remote model doesn't report capabilities, and on doubt we keep). They resolve as Free (the ollama provider is free-tier) but non-Local (see localModel), so the benchmark parallelizes them instead of serializing on the single GPU.

func (*Client) ExtractRenameMetadata 

func (c *Client) ExtractRenameMetadata(ctx context.Context, rawName string) (*RenameMetadata, string, error)

func (*Client) ExtractRenameMetadataWithContext 

func (c *Client) ExtractRenameMetadataWithContext(ctx context.Context, rawName, taxonomyHint string) (*RenameMetadata, string, error)

ExtractRenameMetadataWithContext is ExtractRenameMetadata plus an optional taxonomy hint. The USER message stays the bare rawName (so title extraction is unaffected and the benchmark/parser behaviour is identical); the hint is appended to the SYSTEM prompt as an extra instruction telling the model which destination category folders already exist and to prefer reusing one. An empty hint is the exact legacy call.

func (*Client) IdentifyTitle 

func (c *Client) IdentifyTitle(ctx context.Context, rawName string) (*TitleResult, string, error)

IdentifyTitle walks the chain until a slot returns a usable title. Returns the result and the slot id that produced it. A nil result with nil error means no slot could parse a title (caller falls back to regex cleaning).

func (*Client) IdentifyWithSlot 

func (c *Client) IdentifyWithSlot(ctx context.Context, slotID, rawName string) (*TitleResult, time.Duration, error)

IdentifyWithSlot runs a single named slot, bypassing the breaker. Used by the benchmark to measure each model independently.

func (*Client) MusicQuery 

func (c *Client) MusicQuery(ctx context.Context, rawName string) string

MusicQuery asks the chain to build a cover-art search query from a messy music release name (e.g. "Disturbed - Discography 2000-2019 [FLAC]" → "Disturbed"). Walks the chain like IdentifyTitle; returns "" if nothing usable came back.

func (*Client) ParseSchedule 

func (c *Client) ParseSchedule(ctx context.Context, text string) (*ScheduleResult, error)

ParseSchedule walks the chain until a slot converts the free-text phrase into a schedule. ErrInvalidSchedule (checked with errors.Is) covers both "the model says this isn't a schedule" and "no model produced parseable JSON"; any other error means the chain itself failed (network/rate limit — AI unavailable).

func (*Client) Providers 

func (c *Client) Providers() []string

Providers returns the list of configured provider names (e.g. "ollama", "groq").

func (*Client) RerunIncomplete 

func (c *Client) RerunIncomplete(ctx context.Context, prev []SlotScore, cases []BenchmarkCase) (merged, fresh []SlotScore)

RerunIncomplete returns (merged, fresh): merged is prev with the re-run scores folded in (sorted best-first); fresh is ONLY the slots actually re-measured this call. The caller records history for `fresh` alone — recording the carried-over slots would spuriously bump their failure streak for a run that never happened.

func (*Client) Run 

func (c *Client) Run(ctx context.Context, cases []BenchmarkCase) []SlotScore

Run benchmarks the configured chain. See RunSlots.

func (*Client) RunSlots 

func (c *Client) RunSlots(ctx context.Context, slots []Slot, cases []BenchmarkCase) []SlotScore

RunSlots benchmarks the given slots against the case set and returns scores sorted by composite (best first). Each slot is called directly (bypassing the breaker) so a parked model still gets measured. Used with the configured chain AND with discovered local Ollama models.

func (*Client) RunSlotsProgress 

func (c *Client) RunSlotsProgress(ctx context.Context, slots []Slot, cases []BenchmarkCase, onResult func(SlotScore)) []SlotScore

func (*Client) SetCostConfig 

func (c *Client) SetCostConfig(cc CostConfig)

SetCostConfig swaps in new cost knobs live (watts falls back to a default).

func (*Client) Slots 

func (c *Client) Slots() []Slot

Slots returns a copy of the resolved chain (for the benchmark + status UI).

type CostConfig 

type CostConfig struct {
	MaxCostPer1M float64 `json:"maxCostPer1M"` // ceiling for testing paid models ($/1M); 0 = free only
	KWhPrice     float64 `json:"kwhPrice"`     // electricity tariff ($/kWh); 0 = local stays free
	LocalWatts   float64 `json:"localWatts"`   // GPU power draw under load (W)
}

CostConfig holds the knobs that drive the value-based score: the benchmark cost ceiling, the electricity tariff, and the GPU power draw used to price local models' energy. See SetCostConfig / CostConfig().

type RenameMetadata 

type RenameMetadata struct {
	Title        string `json:"title"`
	Year         int    `json:"year"`
	Kind         string `json:"kind"` // "movie" | "tv"
	Season       int    `json:"season"`
	Episode      int    `json:"episode"`
	EpisodeTitle string `json:"episode_title"`
}

type ScheduleResult 

type ScheduleResult struct {
	Kind    string `json:"kind"` // "interval" | "daily" | "weekly"
	Minutes int    `json:"minutes"`
	Weekday int    `json:"weekday"` // 0=Sunday … 6=Saturday
	Hour    int    `json:"hour"`
	Minute  int    `json:"minute"`
}

ScheduleResult is the model's strict-JSON answer for a scheduling phrase. Mirrors watchlist.Schedule but lives here so the ai package stays decoupled.

type Slot 

type Slot struct {
	ID       string
	Provider string
	Model    string
	BaseURL  string

	Free  bool // true when the model is free (CostPer1M == 0)
	Local bool // true for a model served by the LOCAL Ollama GPU (see localModel)
	// CostPer1M is the blended (prompt+completion)/2 price in USD per 1M tokens.
	// 0 = free; -1 = UNKNOWN (a metered provider that doesn't expose pricing, e.g.
	// OpenCode Zen) — those are excluded from the benchmark so we never call a
	// model we can't price. Discovery fills it from /models; resolveSlot can only
	// tell free (0) from unknown (-1) for chain models (no pricing data there).
	CostPer1M float64
	// contains filtered or unexported fields
}

Slot is one resolved chain entry — a provider's base URL + key bound to a specific model id.

type SlotScore 

type SlotScore struct {
	SlotID        string  `json:"slotId"`
	Provider      string  `json:"provider"`
	Model         string  `json:"model"`
	Accuracy      float64 `json:"accuracy"`     // 0..1 — mean of the per-task accuracies (so every task weighs equally)
	AvgLatencyMs  int64   `json:"avgLatencyMs"` // MEDIAN wall-clock per call (resilient to model-load residual)
	Composite     float64 `json:"composite"`    // accuracy / sqrt(latencySeconds) / (1+cost)
	Samples       int     `json:"samples"`      // cases that produced a usable reply
	Free          bool    `json:"free"`         // true when CostPer1M == 0
	CostPer1M     float64 `json:"costPer1M"`    // blended USD per 1M tokens (0 = free); drives the composite
	FailureReason string  `json:"failureReason,omitempty"`
	// Tasks is the per-task accuracy breakdown (keyed by task id: "rename",
	// "identify", "schedule"). Optional in the JSON — older persisted rows and the
	// single-task default leave it nil and the UI falls back to the global Accuracy.
	Tasks map[string]TaskScore `json:"tasks,omitempty"`
	// Incomplete is true when some cases were transiently SKIPPED (rate limit
	// after retries, network) so the model wasn't measured on the full set. These
	// are the ones the "Rodar faltantes" button re-runs later, outside the
	// rate-limit window. A model fully tested (even if some cases failed hard) is
	// NOT incomplete.
	Incomplete bool `json:"incomplete,omitempty"`
	// Completeness is the fraction of cases (0..1) that produced a scored result. 1.0 for a
	// full run; lower when cases were skipped (rate limit). RankBefore uses it to demote only
	// SPARSELY-measured runs, not a run that covered most cases at high accuracy. 0 on legacy
	// rows persisted before this field existed — RankBefore falls back to Incomplete there.
	Completeness float64 `json:"completeness,omitempty"`
	// History fields are OUTPUT-ONLY: they're populated by BenchmarkStore.Results
	// from the durable benchmark_history table (NOT by a live RunSlots measurement,
	// which has no past to look at). They answer "did this run succeed or error,
	// did the error persist, and when did it last succeed" without re-running.
	// Empty/zero on a fresh measurement and on legacy rows with no recorded history.
	LastOutcome         string `json:"lastOutcome,omitempty"`         // "ok" | "incomplete" | "error" of the last actual run
	LastError           string `json:"lastError,omitempty"`           // failure reason of the last failing run; "" once it succeeds again. Durable (survives the SaveResults re-baseline that wipes FailureReason)
	LastSuccessAt       string `json:"lastSuccessAt,omitempty"`       // RFC3339 of the last "ok" run; "" = never succeeded
	LastRunAt           string `json:"lastRunAt,omitempty"`           // RFC3339 of the last run (any outcome)
	FirstFailureAt      string `json:"firstFailureAt,omitempty"`      // RFC3339 the current error streak began; "" = not failing
	ConsecutiveFailures int    `json:"consecutiveFailures,omitempty"` // # of consecutive "error" runs (resets on a usable run)
}

SlotScore is one model's aggregate result over the whole case set.

type TaskScore 

type TaskScore struct {
	Accuracy float64 `json:"accuracy"` // 0..1 mean over this task's cases
	Samples  int     `json:"samples"`  // this task's cases that produced a usable reply
	Scored   int     `json:"scored"`   // usable + bad-output (the accuracy denominator)
}

TaskScore is one model's accuracy on ONE task within a benchmark run. It's the per-task breakdown behind the global Accuracy/Composite — the UI can show it as extra columns, and the composite averages across tasks so the chain is ranked on ALL its jobs, not just rename.

type TitleResult 

type TitleResult struct {
	Title string `json:"title"`
	Year  int    `json:"year"`
	Kind  string `json:"kind"` // "movie" | "tv" | "unknown"
}

TitleResult is what IdentifyTitle extracts from a raw torrent name.

func (*TitleResult) Query 

func (r *TitleResult) Query() string

Query returns the title (plus year when known) formatted for a TMDB search.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.