store

package
v0.0.0-...-40fc603 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 8, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Hand-written tables for the Berkeley class schedule novel commands. These live outside the generator's migrate path on purpose: the generator regenerates store.go from a spec but never touches files with this naming convention.

Package store provides local SQLite persistence for berkeley-classes. Uses modernc.org/sqlite (pure Go, no CGO) for zero-dependency cross-compilation. FTS5 full-text search indexes are created for searchable content.

Index

Constants

View Source 
const StoreSchemaVersion = 3

StoreSchemaVersion is the on-disk schema version this binary understands. It is stamped into SQLite's PRAGMA user_version on fresh databases and checked on every open. Non-learn CLIs advance to v3 for the resources_fts rowid rehash.

Variables

This section is empty.

Functions

func DecodeJSONObject 

func DecodeJSONObject(data json.RawMessage) (map[string]any, error)

DecodeJSONObject decodes data into an object while preserving JSON numbers. Plain json.Unmarshal turns numbers into float64, and fmt on those values can render large integer IDs as scientific notation before they reach resources.id.

func EnsureBerkeleyTables 

func EnsureBerkeleyTables(ctx context.Context, db *sql.DB) error

EnsureBerkeleyTables runs all berkeley table migrations against db. Safe to call from any RunE prologue; the IF NOT EXISTS clauses make repeated calls a no-op.

func ExtractResourceID 

func ExtractResourceID(resourceType string, obj map[string]any) string

ExtractResourceID resolves the primary key UpsertBatch would use for a resource item. Callers that need to gate best-effort writes can use this to avoid passing non-entity envelopes into the batch path.

func FindSubjectIDByName 

func FindSubjectIDByName(ctx context.Context, db *sql.DB, q string) (string, error)

FindSubjectIDByName best-effort matches a subject name (case-insensitive substring) and returns its id, or "" if not found.

func FindTermIDByName 

func FindTermIDByName(ctx context.Context, db *sql.DB, q string) (string, error)

FindTermIDByName best-effort matches a term name and returns its id.

func IsUUID 

func IsUUID(s string) bool

IsUUID returns true if the input looks like a UUID.

func LookupFieldValue 

func LookupFieldValue(obj map[string]any, snakeKey string) any

LookupFieldValue resolves a field value from a JSON object map, trying the snake_case key first, then the camelCase rendering, then the PascalCase rendering. Exported so the sync command's extractID and the upsert path resolve fields the same way — a divergence here produces silent drops on heterogeneous payloads. The PascalCase pass handles .NET-shaped responses (`Id`, `Name`, `OrderId`) without forcing each spec to declare casing.

func ResourceIDString 

func ResourceIDString(v any) string

ResourceIDString returns the stable text form used for resources.id.

func SnapshotEnrollment 

func SnapshotEnrollment(ctx context.Context, db *sql.DB, ccn int, openSeats, enrolled, waitlisted, capacity int) error

SnapshotEnrollment writes one snapshot row to bc_section_snapshots.

func UpsertSection 

func UpsertSection(ctx context.Context, db *sql.DB, s BCSection) error

UpsertSection inserts or replaces one row in bc_sections. Returns no error if ccn==0; the caller should pre-validate.

func UpsertSubject 

func UpsertSubject(ctx context.Context, db *sql.DB, id, name, code string) error

UpsertSubject caches one subject in bc_subjects.

func UpsertTerm 

func UpsertTerm(ctx context.Context, db *sql.DB, id, name, kind string) error

UpsertTerm caches one term in bc_terms.

Types

type BCSection 

type BCSection struct {
	CCN             int    `json:"ccn"`
	TermID          string `json:"term_id,omitempty"`
	SubjectName     string `json:"subject_name,omitempty"`
	CourseCode      string `json:"course_code,omitempty"`
	CourseNumber    string `json:"course_number,omitempty"`
	SectionNumber   string `json:"section_number,omitempty"`
	SectionType     string `json:"section_type,omitempty"`
	Title           string `json:"title,omitempty"`
	Instructors     string `json:"instructors,omitempty"`
	Units           string `json:"units,omitempty"`
	InstructionMode string `json:"instruction_mode,omitempty"`
	MeetingDates    string `json:"meeting_dates,omitempty"`
	MeetingDays     string `json:"meeting_days,omitempty"`
	MeetingTime     string `json:"meeting_time,omitempty"`
	Location        string `json:"location,omitempty"`
	Slug            string `json:"slug,omitempty"`
	Description     string `json:"description,omitempty"`
	OpenSeats       int    `json:"open_seats"`
	Enrolled        int    `json:"enrolled"`
	Waitlisted      int    `json:"waitlisted"`
	Capacity        int    `json:"capacity"`
	LastSynced      int64  `json:"last_synced,omitempty"`
}

BCSection is the row shape returned by QuerySections and friends.

func LookupSectionByCCN 

func LookupSectionByCCN(ctx context.Context, db *sql.DB, ccn int) (*BCSection, error)

LookupSectionByCCN returns one section by primary key, or nil row not found.

func QuerySections 

func QuerySections(ctx context.Context, db *sql.DB, opts QuerySectionsOpts) ([]BCSection, error)

QuerySections returns sections matching the optional filters. Empty filters match all rows. Pass courseCode like "COMPSCI 61A" — wildcard is applied automatically.

type BCSubjectRow 

type BCSubjectRow struct {
	ID   string `json:"id"`
	Name string `json:"name"`
	Code string `json:"code,omitempty"`
}

BCSubjectRow is one bc_subjects row.

func ListBCSubjects 

func ListBCSubjects(ctx context.Context, db *sql.DB) ([]BCSubjectRow, error)

ListBCSubjects returns every cached subject row.

type BCTermRow 

type BCTermRow struct {
	ID   string `json:"id"`
	Name string `json:"name"`
	Kind string `json:"kind,omitempty"`
}

BCTermRow is one bc_terms row.

func ListBCTerms 

func ListBCTerms(ctx context.Context, db *sql.DB) ([]BCTermRow, error)

ListBCTerms returns every cached term row.

type QuerySectionsOpts 

type QuerySectionsOpts struct {
	CourseCode      string
	CourseCodeLike  string
	TermID          string
	InstructorLike  string
	SubjectNameLike string
	CCN             int
	OpenOnly        bool
	Limit           int
}

QuerySectionsOpts is a small struct of filters; embed it directly.

type SnapshotRow 

type SnapshotRow struct {
	CCN        int   `json:"ccn"`
	TakenAt    int64 `json:"taken_at"`
	OpenSeats  int   `json:"open_seats"`
	Enrolled   int   `json:"enrolled"`
	Waitlisted int   `json:"waitlisted"`
	Capacity   int   `json:"capacity"`
}

LatestSnapshotsSince returns the most recent snapshot per CCN whose taken_at is at-or-after sinceUnix.

func SnapshotsSince 

func SnapshotsSince(ctx context.Context, db *sql.DB, sinceUnix int64) ([]SnapshotRow, error)

SnapshotsSince returns every snapshot row with taken_at >= sinceUnix, ordered by ccn,taken_at ascending. Callers that only need the earliest-vs-latest pair per ccn should do that grouping in Go.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

func Open 

func Open(dbPath string) (*Store, error)

Open opens or creates the SQLite store at dbPath using the background context. Prefer OpenWithContext from a Cobra command so SIGINT during a slow migration interrupts the open instead of stranding the caller.

func OpenReadOnly 

func OpenReadOnly(dbPath string) (*Store, error)

OpenReadOnly opens an existing SQLite store at dbPath in read-only mode. mode=ro rejects direct and CTE-wrapped writes (INSERT, UPDATE, DELETE, REPLACE, "WITH x AS (...) INSERT ...") at the driver level. Skips MkdirAll and migrate; the file is expected to exist.

The file: URI prefix is load-bearing: modernc.org/sqlite only honors SQLite's URI query parameters (mode, cache, etc.) when the DSN starts with "file:". Without the prefix, "?mode=ro" is silently dropped and the connection opens read-write. Pragmas use the driver's _pragma= name(value) syntax — modernc.org/sqlite does NOT recognize the mattn/go-sqlite3 _journal_mode=WAL / _busy_timeout=5000 form and drops those keys silently, so the busy_timeout below is what keeps a read concurrent with a writer from failing immediately with SQLITE_BUSY.

Deliberately no journal_mode pragma here: journal mode is a property of the database file, set by the read-write open, not the connection. Issuing PRAGMA journal_mode=WAL on a read-only handle to a DB still in the default delete mode (e.g. a pre-WAL database opened by an old binary before its first read-write open) errors with "attempt to write a readonly database".

func OpenWithContext 

func OpenWithContext(ctx context.Context, dbPath string) (*Store, error)

OpenWithContext opens or creates the SQLite store at dbPath. The context is honored by the migration path: cancellation interrupts the retry-on-SQLITE_BUSY loop and propagates ctx.Err() back to the caller instead of waiting out the full migrationLockTimeout.

func (*Store) ClearSyncCursors 

func (s *Store) ClearSyncCursors() error

ClearSyncCursors resets all sync state for a full resync.

func (*Store) Close 

func (s *Store) Close() error

func (*Store) Count 

func (s *Store) Count(resourceType string) (int, error)

func (*Store) DB 

func (s *Store) DB() *sql.DB

DB exposes the underlying *sql.DB for callers that need to run ad-hoc queries (e.g., doctor's cache inspection, share snapshot import). Callers must not call Close on the returned handle.

func (*Store) Get 

func (s *Store) Get(resourceType, id string) (json.RawMessage, error)

Propagates sql.ErrNoRows on a miss so callers can distinguish absence from other scan errors via errors.Is.

func (*Store) GetLastSyncedAt 

func (s *Store) GetLastSyncedAt(resourceType string) string

GetLastSyncedAt returns the last sync timestamp for a resource type.

func (*Store) GetSyncCursor 

func (s *Store) GetSyncCursor(resourceType string) string

GetSyncCursor returns the last pagination cursor for a resource type.

func (*Store) GetSyncState 

func (s *Store) GetSyncState(resourceType string) (cursor string, lastSynced time.Time, count int, err error)

func (*Store) List 

func (s *Store) List(resourceType string, limit int) ([]json.RawMessage, error)

func (*Store) ListField 

func (s *Store) ListField(resourceType, field string) ([]string, error)

ListField returns values of a named field from a resource's domain table, or from the generic resources table via json_extract when no typed column exists. Used by dependent sync to iterate parents when a spec-declared walker extracts a non-PK field (Endpoint.Walker.KeyField in the upstream repo) for the child path's placeholder.

Defense in depth: field is validated against validIdentifierRE at entry — the regex pins it to SQL-safe identifier shape covering both the typed-column primary path AND the json_extract fallback (where pragma_table_info validation would never run if the parent's domain table doesn't exist yet). resourceType is never interpolated into SQL directly; we resolve it to a real table name via a parameterized sqlite_master lookup. Only validated names are substituted (double-quoted) into the SELECT. Mirrors ListIDs's defense pattern so callers may pass any string.

func (*Store) ListFieldSets 

func (s *Store) ListFieldSets(resourceType string, fields []string) ([]map[string]string, error)

ListFieldSets returns row-correlated values from the generic resources table. Dependent sync uses this for multi-placeholder paths where values such as owner/repo or server/webapp must stay paired per parent row.

func (*Store) ListIDs 

func (s *Store) ListIDs(resourceType string) ([]string, error)

ListIDs returns all IDs from a resource's domain table, or from the generic resources table if no domain table exists. Used by dependent sync to iterate parents.

resourceType is never interpolated into SQL directly. We resolve it to a real table name via a parameterized sqlite_master lookup; only that trusted name is substituted (double-quoted) into the SELECT. Callers may pass any string.

func (*Store) Path 

func (s *Store) Path() string

Path returns the on-disk path of the backing SQLite file.

func (*Store) Query 

func (s *Store) Query(query string, args ...any) (*sql.Rows, error)

Query executes a raw SQL query and returns the rows. Used by workflow commands that need custom queries against the local store.

func (*Store) ResolveByName 

func (s *Store) ResolveByName(resourceType string, input string, matchFields ...string) (string, error)

ResolveByName resolves a human-readable name to a UUID from synced data. If the input is already a UUID, it is returned as-is. matchFields are JSON field names to search against (e.g., "name", "key", "email").

json_extract path components cannot be bound as SQL parameters, so each field is validated against validIdentifierRE before being spliced into the query.

func (*Store) SaveSyncCursor 

func (s *Store) SaveSyncCursor(resourceType, cursor string) error

SaveSyncCursor stores the pagination cursor for a resource type.

func (*Store) SaveSyncState 

func (s *Store) SaveSyncState(resourceType, cursor string, count int) error

func (*Store) SchemaVersion 

func (s *Store) SchemaVersion() (int, error)

SchemaVersion reads PRAGMA user_version, which is stamped by migrate(). A zero value means the database predates the schema-version gate — not a bug, but the caller may want to warn.

func (*Store) Search 

func (s *Store) Search(query string, limit int) ([]json.RawMessage, error)

func (*Store) SearchSections 

func (s *Store) SearchSections(query string, limit int) ([]json.RawMessage, error)

SearchSections searches the sections_fts index with optional filters.

func (*Store) Status 

func (s *Store) Status() (map[string]int, error)

func (*Store) Upsert 

func (s *Store) Upsert(resourceType, id string, data json.RawMessage) error

func (*Store) UpsertBatch 

func (s *Store) UpsertBatch(resourceType string, items []json.RawMessage) (int, int, error)

UpsertBatch inserts or replaces multiple records in a single transaction and returns (stored, extractFailures, err). stored counts rows landed in the generic resources table; extractFailures counts items that survived JSON unmarshal but had no extractable primary key (templated IDField AND generic fallback both missed). callers (sync.go.tmpl) compare these against len(items) to emit the per-item primary_key_unresolved warning and the F4b stored_count_zero_after_extraction probe.

For resource types that have a domain-specific typed table, the per-item generic insert is followed by a dispatch to the matching upsert<Pascal>Tx inside the same transaction. Without that dispatch, paginated syncs would only populate the generic resources table — typed tables (and indexed columns like parent_id added by dependent-resource sync) would stay empty.

Each typed-table dispatch runs inside a per-item SAVEPOINT so a constraint failure in the typed insert (e.g. NOT NULL parent FK when the generator didn't populate the parent path placeholder) rolls back only that typed upsert. The generic resources row inserted just above it survives the rollback, so successful API fetches never strand in memory because one downstream typed table is misconfigured. Failures are surfaced via a trailing stderr warning rather than aborting the batch.

func (*Store) UpsertFacets 

func (s *Store) UpsertFacets(data json.RawMessage) error

UpsertFacets inserts or updates a facets record with domain-specific columns.

func (*Store) UpsertSections 

func (s *Store) UpsertSections(data json.RawMessage) error

UpsertSections inserts or updates a sections record with domain-specific columns.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.