Documentation
¶
Overview ¶
Package tests contains shared test utilities for streaming Wiktionary data dumps and performing JSON round-trip validation against language-specific schema types.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DoUseMirror ¶
func DoUseMirror() bool
DoUseMirror returns true when the USE_WIKTIONARY_DATA_MIRROR environment variable is set to a truthy value ("1", "yes", "on", "true" – case insensitive). Language-specific test files can use this to swap their data URLs for a mirror.
func RunStreamTest ¶
func RunStreamTest(t *testing.T, r io.Reader, processEntry ProcessEntryFunc)
RunStreamTest reads newline-delimited JSON records from r and dispatches each record to a pool of worker goroutines (sized to GOMAXPROCS). Every worker calls processEntry for each received line.
The streaming reader is consumed sequentially on the calling goroutine, while the CPU-intensive JSON work runs concurrently. If any worker reports an error via processEntry, the context is cancelled, the reader loop stops early, and the test fails after all workers have drained.
func StreamWiktionaryData ¶
StreamWiktionaryData downloads a gzip-compressed JSONL file from url, returning a Reader for the decompressed stream. The caller must invoke the returned cleanup function to release the HTTP connection and the gzip reader.
Types ¶
type ProcessEntryFunc ¶
ProcessEntryFunc is the signature for a per-line processing function. It receives a single JSON line (without the trailing newline) and should validate it – typically by unmarshalling into a schema type, marshalling back, and optionally unmarshalling again to catch symmetry problems.
The function MUST be safe to call concurrently from multiple goroutines; it may call t.Errorf / t.Logf (both are concurrency-safe) but MUST NOT call t.Fatalf / t.FailNow or any other function that is not goroutine-safe.
Source Files