Documentation
¶
Overview ¶
Package javayamsql parses Apple's `.yamsql` acceptance-test format — the multi-document YAML dialect driving the Java record layer's yaml-tests suite.
The vendored corpus lives at third_party/apple/fdb-record-layer/ (see the README there). This package only *parses*; nothing here executes a query.
The parser is deliberately STRICTER than Java's. Java probes its option maps with containsKey and silently ignores any key it does not recognise, so a typo'd or newly-added upstream directive reads as "absent" and the test quietly checks less than it claims to. Here every block key, command, config key and YAML tag must be in the known set or parsing fails. That is the whole point of the corpus gate: when a version bump introduces a directive, it must fail loudly and by name rather than being dropped on the floor.
Index ¶
- Constants
- Variables
- func Composition() map[Polarity]int
- func IsKnownTag(tag string) bool
- func IsRowTag(tag string) bool
- func IsSegmentTag(tag string) bool
- func KnownTags() []string
- func ParseLevelNegatives() []string
- func SelectedAtCurrentVersion(atLeast, lessThan *Version) bool
- func SupportedAtCurrentVersion(min *Version) bool
- type Block
- type BlockKind
- type Cell
- type Census
- type Command
- type CommandKind
- type Config
- type ConfigKind
- type CopyBlock
- type CopyEndpoint
- type Corpus
- type Entry
- type File
- type IncludeBlock
- type InertDirective
- type ManifestEntry
- type NamedSetup
- type OptionsBlock
- type ParseError
- type Polarity
- type Row
- type SchemaTemplateBlock
- type SchemaTemplateVariant
- type Segment
- type SegmentKind
- type SetupBlock
- type Test
- type TestBlock
- type TestBlockOptions
- type TransactionSetupsBlock
- type Value
- type ValueKind
- type Version
Constants ¶
const ( TagLong = "!l" TagFloat = "!f" TagBytes = "!b" TagIgnore = "!ignore" TagNull = "!null" TagNotNull = "!not_null" TagStringContains = "!sc" TagUUID = "!uuid" TagPos = "!pos" TagRandomStr = "!randomStr" TagVector16 = "!v16" TagVector32 = "!v32" TagVector64 = "!v64" TagCurrentVersion = "!current_version" TagRandom = "!r" TagArray = "!a" TagInList = "!in" TagNullArg = "!n" )
The custom YAML tags the corpus uses.
Two registries exist upstream and they are NOT the same set:
- Row/config tags, registered through the CustomTag ServiceLoader and installed by CustomYamlConstructor. These appear in `result:` rows, `supported_version:` values and so on.
- Parameter-injection tags, registered by QueryInterpreter's private QueryParameterYamlConstructor. These appear only inside a `!!…!!` segment of a query string.
The two overlap but neither contains the other: `!not_null`/`!null`/`!ignore`/ `!pos`/`!sc`/`!current_version` are row-only, while `!r`/`!a`/`!in`/`!n` are segment-only. Mixing them up is silently wrong in Java (an unknown tag inside a segment falls through to a plain scalar), so the two sets are kept apart here and checked against the context they appear in.
const CorpusSubdir = "third_party/apple/fdb-record-layer/yaml-tests/src/test/resources"
CorpusSubdir is the vendored corpus's location relative to the repo root.
Variables ¶
var KnownInertDirectives = []struct { Path string Where string Key string Line int }{ {"between.yamsql", "test_block", "supported_version", 97}, {"index-ddl-aggregates-only.yamsql", "options", "noChecks", 27}, {"uuid-non-prepared.yamsql", "test_block", "supported_version", 62}, {"uuid-prepared.yamsql", "test_block", "supported_version", 64}, }
KnownInertDirectives are the corpus's dead keys: written by an author who plainly meant them to do something, and read by nothing.
These are upstream defects, not parser gaps. `supported_version` belongs under a test_block's `options:` map; written directly under `test_block:` it is never looked at, so all three files below run unconditionally against every version instead of being gated. `noChecks` is a query-config name that has no meaning in a file preamble.
They are pinned so the set cannot grow silently. A new entry appearing here means someone added another directive that does nothing.
var Manifest = buildManifest()
Manifest maps corpus-root-relative paths to their polarity. Files absent from it are Positive — the overwhelming majority, and listing 200-odd of them would bury the 60 that carry information.
Every entry below was read off the Java test class named in Source, not inferred from the directory name. That distinction matters: the `include-block/includes/` directory looks like a bag of fragments and is not one — IncludeBlockTest runs all ten standalone, expecting eight to fail.
Functions ¶
func Composition ¶
Composition counts the manifest's entries by polarity.
It exists because the manifest's totals get quoted in prose, and prose denominators drift: "42 execution-level negatives" (an entry count) and "20 booked as polarity:negative-execution" (a ledger outcome) are different numbers about different things, and fusing them is how a corrected figure becomes a wrong one. Anything quoting a manifest total should read it from here.
Positive entries are the ones recorded EXPLICITLY — files carrying a reason worth keeping even though they pass. The overwhelming majority of positives are absent from the manifest entirely and are not counted here.
func IsKnownTag ¶
IsKnownTag reports whether tag is a custom tag this format defines anywhere.
func IsSegmentTag ¶
IsSegmentTag reports whether tag is legal inside a `!!…!!` segment.
func KnownTags ¶
func KnownTags() []string
KnownTags returns every custom tag, for census and drift reporting.
func ParseLevelNegatives ¶
func ParseLevelNegatives() []string
ParseLevelNegatives returns the files the parser is required to reject.
func SelectedAtCurrentVersion ¶
SelectedAtCurrentVersion reports whether the half-open range [atLeast, lessThan) contains the version under test, with a nil bound widening to MIN / MAX exactly as SemanticVersionRanges.atLeast/lessThan do.
"The version under test" is the `!current_version` singleton, because that is what a single-version runner reports: EmbeddedYamlConnectionFactory's getVersionsUnderTest is {SemanticVersion.current()}, and CURRENT sorts above every literal version. Two consequences fall out of that ordering and are the whole of how a Go run resolves the corpus's version machinery: an `initialVersionAtLeast: <literal>` branch is always selected, and an `initialVersionLessThan:` branch never is — which is the same thing as saying a version-variant list collapses to its current-version branch.
func SupportedAtCurrentVersion ¶
SupportedAtCurrentVersion reports whether a `supported_version` gate admits the version under test, mirroring SupportedVersionCheck.parse: the gate excludes the run when any version under test is strictly older than the declared minimum.
With a single version under test, and that version being the CURRENT singleton which sorts above every literal, the gate ADMITS unconditionally. The function exists to make that a stated, testable fact rather than an omission — if the runner ever gains a second version under test, this is the one place the answer stops being constant.
Types ¶
type Block ¶
type Block struct {
Kind BlockKind
Line int
// Inert lists keys within this block that Java's parser silently ignores.
Inert []InertDirective
Options *OptionsBlock
SchemaTemplate *SchemaTemplateBlock
Setup *SetupBlock
Test *TestBlock
Include *IncludeBlock
TransactionSetups *TransactionSetupsBlock
Copy *CopyBlock
}
Block is a single YAML document region. Exactly one of the typed payload pointers is non-nil, selected by Kind.
type BlockKind ¶
type BlockKind string
BlockKind names the seven top-level document keys Java's Block.parse dispatches on. Anything else is "Cannot recognize the type of block".
const ( BlockOptions BlockKind = "options" BlockSchemaTemplate BlockKind = "schema_template" BlockSetup BlockKind = "setup" BlockTest BlockKind = "test_block" BlockInclude BlockKind = "include" BlockTransactionSetups BlockKind = "transaction_setups" BlockCopy BlockKind = "copy_block" )
The recognised block keys.
type Cell ¶
type Cell struct {
Line int
// Key is the mapping key node, which carries the !pos tag when present.
Key *Value
// Val is the mapping value node, always non-nil; YAML null when elided.
Val *Value
// Pos is the explicit 1-based column from a `!pos n` key, else nil.
Pos *int64
}
Cell is one expected column value within a Row.
The by-name/positional split is Java's Matchers.matchMap, and it is entirely driven by whether the mapping entry's VALUE is YAML null:
{ID: 10} → by name "ID", expecting 10
{10} → positional column 1, expecting 10 (value is null, key is the expectation)
{ID: !null} → by name "ID", expecting SQL NULL (!null is a tag, not YAML null)
{!pos 2: 10} → positional column 2, expecting 10
See Value.IsNull for why the third case is not the second.
func (Cell) ColumnIndex ¶
ColumnIndex returns the 1-based column this cell matches, given its ordinal position in the row. An explicit `!pos` overrides the ordinal.
func (Cell) ColumnName ¶
ColumnName returns the column name for a by-name cell. It is meaningless for a positional cell and reports ok=false there.
func (Cell) Expected ¶
Expected returns the value this cell asserts: the mapping value, or the key itself when the value is YAML null.
func (Cell) Positional ¶
Positional reports whether this cell is matched by column number rather than by column name.
type Census ¶
type Census struct {
Files int
Blocks map[BlockKind]int
Queries int
// Commands counts every command including queries, so
// Commands[CommandQuery] == Queries.
Commands map[CommandKind]int
Configs map[ConfigKind]int
// Tags counts custom-tag occurrences anywhere in a file, including inside
// parameter-injection segments.
Tags map[string]int
// Rows and Cells count expected result rows and their cells.
Rows int
Cells int
// PositionalCells is the subset of Cells matched by column number.
PositionalCells int
// Segments counts `!!…!!` parameter injections.
Segments int
// Includes counts include blocks (not distinct targets).
Includes int
}
Census counts the directive and tag surface actually exercised by a set of parsed files.
It exists so that a version bump fails *describably*. A parse gate alone reports "still parses"; it cannot notice that upstream deleted every use of a directive, or doubled the query count. The census turns those into a diff.
type Command ¶
type Command struct {
Kind CommandKind
Line int
// Query is the raw query text, parameter-injection segments included.
Query string
// Segments are the `!!…!!` parameter injections found in Query, in order.
Segments []Segment
// Payload is the value of a load-schema-template / set-schema-state command.
Payload string
// Configs are the checks attached to a query command. Empty means Java
// substitutes an implicit noChecks config.
Configs []*Config
}
Command is one executable step: a query with its configs, or a metadata command carrying an opaque payload.
func (*Command) ImplicitNoChecks ¶
ImplicitNoChecks reports whether Java would attach a synthetic `noChecks` config to this command because it declares none of its own.
type CommandKind ¶
type CommandKind string
CommandKind names the three command keys Java's Command.parse dispatches on.
const ( CommandQuery CommandKind = "query" CommandLoadSchemaTemplate CommandKind = "load schema template" CommandSetSchemaState CommandKind = "set schema state" )
The recognised command keys. Note the two non-query ones contain a space.
type Config ¶
type Config struct {
Kind ConfigKind
Line int
// Raw is the directive's value exactly as parsed, always non-nil.
Raw *Value
// Rows is populated for result/unorderedResult when the value is a list of
// row mappings. It stays nil for `result:` with a null or `!ignore` value,
// both of which Java treats specially rather than as a row list.
Rows []Row
// Version is populated for supported_version and initialVersion*.
Version *Version
// ErrorCode is the resolved 5-character SQLSTATE for an `error:` config.
ErrorCode string
// Number is populated for count, planHash and maxRows.
Number *int64
// Text is populated for explain, explainContains, setup, setupReference
// and debugger.
Text string
}
Config is one check attached to a query.
type ConfigKind ¶
type ConfigKind string
ConfigKind names a query-config directive.
const ( ConfigResult ConfigKind = "result" ConfigUnorderedResult ConfigKind = "unorderedResult" ConfigExplain ConfigKind = "explain" ConfigExplainContains ConfigKind = "explainContains" ConfigCount ConfigKind = "count" ConfigError ConfigKind = "error" ConfigPlanHash ConfigKind = "planHash" ConfigMaxRows ConfigKind = "maxRows" ConfigSupportedVersion ConfigKind = "supported_version" ConfigInitialVersionAtLeast ConfigKind = "initialVersionAtLeast" ConfigInitialVersionLessThan ConfigKind = "initialVersionLessThan" ConfigSetup ConfigKind = "setup" ConfigSetupReference ConfigKind = "setupReference" ConfigDebugger ConfigKind = "debugger" ConfigResultMetadata ConfigKind = "resultMetadata" )
The config keys QueryConfig.parseConfig accepts from YAML.
const ConfigNoChecks ConfigKind = "noChecks"
ConfigNoChecks is the implicit config Java attaches to a query with no configs of its own. It is never written in a corpus file, so it is not in configKinds and parsing a literal `noChecks:` is an error, exactly as in Java.
func (ConfigKind) IsResultConfig ¶
func (k ConfigKind) IsResultConfig() bool
IsResultConfig reports membership of QueryConfig.RESULT_CONFIGS — the set that, once seen, forbids any later non-result config.
func (ConfigKind) IsResultConsumingConfig ¶
func (k ConfigKind) IsResultConsumingConfig() bool
IsResultConsumingConfig reports membership of QueryConfig.RESULT_CONSUMING_CONFIGS — the configs that actually draw rows, and so the ones `resultMetadata` requires at least one of.
func (ConfigKind) IsVersionDependentConfig ¶
func (k ConfigKind) IsVersionDependentConfig() bool
IsVersionDependentConfig reports membership of QueryConfig.VERSION_DEPENDENT_RESULT_CONFIGS.
type CopyBlock ¶
type CopyBlock struct {
Source CopyEndpoint
Dest CopyEndpoint
ExportLimit *int64
ImportChunkSize *int64
}
CopyBlock is a cross-cluster COPY. export_limit and import_chunk_size live on the block itself, not inside the source/dest maps.
type CopyEndpoint ¶
CopyEndpoint is one side of a CopyBlock.
type Corpus ¶
Corpus is a rooted view of the `.yamsql` files.
Include targets are resolved against this root, never against the including file's directory: Java hands the include string straight to ClassLoader.getResourceAsStream, so `include: include-block/includes/x.yamsql` means the same thing no matter which file writes it.
func OpenCorpus ¶
OpenCorpus locates the vendored corpus by walking up from the working directory until CorpusSubdir is found.
Walking up is what makes the package work identically under `go test` (cwd is the package dir) and under `bazel test` (cwd is the runfiles tree), without either needing to know the other's layout.
type Entry ¶
Entry is one key/value pair of a mapping, kept in source order.
Order is load-bearing: a result row is a mapping whose entry order IS the column order (Java's Matchers.matchMap walks entrySet() with a positional counter). A Go map would destroy that, so mappings are slices.
type File ¶
type File struct {
// Path is the corpus-root-relative path, which is also the string an
// `include:` uses to name this file.
Path string
Blocks []*Block
}
File is one parsed `.yamsql` file.
type IncludeBlock ¶
type IncludeBlock struct {
Resource string
// Resolved is the parsed target, populated by [Corpus.ParseFile]. It is nil
// when includes were not followed.
Resolved *File
}
IncludeBlock names another corpus file. Resource is root-relative, exactly as written; Java feeds it straight to ClassLoader.getResourceAsStream.
type InertDirective ¶
type InertDirective struct {
// Where names the construct that ignored the key, e.g. "test_block".
Where string
Key string
Line int
}
InertDirective is a key the corpus writes that the Java parser never reads.
These are not errors — Java probes its option maps with containsKey and leaves anything else alone — but they are always defects in the corpus, because the author plainly expected the key to do something. `supported_version` written directly under `test_block:` instead of under its `options:` is the canonical example: it gates nothing at all.
func (InertDirective) String ¶
func (d InertDirective) String() string
type ManifestEntry ¶
type ManifestEntry struct {
Path string
Polarity Polarity
// Reason is the mechanism, not a restatement of the file name.
Reason string
// Source is the Java test class that asserts this polarity.
Source string
}
ManifestEntry records one file's polarity and why.
type NamedSetup ¶
NamedSetup is one `transaction_setups` entry.
type OptionsBlock ¶
type OptionsBlock struct {
SupportedVersion *Version
RequiredClusters *int64
ConnectionOptions []Entry
}
OptionsBlock is the file preamble. Java requires it to be document 0 of a top-level file; an included file carrying one is a parse error unless that file is itself the entry point.
type ParseError ¶
ParseError locates a parse failure within a corpus file.
func (*ParseError) Error ¶
func (e *ParseError) Error() string
func (*ParseError) Unwrap ¶
func (e *ParseError) Unwrap() error
type Polarity ¶
type Polarity int
Polarity is what upstream expects a corpus file to do when it is run.
const ( // Positive: the file is expected to run to completion. Positive Polarity = iota // NegativeParse: the file is expected to fail, and to fail while the YAML // is being loaded. The Go parser must reject these too. NegativeParse // NegativeExecution: the file is expected to fail, but only once a query // runs and an assertion is checked. The Go parser must ACCEPT these — the // document structure is entirely valid. NegativeExecution // Fragment: the file is never run on its own, only pulled in by an // `include:`. It is not self-sufficient and asserts nothing standalone. Fragment // FixedVersionMeta: the file's polarity is defined ONLY against a version // the Java test class pins, and it is not run by any current-version // stream. A single-current-version runner cannot evaluate it in either // direction — the gate it exercises resolves the other way, so both "it // passed" and "it failed" are meaningless. // // These are meta-tests of the version machinery itself, not of the engine. FixedVersionMeta )
func PolarityOf ¶
PolarityOf returns the recorded polarity, defaulting to Positive.
type SchemaTemplateBlock ¶
type SchemaTemplateBlock struct {
Variants []SchemaTemplateVariant
// ListForm records whether the source used the list-of-variants syntax,
// which is the form subject to overlap/coverage validation.
ListForm bool
}
SchemaTemplateBlock holds one or more DDL variants. The plain-string form yields a single variant covering all versions.
type SchemaTemplateVariant ¶
SchemaTemplateVariant is one version-gated `CREATE SCHEMA TEMPLATE` body.
type Segment ¶
type Segment struct {
// Raw is the full segment including both `!!` delimiters, which is the
// exact substring Java replaces when adapting the query.
Raw string
// Body is the mini-YAML between the delimiters.
Body string
// Value is Body parsed with the parameter-injection tag set.
Value *Value
Kind SegmentKind
// Offset is Raw's byte offset within the query string.
Offset int
}
Segment is one `!!…!!` parameter injection inside a query string.
Phase 0 represents these; binding and substitution belong to execution.
func ParseSegments ¶
ParseSegments extracts the `!!…!!` parameter-injection segments from a query.
This mirrors QueryInterpreter.getInjections: scan for `!!`, find the next `!!` after it, and load the text between them as its own little YAML document using the parameter-injection tag set. Delimiters do not nest and are not escapable, so the scan is a plain left-to-right pairing — a query containing an odd number of `!!` is malformed, which Java asserts on.
Phase 0 stops at representation. Binding an unbound generator to a value needs a seeded Random and belongs to execution.
type SegmentKind ¶
type SegmentKind uint8
SegmentKind distinguishes a parameter-injection segment's shape.
const ( // SegmentLiteral is a bound literal: `!! 10 !!`. SegmentLiteral SegmentKind = iota // SegmentUnbound is a generator that a Random binds at execution: // `!! !r [2, 9] !!`. SegmentUnbound )
Segment shapes.
type SetupBlock ¶
SetupBlock is the manual `setup:` block: a connect target plus ordered steps.
type TestBlock ¶
type TestBlock struct {
Name string
Connect *Value
Preset string
Options TestBlockOptions
Tests []*Test
}
TestBlock is a named group of tests plus the knobs controlling how they run.
type TestBlockOptions ¶
type TestBlockOptions struct {
Mode string
Repetition *int64
Seed *int64
CheckCache *bool
ConnectionLifecycle string
StatementType string
SupportedVersion *Version
ConnectionOptions []Entry
InitialVersionUnused bool
}
TestBlockOptions mirrors TestBlock.TestBlockOptions' YAML surface. Pointers distinguish "absent" from "set to the default", which matters because Java layers preset < options-map < execution-context.
type TransactionSetupsBlock ¶
type TransactionSetupsBlock struct {
Setups []NamedSetup
}
TransactionSetupsBlock maps names to reusable setup SQL, in source order.
type Value ¶
type Value struct {
Kind ValueKind
Line int
Col int
Bool bool
Int int64
Float float64
Str string
Seq []*Value
Map []Entry
// Tag and Inner are set only for KindTagged. Inner is never nil; a bare
// tag with no argument (`!null`, `!not_null`, `!ignore`) wraps a KindNull.
Tag string
Inner *Value
}
Value is a parsed YAML node with its custom tag preserved.
Only the fields belonging to Kind are meaningful.
func (*Value) AsString ¶
AsString returns the string content of a scalar, and whether v was one. A tagged scalar reports the tag's argument (`!sc "ell"` yields "ell").
func (*Value) IsNull ¶
IsNull reports whether v is the YAML null that Java's SafeConstructor turns into a Java null.
This is the hinge of the whole row model. Java's Matchers.valueElseKey tests `entry.getValue() == null`, and SnakeYAML constructs Java null for an elided value (`{A: }`), an explicit `null`, and `~` alike — so all three are the same thing and all three mean "positional cell". A custom-tagged `!null` is NOT this: it constructs an IsNullMatcher object, which is non-null in Java and therefore an ordinary by-column-name cell that happens to expect SQL NULL.
type ValueKind ¶
type ValueKind uint8
ValueKind discriminates Value.
type Version ¶
type Version struct {
// Current is true for the `!current_version` tag, which the release process
// rewrites into a literal version.
Current bool
Major int
Minor int
Build int
Patch int
}
Version is a yamsql semantic version: either the `!current_version` sentinel or a literal four-part version, mirroring SemanticVersion.parse.