javayamsql

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 26, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package javayamsql parses Apple's `.yamsql` acceptance-test format — the multi-document YAML dialect driving the Java record layer's yaml-tests suite.

The vendored corpus lives at third_party/apple/fdb-record-layer/ (see the README there). This package only *parses*; nothing here executes a query.

The parser is deliberately STRICTER than Java's. Java probes its option maps with containsKey and silently ignores any key it does not recognise, so a typo'd or newly-added upstream directive reads as "absent" and the test quietly checks less than it claims to. Here every block key, command, config key and YAML tag must be in the known set or parsing fails. That is the whole point of the corpus gate: when a version bump introduces a directive, it must fail loudly and by name rather than being dropped on the floor.

Index

Constants

View Source
const (
	TagLong           = "!l"
	TagFloat          = "!f"
	TagBytes          = "!b"
	TagIgnore         = "!ignore"
	TagNull           = "!null"
	TagNotNull        = "!not_null"
	TagStringContains = "!sc"
	TagUUID           = "!uuid"
	TagPos            = "!pos"
	TagRandomStr      = "!randomStr"
	TagVector16       = "!v16"
	TagVector32       = "!v32"
	TagVector64       = "!v64"
	TagCurrentVersion = "!current_version"

	TagRandom  = "!r"
	TagArray   = "!a"
	TagInList  = "!in"
	TagNullArg = "!n"
)

The custom YAML tags the corpus uses.

Two registries exist upstream and they are NOT the same set:

  • Row/config tags, registered through the CustomTag ServiceLoader and installed by CustomYamlConstructor. These appear in `result:` rows, `supported_version:` values and so on.
  • Parameter-injection tags, registered by QueryInterpreter's private QueryParameterYamlConstructor. These appear only inside a `!!…!!` segment of a query string.

The two overlap but neither contains the other: `!not_null`/`!null`/`!ignore`/ `!pos`/`!sc`/`!current_version` are row-only, while `!r`/`!a`/`!in`/`!n` are segment-only. Mixing them up is silently wrong in Java (an unknown tag inside a segment falls through to a plain scalar), so the two sets are kept apart here and checked against the context they appear in.

View Source
const CorpusSubdir = "third_party/apple/fdb-record-layer/yaml-tests/src/test/resources"

CorpusSubdir is the vendored corpus's location relative to the repo root.

Variables

View Source
var KnownInertDirectives = []struct {
	Path  string
	Where string
	Key   string
	Line  int
}{
	{"between.yamsql", "test_block", "supported_version", 97},
	{"index-ddl-aggregates-only.yamsql", "options", "noChecks", 27},
	{"uuid-non-prepared.yamsql", "test_block", "supported_version", 62},
	{"uuid-prepared.yamsql", "test_block", "supported_version", 64},
}

KnownInertDirectives are the corpus's dead keys: written by an author who plainly meant them to do something, and read by nothing.

These are upstream defects, not parser gaps. `supported_version` belongs under a test_block's `options:` map; written directly under `test_block:` it is never looked at, so all three files below run unconditionally against every version instead of being gated. `noChecks` is a query-config name that has no meaning in a file preamble.

They are pinned so the set cannot grow silently. A new entry appearing here means someone added another directive that does nothing.

View Source
var Manifest = buildManifest()

Manifest maps corpus-root-relative paths to their polarity. Files absent from it are Positive — the overwhelming majority, and listing 200-odd of them would bury the 60 that carry information.

Every entry below was read off the Java test class named in Source, not inferred from the directory name. That distinction matters: the `include-block/includes/` directory looks like a bag of fragments and is not one — IncludeBlockTest runs all ten standalone, expecting eight to fail.

Functions

func Composition

func Composition() map[Polarity]int

Composition counts the manifest's entries by polarity.

It exists because the manifest's totals get quoted in prose, and prose denominators drift: "42 execution-level negatives" (an entry count) and "20 booked as polarity:negative-execution" (a ledger outcome) are different numbers about different things, and fusing them is how a corrected figure becomes a wrong one. Anything quoting a manifest total should read it from here.

Positive entries are the ones recorded EXPLICITLY — files carrying a reason worth keeping even though they pass. The overwhelming majority of positives are absent from the manifest entirely and are not counted here.

func IsKnownTag

func IsKnownTag(tag string) bool

IsKnownTag reports whether tag is a custom tag this format defines anywhere.

func IsRowTag

func IsRowTag(tag string) bool

IsRowTag reports whether tag is legal outside a parameter-injection segment.

func IsSegmentTag

func IsSegmentTag(tag string) bool

IsSegmentTag reports whether tag is legal inside a `!!…!!` segment.

func KnownTags

func KnownTags() []string

KnownTags returns every custom tag, for census and drift reporting.

func ParseLevelNegatives

func ParseLevelNegatives() []string

ParseLevelNegatives returns the files the parser is required to reject.

func SelectedAtCurrentVersion

func SelectedAtCurrentVersion(atLeast, lessThan *Version) bool

SelectedAtCurrentVersion reports whether the half-open range [atLeast, lessThan) contains the version under test, with a nil bound widening to MIN / MAX exactly as SemanticVersionRanges.atLeast/lessThan do.

"The version under test" is the `!current_version` singleton, because that is what a single-version runner reports: EmbeddedYamlConnectionFactory's getVersionsUnderTest is {SemanticVersion.current()}, and CURRENT sorts above every literal version. Two consequences fall out of that ordering and are the whole of how a Go run resolves the corpus's version machinery: an `initialVersionAtLeast: <literal>` branch is always selected, and an `initialVersionLessThan:` branch never is — which is the same thing as saying a version-variant list collapses to its current-version branch.

func SupportedAtCurrentVersion

func SupportedAtCurrentVersion(min *Version) bool

SupportedAtCurrentVersion reports whether a `supported_version` gate admits the version under test, mirroring SupportedVersionCheck.parse: the gate excludes the run when any version under test is strictly older than the declared minimum.

With a single version under test, and that version being the CURRENT singleton which sorts above every literal, the gate ADMITS unconditionally. The function exists to make that a stated, testable fact rather than an omission — if the runner ever gains a second version under test, this is the one place the answer stops being constant.

Types

type Block

type Block struct {
	Kind BlockKind
	Line int

	// Inert lists keys within this block that Java's parser silently ignores.
	Inert []InertDirective

	Options           *OptionsBlock
	SchemaTemplate    *SchemaTemplateBlock
	Setup             *SetupBlock
	Test              *TestBlock
	Include           *IncludeBlock
	TransactionSetups *TransactionSetupsBlock
	Copy              *CopyBlock
}

Block is a single YAML document region. Exactly one of the typed payload pointers is non-nil, selected by Kind.

type BlockKind

type BlockKind string

BlockKind names the seven top-level document keys Java's Block.parse dispatches on. Anything else is "Cannot recognize the type of block".

const (
	BlockOptions           BlockKind = "options"
	BlockSchemaTemplate    BlockKind = "schema_template"
	BlockSetup             BlockKind = "setup"
	BlockTest              BlockKind = "test_block"
	BlockInclude           BlockKind = "include"
	BlockTransactionSetups BlockKind = "transaction_setups"
	BlockCopy              BlockKind = "copy_block"
)

The recognised block keys.

type Cell

type Cell struct {
	Line int
	// Key is the mapping key node, which carries the !pos tag when present.
	Key *Value
	// Val is the mapping value node, always non-nil; YAML null when elided.
	Val *Value
	// Pos is the explicit 1-based column from a `!pos n` key, else nil.
	Pos *int64
}

Cell is one expected column value within a Row.

The by-name/positional split is Java's Matchers.matchMap, and it is entirely driven by whether the mapping entry's VALUE is YAML null:

{ID: 10}        → by name "ID", expecting 10
{10}            → positional column 1, expecting 10   (value is null, key is the expectation)
{ID: !null}     → by name "ID", expecting SQL NULL    (!null is a tag, not YAML null)
{!pos 2: 10}    → positional column 2, expecting 10

See Value.IsNull for why the third case is not the second.

func (Cell) ColumnIndex

func (c Cell) ColumnIndex(ordinal int) int64

ColumnIndex returns the 1-based column this cell matches, given its ordinal position in the row. An explicit `!pos` overrides the ordinal.

func (Cell) ColumnName

func (c Cell) ColumnName() (string, bool)

ColumnName returns the column name for a by-name cell. It is meaningless for a positional cell and reports ok=false there.

func (Cell) Expected

func (c Cell) Expected() *Value

Expected returns the value this cell asserts: the mapping value, or the key itself when the value is YAML null.

func (Cell) Positional

func (c Cell) Positional() bool

Positional reports whether this cell is matched by column number rather than by column name.

type Census

type Census struct {
	Files   int
	Blocks  map[BlockKind]int
	Queries int
	// Commands counts every command including queries, so
	// Commands[CommandQuery] == Queries.
	Commands map[CommandKind]int
	Configs  map[ConfigKind]int
	// Tags counts custom-tag occurrences anywhere in a file, including inside
	// parameter-injection segments.
	Tags map[string]int
	// Rows and Cells count expected result rows and their cells.
	Rows  int
	Cells int
	// PositionalCells is the subset of Cells matched by column number.
	PositionalCells int
	// Segments counts `!!…!!` parameter injections.
	Segments int
	// Includes counts include blocks (not distinct targets).
	Includes int
}

Census counts the directive and tag surface actually exercised by a set of parsed files.

It exists so that a version bump fails *describably*. A parse gate alone reports "still parses"; it cannot notice that upstream deleted every use of a directive, or doubled the query count. The census turns those into a diff.

func NewCensus

func NewCensus() *Census

NewCensus returns an empty census with its maps ready.

func (*Census) Add

func (c *Census) Add(f *File)

Add folds one file into the census. Included files are NOT followed; count them separately so each file contributes exactly once.

func (*Census) Line

func (c *Census) Line() string

Line renders the census as one stable, greppable line.

type Command

type Command struct {
	Kind CommandKind
	Line int

	// Query is the raw query text, parameter-injection segments included.
	Query string
	// Segments are the `!!…!!` parameter injections found in Query, in order.
	Segments []Segment
	// Payload is the value of a load-schema-template / set-schema-state command.
	Payload string
	// Configs are the checks attached to a query command. Empty means Java
	// substitutes an implicit noChecks config.
	Configs []*Config
}

Command is one executable step: a query with its configs, or a metadata command carrying an opaque payload.

func (*Command) ImplicitNoChecks

func (c *Command) ImplicitNoChecks() bool

ImplicitNoChecks reports whether Java would attach a synthetic `noChecks` config to this command because it declares none of its own.

type CommandKind

type CommandKind string

CommandKind names the three command keys Java's Command.parse dispatches on.

const (
	CommandQuery              CommandKind = "query"
	CommandLoadSchemaTemplate CommandKind = "load schema template"
	CommandSetSchemaState     CommandKind = "set schema state"
)

The recognised command keys. Note the two non-query ones contain a space.

type Config

type Config struct {
	Kind ConfigKind
	Line int

	// Raw is the directive's value exactly as parsed, always non-nil.
	Raw *Value

	// Rows is populated for result/unorderedResult when the value is a list of
	// row mappings. It stays nil for `result:` with a null or `!ignore` value,
	// both of which Java treats specially rather than as a row list.
	Rows []Row

	// Version is populated for supported_version and initialVersion*.
	Version *Version
	// ErrorCode is the resolved 5-character SQLSTATE for an `error:` config.
	ErrorCode string
	// Number is populated for count, planHash and maxRows.
	Number *int64
	// Text is populated for explain, explainContains, setup, setupReference
	// and debugger.
	Text string
}

Config is one check attached to a query.

type ConfigKind

type ConfigKind string

ConfigKind names a query-config directive.

const (
	ConfigResult                 ConfigKind = "result"
	ConfigUnorderedResult        ConfigKind = "unorderedResult"
	ConfigExplain                ConfigKind = "explain"
	ConfigExplainContains        ConfigKind = "explainContains"
	ConfigCount                  ConfigKind = "count"
	ConfigError                  ConfigKind = "error"
	ConfigPlanHash               ConfigKind = "planHash"
	ConfigMaxRows                ConfigKind = "maxRows"
	ConfigSupportedVersion       ConfigKind = "supported_version"
	ConfigInitialVersionAtLeast  ConfigKind = "initialVersionAtLeast"
	ConfigInitialVersionLessThan ConfigKind = "initialVersionLessThan"
	ConfigSetup                  ConfigKind = "setup"
	ConfigSetupReference         ConfigKind = "setupReference"
	ConfigDebugger               ConfigKind = "debugger"
	ConfigResultMetadata         ConfigKind = "resultMetadata"
)

The config keys QueryConfig.parseConfig accepts from YAML.

const ConfigNoChecks ConfigKind = "noChecks"

ConfigNoChecks is the implicit config Java attaches to a query with no configs of its own. It is never written in a corpus file, so it is not in configKinds and parsing a literal `noChecks:` is an error, exactly as in Java.

func (ConfigKind) IsResultConfig

func (k ConfigKind) IsResultConfig() bool

IsResultConfig reports membership of QueryConfig.RESULT_CONFIGS — the set that, once seen, forbids any later non-result config.

func (ConfigKind) IsResultConsumingConfig

func (k ConfigKind) IsResultConsumingConfig() bool

IsResultConsumingConfig reports membership of QueryConfig.RESULT_CONSUMING_CONFIGS — the configs that actually draw rows, and so the ones `resultMetadata` requires at least one of.

func (ConfigKind) IsVersionDependentConfig

func (k ConfigKind) IsVersionDependentConfig() bool

IsVersionDependentConfig reports membership of QueryConfig.VERSION_DEPENDENT_RESULT_CONFIGS.

type CopyBlock

type CopyBlock struct {
	Source          CopyEndpoint
	Dest            CopyEndpoint
	ExportLimit     *int64
	ImportChunkSize *int64
}

CopyBlock is a cross-cluster COPY. export_limit and import_chunk_size live on the block itself, not inside the source/dest maps.

type CopyEndpoint

type CopyEndpoint struct {
	Cluster *int64
	Path    string
}

CopyEndpoint is one side of a CopyBlock.

type Corpus

type Corpus struct {
	FS fs.FS
}

Corpus is a rooted view of the `.yamsql` files.

Include targets are resolved against this root, never against the including file's directory: Java hands the include string straight to ClassLoader.getResourceAsStream, so `include: include-block/includes/x.yamsql` means the same thing no matter which file writes it.

func OpenCorpus

func OpenCorpus() (*Corpus, error)

OpenCorpus locates the vendored corpus by walking up from the working directory until CorpusSubdir is found.

Walking up is what makes the package work identically under `go test` (cwd is the package dir) and under `bazel test` (cwd is the runfiles tree), without either needing to know the other's layout.

func (*Corpus) List

func (c *Corpus) List() ([]string, error)

List returns every `.yamsql` path in the corpus, root-relative and sorted.

func (*Corpus) ParseFile

func (c *Corpus) ParseFile(p string) (*File, error)

ParseFile parses one corpus file as a top-level entry point, expanding its include graph inline.

type Entry

type Entry struct {
	Key *Value
	Val *Value
}

Entry is one key/value pair of a mapping, kept in source order.

Order is load-bearing: a result row is a mapping whose entry order IS the column order (Java's Matchers.matchMap walks entrySet() with a positional counter). A Go map would destroy that, so mappings are slices.

type File

type File struct {
	// Path is the corpus-root-relative path, which is also the string an
	// `include:` uses to name this file.
	Path   string
	Blocks []*Block
}

File is one parsed `.yamsql` file.

func Parse

func Parse(path string, src []byte) (*File, error)

Parse parses the documents of one `.yamsql` file.

It does not follow `include:` directives — the include target is resolved against the corpus root, which Parse has no handle on. Use Corpus.ParseFile for that.

type IncludeBlock

type IncludeBlock struct {
	Resource string
	// Resolved is the parsed target, populated by [Corpus.ParseFile]. It is nil
	// when includes were not followed.
	Resolved *File
}

IncludeBlock names another corpus file. Resource is root-relative, exactly as written; Java feeds it straight to ClassLoader.getResourceAsStream.

type InertDirective

type InertDirective struct {
	// Where names the construct that ignored the key, e.g. "test_block".
	Where string
	Key   string
	Line  int
}

InertDirective is a key the corpus writes that the Java parser never reads.

These are not errors — Java probes its option maps with containsKey and leaves anything else alone — but they are always defects in the corpus, because the author plainly expected the key to do something. `supported_version` written directly under `test_block:` instead of under its `options:` is the canonical example: it gates nothing at all.

func (InertDirective) String

func (d InertDirective) String() string

type ManifestEntry

type ManifestEntry struct {
	Path     string
	Polarity Polarity
	// Reason is the mechanism, not a restatement of the file name.
	Reason string
	// Source is the Java test class that asserts this polarity.
	Source string
}

ManifestEntry records one file's polarity and why.

type NamedSetup

type NamedSetup struct {
	Name string
	SQL  string
	Line int
}

NamedSetup is one `transaction_setups` entry.

type OptionsBlock

type OptionsBlock struct {
	SupportedVersion  *Version
	RequiredClusters  *int64
	ConnectionOptions []Entry
}

OptionsBlock is the file preamble. Java requires it to be document 0 of a top-level file; an included file carrying one is a parse error unless that file is itself the entry point.

type ParseError

type ParseError struct {
	Path string
	Line int
	Err  error
}

ParseError locates a parse failure within a corpus file.

func (*ParseError) Error

func (e *ParseError) Error() string

func (*ParseError) Unwrap

func (e *ParseError) Unwrap() error

type Polarity

type Polarity int

Polarity is what upstream expects a corpus file to do when it is run.

const (
	// Positive: the file is expected to run to completion.
	Positive Polarity = iota
	// NegativeParse: the file is expected to fail, and to fail while the YAML
	// is being loaded. The Go parser must reject these too.
	NegativeParse
	// NegativeExecution: the file is expected to fail, but only once a query
	// runs and an assertion is checked. The Go parser must ACCEPT these — the
	// document structure is entirely valid.
	NegativeExecution
	// Fragment: the file is never run on its own, only pulled in by an
	// `include:`. It is not self-sufficient and asserts nothing standalone.
	Fragment
	// FixedVersionMeta: the file's polarity is defined ONLY against a version
	// the Java test class pins, and it is not run by any current-version
	// stream. A single-current-version runner cannot evaluate it in either
	// direction — the gate it exercises resolves the other way, so both "it
	// passed" and "it failed" are meaningless.
	//
	// These are meta-tests of the version machinery itself, not of the engine.
	FixedVersionMeta
)

func PolarityOf

func PolarityOf(path string) Polarity

PolarityOf returns the recorded polarity, defaulting to Positive.

func (Polarity) String

func (p Polarity) String() string

type Row

type Row struct {
	Line  int
	Cells []Cell
}

Row is one expected result row: an ordered list of cells.

type SchemaTemplateBlock

type SchemaTemplateBlock struct {
	Variants []SchemaTemplateVariant
	// ListForm records whether the source used the list-of-variants syntax,
	// which is the form subject to overlap/coverage validation.
	ListForm bool
}

SchemaTemplateBlock holds one or more DDL variants. The plain-string form yields a single variant covering all versions.

type SchemaTemplateVariant

type SchemaTemplateVariant struct {
	Line       int
	Definition string
	AtLeast    *Version
	LessThan   *Version
}

SchemaTemplateVariant is one version-gated `CREATE SCHEMA TEMPLATE` body.

type Segment

type Segment struct {
	// Raw is the full segment including both `!!` delimiters, which is the
	// exact substring Java replaces when adapting the query.
	Raw string
	// Body is the mini-YAML between the delimiters.
	Body string
	// Value is Body parsed with the parameter-injection tag set.
	Value *Value
	Kind  SegmentKind
	// Offset is Raw's byte offset within the query string.
	Offset int
}

Segment is one `!!…!!` parameter injection inside a query string.

Phase 0 represents these; binding and substitution belong to execution.

func ParseSegments

func ParseSegments(query string, line int) ([]Segment, error)

ParseSegments extracts the `!!…!!` parameter-injection segments from a query.

This mirrors QueryInterpreter.getInjections: scan for `!!`, find the next `!!` after it, and load the text between them as its own little YAML document using the parameter-injection tag set. Delimiters do not nest and are not escapable, so the scan is a plain left-to-right pairing — a query containing an odd number of `!!` is malformed, which Java asserts on.

Phase 0 stops at representation. Binding an unbound generator to a value needs a seeded Random and belongs to execution.

type SegmentKind

type SegmentKind uint8

SegmentKind distinguishes a parameter-injection segment's shape.

const (
	// SegmentLiteral is a bound literal: `!! 10 !!`.
	SegmentLiteral SegmentKind = iota
	// SegmentUnbound is a generator that a Random binds at execution:
	// `!! !r [2, 9] !!`.
	SegmentUnbound
)

Segment shapes.

type SetupBlock

type SetupBlock struct {
	Connect           *Value
	Steps             []*Command
	ConnectionOptions []Entry
}

SetupBlock is the manual `setup:` block: a connect target plus ordered steps.

type Test

type Test struct {
	Line    int
	Command *Command
}

Test is one entry of a test_block's `tests:` list: a command plus its configs.

type TestBlock

type TestBlock struct {
	Name    string
	Connect *Value
	Preset  string
	Options TestBlockOptions
	Tests   []*Test
}

TestBlock is a named group of tests plus the knobs controlling how they run.

type TestBlockOptions

type TestBlockOptions struct {
	Mode                 string
	Repetition           *int64
	Seed                 *int64
	CheckCache           *bool
	ConnectionLifecycle  string
	StatementType        string
	SupportedVersion     *Version
	ConnectionOptions    []Entry
	InitialVersionUnused bool
}

TestBlockOptions mirrors TestBlock.TestBlockOptions' YAML surface. Pointers distinguish "absent" from "set to the default", which matters because Java layers preset < options-map < execution-context.

type TransactionSetupsBlock

type TransactionSetupsBlock struct {
	Setups []NamedSetup
}

TransactionSetupsBlock maps names to reusable setup SQL, in source order.

type Value

type Value struct {
	Kind ValueKind
	Line int
	Col  int

	Bool  bool
	Int   int64
	Float float64
	Str   string
	Seq   []*Value
	Map   []Entry

	// Tag and Inner are set only for KindTagged. Inner is never nil; a bare
	// tag with no argument (`!null`, `!not_null`, `!ignore`) wraps a KindNull.
	Tag   string
	Inner *Value
}

Value is a parsed YAML node with its custom tag preserved.

Only the fields belonging to Kind are meaningful.

func (*Value) AsInt

func (v *Value) AsInt() (int64, bool)

AsInt returns the integer content of a scalar, and whether v was one.

func (*Value) AsString

func (v *Value) AsString() (string, bool)

AsString returns the string content of a scalar, and whether v was one. A tagged scalar reports the tag's argument (`!sc "ell"` yields "ell").

func (*Value) IsNull

func (v *Value) IsNull() bool

IsNull reports whether v is the YAML null that Java's SafeConstructor turns into a Java null.

This is the hinge of the whole row model. Java's Matchers.valueElseKey tests `entry.getValue() == null`, and SnakeYAML constructs Java null for an elided value (`{A: }`), an explicit `null`, and `~` alike — so all three are the same thing and all three mean "positional cell". A custom-tagged `!null` is NOT this: it constructs an IsNullMatcher object, which is non-null in Java and therefore an ordinary by-column-name cell that happens to expect SQL NULL.

func (*Value) TagName

func (v *Value) TagName() string

TagName returns the custom tag on v, or "" if untagged.

func (*Value) Untag

func (v *Value) Untag() *Value

Untag returns the value beneath any custom tag.

type ValueKind

type ValueKind uint8

ValueKind discriminates Value.

const (
	KindNull ValueKind = iota
	KindBool
	KindInt
	KindFloat
	KindString
	KindSeq
	KindMap
	KindTagged
)

The kinds a Value may take. KindTagged wraps any of the others with one of the corpus's custom YAML tags (see tags.go).

func (ValueKind) String

func (k ValueKind) String() string

type Version

type Version struct {
	// Current is true for the `!current_version` tag, which the release process
	// rewrites into a literal version.
	Current bool
	Major   int
	Minor   int
	Build   int
	Patch   int
}

Version is a yamsql semantic version: either the `!current_version` sentinel or a literal four-part version, mirroring SemanticVersion.parse.

func (Version) String

func (v Version) String() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL