ptah-ydb-plugin

README

Ptah YDB provider

This repository owns the executable YDB provider for Ptah #4204. It is under development. Inspection covers row tables and their indexes and changefeeds, column tables, topics, views, coordination nodes, external data sources, external tables, and scheme secrets. Native SQL execution and session-bound data transactions are implemented, together with provider-owned migration history, event journals, tags, and coordination locks. Offline YAML and Go annotation decoding is available for coordination-node, topic, external data source external table and secret declarations, and YAML and Go annotation decoding for the partitioning, TTL, column families and changefeeds of row tables, including the columns of each family. Offline comparison, planning, and rendering produce native coordination, topic, changefeed, secret and external object operations (see Secrets, Topics, Changefeeds and External data sources and external tables). Native row-table creation and alteration of column families, columns, the TTL and the automatic partitioning settings are available through captured SDK planning requests. Execution verifies captured preconditions and reads back the result. Offline migration lint splits YQL migrations and checks YD rules; see Migration lint.

The executable opens native YDB connections and reads row-table schemas through the public Ptah SDK. It supports anonymous, URL user/password, and URL token authentication. Session receipts contain the selected database and measured server version. The implementation does not import the Ptah application module.

Install it with ptah providers install ydb. The documentation says what YDB does in Ptah, from installing and selecting the provider to the objects it declares and how it runs a migration. It is published on docs.ptah.run, and its pages live in docs/.

The provider ID is github.com/stokaro/ptah-ydb-plugin. Model kinds use the github.com/stokaro/ptah-ydb-plugin/ namespace. This changes the experimental model identities; pre-v1, no compatibility with the earlier unpublished names is owed. Older envelopes and plans are refused.

Releases

A tag that starts with v runs .github/workflows/release.yml, which publishes a GitHub release through GoReleaser (.goreleaser.yaml). Each release carries one archive per platform, two JSON documents and a checksums.txt file:

ptah-plugin-ydb_<version>_<os>_<arch>.tar.gz   (.zip on Windows)
ptah-provider.json
release-lines.json
checksums.txt

<version> is the tag without the leading v. The platforms are linux, darwin and windows on amd64 and arm64. Every archive holds the ptah-plugin-ydb binary (ptah-plugin-ydb.exe on Windows), README.md and LICENSE. The binaries are built with CGO_ENABLED=0 and -trimpath.

ptah-provider.json names the provider, the release version, the oldest provider contract a host must implement to start it (min_contract, a major and a minor version), its engines and the archive and executable for each platform. A host starts the release when its contract has the same major version and a minor version at least min_contract.minor. The document states the minimum the provider declares in its Describe reply, so the two cannot disagree. This provider requires contract 25.7. It negotiates database.context and values.context to resolve dependency paths, and planning.lifecycle to order native replacements around common table operations. It offers planning.statement-style (contract 25.8), which ptah-compat's migrate diff needs, because it plans in the exact statement style: no comment that only describes a statement, no guard the plan does not need and no CASCADE. The operations the provider plans carry none of these in either style, so an exact plan is the plan the annotated style gets. ptah providers install reads the document to choose the newest release the host can start, without downloading an archive. release-lines.json lists the release lines the provider declares.

checksums.txt lists the SHA-256 digest of every archive and of both documents in sha256sum format (<hex digest> <file name>). A host checks the archive it downloads against the digest listed there.

Pull requests that touch the release inputs run goreleaser check and a snapshot build, then verify the snapshot's checksums.txt and that ptah-provider.json names exactly the archives GoReleaser wrote. To try it locally:

GOWORK=off goreleaser release --snapshot --clean
(cd dist && shasum -a 256 -c checksums.txt)

Build and test

The module pins provider-sdk v0.35.0, which speaks provider contract 25.37. The provider still requires 25.7 of its host and offers each later feature to a host that lists it. Builds require neither the Ptah application source nor a local SDK checkout.

Database context is resolved without connecting or reading credentials. A dev realm carries both its selected root and the authored database root. Context is captured separately from object identity and survives normalization, conversion, and saved operations. Absolute secret and external data source references require this context before selection or planning. Unresolved references retain an explicit pending state; they cannot report an empty or partial dependency list.

Run the unit suite and build the executable:

GOWORK=off go test -race ./...
GOWORK=off CGO_ENABLED=0 go build -o bin/ptah-plugin-ydb ./cmd/ptah-plugin-ydb

The provider is held to Ptah's lint contract: .golangci.yml carries the host's linters and settings, and the Makefile runs the host's other lint tools. Lint as CI does, with golangci-lint v2.15.0 on the PATH; qtlint, nolintguard and teststyle are pinned as tools in go.mod:

make lint       # qtlint, nolintguard, golangci-lint and the test style baseline
make lint-fix   # the automatic fixes of qtlint and golangci-lint, then make lint

Every linter runs over both build contours, the default one and the integration tag. scripts/check-test-style.sh holds the tests to the declarative rule: no if, switch or goto in a test function. The functions that broke it when the rule was adopted are listed in .teststyle-baseline.json; the list may shrink and must not grow.

CI compares .golangci.yml with the host's: every linter and formatter the host enables is enabled, every setting of one is the host's, and the exclusions that name no path are the host's. The gci prefix, the rowserrcheck packages, the depguard messages and the path exclusions name this module and are not compared. To run the comparison against a checkout of Ptah:

GOWORK=off go run ./internal/cmd/lintcontract -host ../ptah/.golangci.yml

The executable requires the SDK's authenticated local subprocess bootstrap. Run it through a Ptah SDK client rather than invoking it as a command-line tool. The committed module has no replace directive or workspace file. scripts/check-module-boundaries.sh rejects local replacements and host-module dependencies.

Live checks require an isolated test database. Schema tests create uniquely named directories, tables, and topics and remove them afterward. Session tests only read server metadata.

PTAH_YDB_TEST_URL='ydb://localhost:2136/local?go_balancer=disable' \
  GOWORK=off go test -race -tags integration ./integration -count=1 -v

A test whose dev database must be on a server other than the target's reads a second isolated server from PTAH_YDB_OTHER_TEST_URL and skips without it. CI starts the second server beside the first.

The test servers enable enable_vector_index and enable_external_data_sources; the 26.2 fixture also enables enable_add_unique_index. A full-text index is refused on 25.1, and the full-text tests that need one skip there; the 25.1 run checks the refusal instead. One full-text test turns enable_fulltext_index off for the whole 26.2 cluster through its dynamic configuration and drops the configuration when it ends.

Ptah's provider conformance suite, from the SDK's package conformance, runs in integration/conformance_e2e_test.go. Its offline part runs in the unit job and needs no database: the handshake, the description and its negotiation under every host profile, every RPC, offline calls, and the model identities of the last release in integration/testdata. Its live part runs with the other live tests: sessions, reads, locks across two processes, transactions, a table created and dropped, a refused statement, and a login whose password is the suite's canary. A release records its model identities there with the suite's command:

GOWORK=off go run github.com/stokaro/ptah/provider-sdk/cmd/ptah-provider-conformance \
  models -executable bin/ptah-plugin-ydb -provider-id github.com/stokaro/ptah-ydb-plugin

CI runs this process test against YDB 25.1.4.7 and 26.2.1.14. It also rejects host-module imports and local module replacements. These checks cover sessions, SQL execution, commit/rollback, range conflicts, durable migration history, atomic revision comparisons, shared data/history/tag transactions, durable event journals, immutable paged metadata snapshots, process lock exclusion, confirmed release, and write refusal after lease loss, native row-table observations, offline normalization, index implementation tables, changefeed consumers and retention, standalone native objects, exact-directory completeness, and refusals for incomplete reads. Coordination tests cover source decoding through planning, rendering, native execution, and inspection. They do not establish complete migration support or host CLI integration.

Native CLI development acceptance

Development checks with the experimental host adapter verified reviewed SQL migration execution and rollback against YDB 25.1.4.7 and 26.2.1.14. The checks also exercised status, journals, tags, and default locking through the executable. Schema changes used --tx-mode none for migrations up and a -- +ptah no_transaction directive in the down file. YDB does not provide transactional schema changes.

The host and provider are built against the published SDK pin with GOWORK=off. The live tests run twice per change and nightly. The blocking run builds the newest published Ptah release whose provider contract this provider can run on (scripts/resolve-host.sh); it is skipped with a notice while no release speaks that contract. A second, non-blocking run builds Ptah's master branch, so a host change that breaks this provider shows up early. A third run, on each image, builds a host a run names: a branch, tag or commit of stokaro/ptah, given as the host_ref input of a manual run or as the first line of .github/host-ref in a pull request whose tests need a host change master does not have yet (scripts/live-lines.sh). It blocks, because the run asked for that host, and a ref that names no commit fails it. Each run also builds the host's ptah-compat from the same checkout and names it in PTAH_HOST_COMPAT_TEST_BIN. The TestHostCompat tests run it with PTAH_PROVIDER_CONFIG naming this provider's manifest, which is how ptah-compat takes one. They skip, with the binary's answer, where ptah-compat still builds in a YDB engine of its own, which claims ydb and refuses this provider's registration; a host without the built-in engine (stokaro/ptah#5298) runs them. This repository owns the YDB end-to-end, live and parity tests; Ptah's own CI checks the provider contract only on its in-tree fixture provider. These are source-build checks; packaged installation and default bundling remain separate requirements. Host schema inspect --format json exports the native desired model, including provider-owned values and coverage. Common collections use names such as Tables, Fields, and Indexes. This JSON is diagnostic output, not a schema file input. Full native CLI planning remains pending.

TestHostAppliesAndAltersARowTable_HappyPath drives ptah schema apply of the current host through this provider: it creates a row table from a Go source, finds nothing to change on the next run, sets a TTL in place, and finds nothing to change again. The run passes --tx-mode none, because the host refuses a provider step that forbids a transaction under the default mode.

Connection addresses

Use ydb://host:2136/database or ydbs://host:2135/database for TLS. The database comes from the path or from the database parameter; naming it twice is refused when the two differ.

The accepted parameters follow the host's embedded engine:

• database, token and use_env_credentials choose the database and the credential. A token, URL user/password and use_env_credentials cannot be combined. use_env_credentials reads the YDB_* variables of the provider process, service-account keys and instance metadata included, and fails when none is set, so an environment that would connect anonymously is refused. An address with no credential source connects anonymously.
• go_balancer and the older balancer choose the balancer; naming both is refused, and so is a value the SDK does not know. go_default_idempotent takes a boolean and prefetch_query_result_parts a non-negative integer. The SDK reads these three.
• go_query_mode and query_mode accept query only. go_fake_tx and go_query_bind are refused by name, because a faked transaction would read outside the snapshot read and the SDK's binders rewrite a ? inside a YQL literal.
• monitoring names the cluster's monitoring endpoint, an http:// or https:// address with no user, path, query or fragment. A session reads the database's feature flags there when it opens (see "Feature flags"), and inspection reads the description of each column table there. Both reads present the connection's own credential in YDB's bare Authorization header, follow no redirect and read at most 4 MiB. A credential that travels over TLS (ydbs://) is never sent to a plain http:// endpoint: the session is refused and asked for https://.

Unknown or repeated parameters, conflicting database paths, and malformed values are refused, and a refusal names the parameter and never its value. Two refusals are stricter than the host's: a fragment, and a query the host reads as empty because it cannot unescape it. The testdata/oracle corpus beside internal/connection holds the host's answer for each address it covers. Credentials travel in the session-open request and are omitted from receipts, SDK addresses, and error diagnostics. Native errors currently report only the failed operation, which limits diagnosis until structured redaction is added.

Feature flags

A release line runs with default feature flags, and an operator can turn a flag on or off, so one cluster can do more or less than its line's defaults say. When the address names a monitoring endpoint, opening a session reads /viewer/json/feature_flags?database=<db> and folds the flags that gate a capability the provider owns into the session's capabilities, false included. A failed read fails the session rather than falling back to the defaults, as in the host's embedded engine. The refusal is a reason the host prints, and it names the right a user lacks: on 26.x the page is served only to a user with DESCRIBE SCHEMA on the database. What the endpoint answered stays out of the reason. Without the endpoint the session reports the defaults of every supported line.

Capability	Flag	Default
tiered_ttl	EnableTieringInColumnShard	false on every line from 25.1 to 26.2
changefeed_topic_auto_partitioning	EnableTopicAutopartitioningForCDC	false on 25.1, true from 25.2
external_data_sources	EnableExternalDataSources	false on every line from 25.1 to 26.2
external_object_replace	EnableReplaceIfExistsForExternalEntities	false on every line from 25.1 to 26.2
resource_pools	EnableResourcePools	false on every line from 25.1 to 26.2
secrets	EnableSchemaSecrets	false through 25.3, true from 25.4
vector_indexes	EnableVectorIndex	false on 25.1, true from 25.2
full_text_indexes	EnableFulltextIndex	false through 26.1, true from 26.2
local_bloom_indexes	EnableLocalBloomFilterIndex	false through 26.1, true from 26.2
local_ngram_indexes	EnableLocalBloomNgramFilterIndex	false through 26.1, true from 26.2
unique_index_on_existing_table	EnableAddUniqueIndex	false on every line; 25.1 and 25.2 do not list the flag
index_rename	EnableMoveIndex	true on every line from 25.1 to 26.2, for a host that lists planning.index-renames
local_min_max_indexes	EnableLocalMinMaxIndex	false on every line from 25.1 to 26.2
column_family_cache_mode	EnableTableCacheModes	false through 25.3, true from 25.4; 25.1 and 25.2 do not list the flag

The release line alone decides the other vector capabilities, as the host's line presets name them: vector_index_maintained_on_write from 25.3 (an index built on 25.1 or 25.2 does not find the rows written after its build), and vector_bit_type from 26.1 (25.1 to 25.4 do not build an index over bit vectors over rows: bit vector type is not supported, Unsupported vector_type: VECTOR_TYPE_BIT).

The host maps more flags than these. The provider reports only the capabilities it plans with, because the host plans with the others itself. index_rename is false for a host that does not list planning.index-renames: such a host has no step to send a renamed index with, and would refuse the plan. internal/featureflags/testdata holds the pages the mapping is checked against, and a live test turns the flag on and off through the server's dynamic configuration and reads the capability each new session reports.

A plan refused for want of a capability a flag decides names the flag, after the host's words for the capability: ...; YDB keeps it behind feature flag <Flag>: turn the flag on, and name the cluster's monitoring endpoint in the URL (monitoring=http://host:8765) so Ptah reads the flags before it plans. A session without the endpoint takes its release line's default, so the flag alone does not change the answer.

Session close releases both the SQL pool and its parent native driver, even when the RPC context is canceled. Cleanup results are retained for repeated closes. A process accepts at most 1,024 lifetime session handles; start a new provider process when that bound is reached.

Dev realms

SQL cannot create a YDB database, so a dev or scratch database on a server the run does not own is a directory. dev_realm=<name> on an address makes the provider treat <database>/ptah_dev/<name> as the whole database of the session. A name is 1 to 64 lowercase letters, digits and underscores, and the address must name the database that holds the realm. Opening such a session creates the directory when it does not exist.

The engine declares database.dev-realms. A host that lists the feature sends a dev database request with the operator's address instead of a spec, and the provider creates a realm of the run's own in the database the address names, as the host's embedded engine does: a name of 16 random hexadecimal digits, and an address that keeps the operator's scheme, host and credentials and names the realm in place of any it named; an address whose dev_realm is not a realm name is refused, as any address that is not a YDB address is. Releasing the lease removes the realm with everything in it, and then ptah_dev when no realm is left in it; an object there is no statement to drop for stops the removal before anything is dropped. The provider also removes the realms it holds when its process ends. A realm is a directory, which YDB creates only for a user who holds ydb.granular.create_directory; a user refused the directory is refused the realm with that right named.

The session receipt reports the realm's path as its database, and inspection, locks, migration history, migration metadata and every native write resolve against it, so the realm reads as an empty database. Every SQL unit starts with PRAGMA TablePathPrefix naming the realm, which resolves a relative path inside it. The pragma does not confine an absolute path, a path that climbs out with .., a name written through a $ expression, or an object the whole database shares, so the provider reads each unit before it dispatches any of them and refuses the batch with ydb.realm-escape when a statement writes outside the realm, moves the prefix, runs code it computes, or creates a user, group, permission, resource pool or backup. An external data source or table, an async replication, a transfer and a streaming query are refused too. A statement the guard does not recognize is refused. The rules are the host's replay guard for a realm; internal/yql/testdata/realm holds the host's answers for 94 statements, and --!syntax_v1 is the one difference.

The provider also serves the replay guard itself (database.dev-replay-guard): the host asks, without a session, about each statement it is about to replay into a dev database, and the provider answers as the host's YDB guard does (internal/devclean/ydbreplay). A run that owns the whole server, because the operator declared it disposable or the run provisioned it, may change anything on it but what reaches past it: an external data source or table, an async replication, a transfer and a streaming query, and a translation setting, which changes how the rest of the text is read. Every other scope confines a statement to the dev database, with the rules above, and refuses --!syntax_v1 as the host does. testdata/realm/scopes.json holds the host's verdict on each statement under every scope.

Dev database cleanup

The provider lists and empties the dev database a session selected (database.dev-cleanup), as the host's embedded engine lists and resets one. The list and the cleanup read one walk of the session's root, each directory's entries in name order and a directory after its contents, so a database the list finds empty is one the cleanup leaves as it is. The walk leaves alone a name that starts with a dot, which belongs to the server, the lock node the session keeps its locks on, and at the root of a database the directory of the dev realms. A request that names schemas walks only those directories, and keeps them.

The cleanup drops every transfer and streaming query, then every async replication, then external tables and column tables, then external data sources, then the other objects, then secrets, and removes the directories, deepest first, with the host's statements. A coordination node is dropped through the coordination service. An object there is no statement for, such as a resource pool, refuses the cleanup before anything is dropped, and the reply names it. A cleanup that stops after its first step is UNKNOWN, and the host treats the database as not clean.

A session on a dev realm takes its locks on a ptah_locks node at the realm's root, and the cleanup of the realm keeps that node, where the host's embedded engine dropped a node there. That engine took a realm session's locks on the node at the database root instead (path.Join(driver.Name(), "ptah_locks") in its internal/dblockext/ydb; measured on 25.1.4.7, a realm run creates the root's node), so the realm's node was no lock of its, and it created the root's node from inside a realm, which the realm's guard refuses for every other write. The provider keeps a realm session inside its realm: its lock node is there, and the read and the cleanup leave it alone as they leave the database root's node alone.

internal/devclean/testdata holds scheme trees and what the host's embedded engine listed, dropped and removed for each, recorded by an oracle program; the tests replay each tree through the provider's walk and cleanup.

A session on the database itself leaves the ptah_dev directory out of everything it reads, and a table owner may not be named ptah_dev.

Dev databases

The provider advertises database.dev. A host request with the spec docker://ydb[/<tag>[/local]] starts ydbplatform/local-ydb:<tag> (latest without a tag) with in-memory disks, publishes its port on loopback only, waits until the server takes DDL, and returns a lease and the address ydb://127.0.0.1:<port>/local?go_balancer=disable. The host removes the container by releasing the lease; the provider also removes every lease it still holds when it receives SIGTERM or SIGHUP or when the host closes it. A provider killed outright leaves its containers, which carry the label ptah-dev for manual removal.

The spec follows the host's docker://ydb rules: the engine is matched as written, one path segment is a tag, and local-ydb serves the database local alone. The host also sends the image form the operator types, docker+ydb://[<host>/]<image>[:<tag>][/local], which starts the image it names and is read as the host reads the image form of its other engines: the last path segment is the database when there are two or more and it holds no colon, the segments before it are the image, a host other than _ leads the image, and an image with no tag is latest. The image has to serve the database local without a credential, as local-ydb does. Parameters are merged over the default go_balancer=disable. Those that would point the connection elsewhere or hand the container a credential (host, hostaddr, port, user, password, dbname, passfile, service, servicefile, database, dev_realm, monitoring, token, use_env_credentials) are refused, and so is any parameter a YDB address does not accept, before a container starts. The host starts other engines itself.

The provider drives the docker command line, which it finds on PATH. The host starts the provider with an explicit environment, so pass PATH, and DOCKER_HOST or DOCKER_CONTEXT when the daemon is not the default. local-ydb takes every connection without a credential, so a daemon on another machine is refused (UNSUPPORTED), and a missing command or an unreachable daemon is UNAVAILABLE.

Execution and transactions

Execution sends native SQL units through YDB's query service. A batch is checked before its first unit runs. Unsupported native envelopes refuse the whole batch. SQL units use a pinned connection; native coordination, table creation and table alteration units use their native services. Dispatched writes are not retried, except a changefeed statement YDB refused because another schema operation held the table's path (see Changefeeds). Request IDs correlate calls and do not provide exactly-once execution.

Results distinguish a confirmed prefix from an uncertain remainder. A dispatched unit YDB refused is NOT_APPLIED, or PARTIAL after a confirmed prefix, when nothing of it can have been applied. That takes a status that establishes the refusal (BAD_REQUEST, SCHEME_ERROR, PRECONDITION_FAILED, ALREADY_EXISTS, NOT_FOUND, UNSUPPORTED, ABORTED, or GENERIC_ERROR for a plain data statement, for one ANALYZE statement, which changes no schema object and no row, or for a text YDB refused while compiling it, which its issues show by naming a position in the text and no execution stage) and a unit that ran in a caller transaction, which YDB applies only by its commit, or that is one query YDB applies whole: a data query or one scheme statement. A native unit whose first statement YDB refused is judged by that statement: the service reads and checks before it writes, and writes only through its statements, so nothing of the unit was applied. That holds for every native unit but a column inventory, which alters a table through the table service first, a rebuild's copy, a table creation or alteration and a coordination node operation, which run no statement. Its message is YDB's own issue text on one line, which no host prints; its reason is the provider's (see Error reasons). Every other failed dispatched unit reports UNKNOWN, even when earlier units succeeded: a block or action call that runs scheme statements, a BATCH statement, a unit of several scheme statements, a native unit past its first statement, a unit that holds a coordination node statement, a unit that expands secret inputs, and a failure that carries no server status. A unit that holds a comment statement runs as its statements one after another, and reads by the first one when that one fails, since nothing of the unit ran before it: a query by the rules above, and a comment statement, one AlterTable YDB applies whole, by a status that establishes the refusal, such as BAD_REQUEST for attributes past YDB's budget. A later one failing leaves the unit UNKNOWN. Cancellation between units reports the confirmed prefix. Diagnostics omit raw driver errors.

A unit YDB refused in a caller transaction ends the transaction: measured on 26.2.1.14, YDB has dropped it, and a rollback answers Transaction not found. The provider rolls it back and retains ROLLED_BACK for the caller's finish. A commit YDB answered ABORTED applied nothing and is ROLLED_BACK.

A unit refused before dispatch reports NOT_STARTED, or PARTIAL after an applied prefix, with the diagnostic ydb.precondition-refused. Where the reason is the provider's own text about the object, such as a plan the object changed after, a path another object holds, or a read the server answered with a status other than success, the message is that reason, naming the object by its path and the server's status by its name. Any other reason, such as a lost lock or a failed system-view query, gets a fixed sentence, so no server text, credential or declared value reaches the host.

Every native creation reads where its object goes before the write: the listing of its directory, or the system view for a streaming query and the resource pools. YDB creates the directories an object needs with the object, so a directory that does not exist holds no occupant, by the rule a filtered read uses (see Schema observations), and the first object of a new directory is created there. On 26.2.1.14, YDB created a topic, a coordination node, a secret, an external data source and an external table, a row table (through YQL and through native CreateTable), a column table, a view and a streaming query two directory levels below an existing directory; 25.1.4.7 did the same for every kind but secrets and streaming queries, which it does not keep. Resource pools and classifiers have no directory.

The provider offers units.statement-timeout. A host that lists it sends a migration's statement timeout with each Execute call, and the provider bounds each unit with it and answers for what YDB does when it fires, as the host's embedded engine does (migration/migrator, ydb_timeouts.go); the reply says the timeout stopped the unit. A unit in the caller's transaction is canceled and the transaction never commits, so nothing of the unit is applied (NOT_APPLIED, or PARTIAL after a prefix), and the provider rolls the transaction back. An index added to a table that exists, and a column added with a default, run as a build that keeps running when the client stops waiting: the provider cancels the build through the operation service and waits for it to end, so a stopped build is not applied, and one that completed first counts as applied. Any other unit, and a build that cannot be found or does not end, may still be applied by the server, and its outcome is UNKNOWN. A host that does not list the feature bounds a statement with the deadline of the Execute call, and a unit that deadline stops is UNKNOWN and is not sent again. The session receipt reports migration_statement_timeout=true either way.

Transaction handles belong to one provider session. DEFAULT and SERIALIZABLE both use YDB's native serializable read/write mode; receipts retain the requested isolation. Begin cancellation remains connected while the database transaction opens, then detaches when the handle is returned. Ending the begin RPC therefore does not roll back later work.

A session permits one active transaction. Execution must name its handle while it is active. Commit and rollback retain their terminal result, including UNKNOWN and an aborted commit, without issuing a second database operation. Rollback of an unused handle fences a late begin. The SDK client sends rollback after a failed begin, including a duplicate begin; callers must not reuse handles. Each session retains at most 1,024 transaction handles.

Closing a session rolls back unfinished work even if the close caller was canceled. Connection acquisition for begin and native transaction calls have a 30-second bound. Unconfirmed cleanup remains an error. YDB schema changes are not transactional; session receipts explicitly report transactional_ddl=false. Live tests verify that DDL and SQL transaction-control statements cannot escape an explicit data transaction. See YDB transaction semantics.

Coordination, external object, row-table creation, row-table alteration, changefeed and topic operations use native envelopes, or YQL statements in the TEXT form; see Text units. Other operation models remain to be extracted. A SQL unit that runs one of the host's coordination node statements is run through the coordination service; the host's extended comment statements are not implemented here.

Migration splits

The provider and the ydb engine list units.migration-split (contract 25.12, ADR 0054 in Ptah) beside the sessions.transactions and sessions.history services it requires. SplitMigration reads one migration body offline and returns the queries YDB runs it as, cut by internal/yqlquery as the host's embedded YDB engine cuts them: each scheme statement and each BATCH statement a query of its own, each run of data statements one query, and every PRAGMA, DECLARE, named expression and action carried into each later query. A unit's sql is that query's text, and its source is the query's own statements as written, which the host hashes and records. The corpus under internal/yqlquery/testdata/corpus holds the host's text and source for each case, recorded by internal/rowdata/testdata/oracle.

A data query is WITH_CHECKPOINT: the host runs it in a serializable transaction together with the history write that records it. A scheme query, a BATCH query, and a block or action call that runs a scheme statement are OUTSIDE, and the host runs each with no transaction. A block or an EVALUATE FOR that runs only data statements is a data query: measured on 26.2.1.14, YDB runs and commits DO BEGIN UPSERT ...; END DO, DO $a(1) over a data action and EVALUATE FOR ... DO BEGIN UPSERT ...; END DO in an interactive serializable transaction, and refuses one that runs a scheme statement there.

The split refuses the whole body, with the byte range of the refused text and a reason that quotes nothing from the body, where the host's embedded engine refuses it before a run: a translation setting other than --!syntax_v1 (such as --!ansi_lexer), a client delimiter directive (DELIMITER or -- atlas:delimiter), a statement that controls a transaction (such as COMMIT), and a block or action call that runs a scheme statement and a statement that reads or writes a table, which YDB refuses whole. Each check assertion gets the verdict RunStatement gives the same text under READ_ONLY before it opens anything, so a dry run refuses what the run refuses. The run still proves the reads local against the catalog.

For a host that lists the feature, a unit YDB aborted for a conflict in a caller transaction (ABORTED, Transaction locks invalidated) is NOT_APPLIED with Execution.transaction_aborted, and a commit YDB aborted is ROLLED_BACK with TransactionResult.aborted. Neither applied anything, and the host may run the transaction again. A host that does not list the feature gets the same outcomes without the flags. An ABORTED answer YDB gives for the transaction itself, not for a conflict, sets neither flag, since every new attempt answers the same: measured on 25.1.4.7, a transaction whose writes do not fit one datashard program, such as the copy of a table rebuild over about 48 MiB, commits with issue 200509 (Datashard program size limit exceeded), and the commit is ROLLED_BACK with the reason that says so.

Text units

The provider lists render.text-units (contract 25.3, ADR 0034 in Ptah). A host that lists it asks for the TEXT form where it writes text: migration files, schema render, SQL diffs and SQL inspection. Every unit is then a YQL statement, the one the host's embedded engine writes for the same change, and this provider's Execute does with it what the native call does:

Operation	Statement
Row-table creation	CREATE TABLE with its columns, key, families, TTL, automatic partitioning, read replicas, key bloom filter and starting layout, then ALTER SEQUENCE ... RESTART WITH for a Serial whose sequence the declaration gives a start or an increment
Serial sequence change	ALTER SEQUENCE on the sequence's absolute path, with START WITH and INCREMENT BY and no restart
Row-table alteration	One ALTER TABLE: families and placements, added or dropped columns, SET (TTL = ...) or RESET (TTL), the whole automatic partitioning set, or ADD/DROP INDEX
Column table	The CREATE TABLE ... STORE = COLUMN and TTL statements the provider runs
Coordination node	CREATE COORDINATION NODE ... WITH (...), ALTER ... SET (...) naming the settings that change, DROP COORDINATION NODE
External data source or external table	CREATE, CREATE OR REPLACE or DROP EXTERNAL DATA SOURCE or EXTERNAL TABLE, with the paths and options as the declaration writes them
Topic	CREATE TOPIC with its consumers and the settings the declaration states, DROP TOPIC, or the ALTER TOPIC statements of an alteration, one to a line: the statements Execute runs natively for the change
Changefeed change	One statement to a line: ALTER TABLE ... DROP CHANGEFEED for each removal, then ALTER TABLE ... ADD CHANGEFEED for each addition with one ALTER TOPIC ... ADD CONSUMER per consumer, then the ALTER TOPIC statements that change a topic in place

YQL has no coordination node statement, so Execute reads one as the host's YDB connection does and runs it through the coordination service: alone in its query after nothing but definitions, with the query's PRAGMA TablePathPrefix, and inside a dev realm relative to the realm. A CREATE checks first that no node is at the path, because the service answers a second creation with success and keeps the node as it is; an ALTER merges its settings with the node's and refuses a result YDB would move into its range. Such a statement in a transaction, or one that cannot be read, refuses the batch before any unit runs.

A change only the native call makes refuses the batch with NO_TEXT_FORM and no unit: user attributes other than Ptah's comments, storage settings, tiering, the partition columns, a family kept in memory, a default of a type without a literal, a sequence on a column that is no Int16, Int32 or Int64, a start or an increment on a 16-bit or 32-bit Serial's sequence, a sequence of a creation or a change that names no database (ALTER SEQUENCE takes only the absolute path), a TTL run interval other than YDB's own, a coordination node that declares its access, an external data source or external table dropped and created again around the plan's other statements, which the native call gives the owner and the access entries of the old one, and every topic operation. A host that does not list the feature gets native units.

Reverse migrations

The provider lists models.reverse (contract 25.19, ADR 0062 in Ptah) and declares one reversal service, ydb-objects, for the changes of its standalone objects: coordination nodes, secrets, topics, streaming queries, external data sources and external tables, resource pools and classifiers, async replications and transfers. ptah migrations generate writes the down file of such a change from this service's reverse of it, as the host's embedded engine writes it from its own:

• The reverse asks for the declaration of what the forward change found, the conversion of the read, and finds what the forward change leaves, the conversion of its declaration. A creation reverses to a drop, a drop to a creation and a change to the settings the object held. The prediction of what the forward change leaves is the forward state the host plans the reverse against; it is never read from the server.
• The down file holds the reverse's statements, in the exact statement style the host plans in, and a -- Recovery limit: line for what the rollback cannot undo, in the embedded engine's words: a dropped topic loses its messages and its consumers' positions, a dropped coordination node its semaphores, a restored workload setting cannot undo the queries run under the forward one.
• A rotated secret has nothing to run in reverse: the value it replaced was never read. A secret created again takes the value of the variable for its path when the rollback runs.
• A changed streaming query text keeps allow_state_reset in the reverse, which resets the query's checkpoint again.
• The reverse asks for the nearest state YDB reaches in place, and the down file says what it keeps. A topic keeps the partitions the forward change added, and keeps auto-partitioning, paused, once the change turned it on. An async replication or a transfer keeps a credential the change added. A replication failed over before the forward change dropped it is not created again, since its tables stayed as ordinary tables. A reverse that leaves nothing YDB can run has no statement, only its recovery limit.
• What the forward change leaves of a replication or a transfer runs in the state the change found it in, so the reverse of a paused replication's connection change is an ALTER that YDB takes.
• A change on a target without the capability its forward change needed, such as secrets or external_data_sources, is not reversed: the reversal refuses it, as the host's embedded engine did.

The service also undoes a change of a table's own settings: a row table's properties, such as its TTL and its partitioning, and a column table's storage. What the forward change leaves is the description the table held with the declared settings in place of its own; a setting the declaration leaves unset stays as the table held it, except the TTL, which a table the declaration gives none loses. A column's move to another family reverses to a move back: what the forward change leaves is the column in the declared family, with a default or storage setting the declaration does not state kept as the column held it. A host that does not project a reverse's state onto a column cannot plan that reverse and refuses it. A change of a global index's partitioning and read replicas reverses to every setting the index held, as the host's embedded engine restores them (restore every setting the index held in place), read replicas the forward change removed included: the declaration keeps every setting it leaves out, so a reverse that stated only some would keep the rest as the forward change left them. What the forward change leaves is the index's own facet with the declared settings in place of the ones it held. YDB removes no maximum partition count, so a maximum the forward change gives an index that held none stays, and the down file says so in the embedded engine's words. The host plans such a reverse with the declaration the forward change started from; the planner gives the index what the reverse change asks for, against the facet the reverse predicts, rather than what that declaration states. A host that does not list the feature gets no reversal service and refuses every such change, naming the change kind that has no reversal.

A topic reverses like any other object, and a migration file holds its statements (see Text units).

Common objects

The provider lists schema.common-objects (contract 25.3, ADR 0033 in Ptah), and the ydb engine declares views, comments, roles and grants. A host that lists the feature compares and orders these objects in its shared code, as it does for every engine; the provider reads them, spells a declaration the way YDB does, and writes each change as the host's embedded engine writes it. A host that does not list it gets none of this: views stay native objects, and a declared comment is refused as a table setting no creation states.

The comments of a table the host creates, of its columns and of its indexes, and of a column or an index it adds to a table, reach the provider as comment steps after the step that creates the object, as for any comment change. The provider takes them off the declaration it creates the table from, and runs the steps after its own steps on that table, with the exceptions below.

Views and comments use these shared stages:

Step	Views	Comments
Inspect	Each view, its query as the server stores it; on a dev realm without the realm's PRAGMA TablePathPrefix, which YDB stores at the head of a view the session created	The comment of each row table, of its columns and indexes, and of each view, from the user attributes YDB keeps them in
PrepareCommonObjects	The query in the server's form: its tokens joined by single spaces, comments and terminators left out	As declared
ValidateSchema	Refuses a view without a query, WITH CHECK OPTION, view attributes and materialized views	Refuses what an attribute cannot hold: a key over 100 bytes, a value over 4096 bytes, text that is not UTF-8, more than 10240 bytes on one table
Render	CREATE VIEW ... WITH (security_invoker = TRUE) AS, the query and ; on lines of their own, then the view's comment; DROP VIEW	Ptah's COMMENT ON TABLE, COLUMN, INDEX ... ON and VIEW, IS NULL to remove one
PlanFeatures	Dropped before every table step and created after each one that does not only remove	After every step on the table that keeps it, except one that only removes; a comment on the table or a column also skips a step that changes only an index

Under schema.view-query-equivalence (contract 25.33, ADR 0108 in Ptah) the host asks the provider whether the query a declared view states and the query the held view keeps are one query, and compares no YDB view query by its own reading, which folds case. The provider reads both into the server's form and compares them as they stand, as the host's embedded engine did: YDB names keep case, so a query that changes only the case of a name, of a keyword or inside a string is another query, and the plan replaces the view; a comment, a line break or a terminator the server drops changes nothing.

The server's form of a query was measured on 25.1.4.7 and 26.2.1.14 alike: a two- or three-character operator is one token except >>, which YDB writes > >; $x is $ x; a number keeps its exponent and its suffix; a quoted string or name, a multi-line string among them, is kept whole. A read of a view applied once therefore equals the preparation of its declaration.

A step that only removes, such as a column's removal, is ordered after the views and comments of the plan, not before them: the host places a removal after its views and comments, and what a removal drops is no longer declared, so no view the plan creates reads it and no comment the plan sets names it. The one comment step that names a dropped column removes its comment: YDB keeps the attribute once the column is gone and would give it to a column added later under that name. The host sends that step for each dropped column that held a comment (stokaro/ptah#5280), in its comment window, so it runs before the removal.

A comment on a table or on one of its columns is not ordered after a step that changes only an index of the table, such as an index's addition. The host places the comment of a column it adds right after the column, before the index additions, and the index step changes nothing such a comment names. A comment on an index still follows the steps on its table.

YQL has no comment statement. Execute reads Ptah's COMMENT ON as the host's YDB connection does and sets the one attribute it names through the table service, after checking that the object is in the session's database or realm, is a row table or a view of the kind the statement names, and for a non-empty comment has the column or the index. A statement whose path holds nothing, or one the session may not see, is refused before any write with a message that names the path. A unit holding a comment statement beside other statements, such as a commented view, runs as the queries a migration file is split into, one statement after another. A comment statement in a transaction, or one that cannot be read, refuses the batch before any unit runs. A comment is never a table property: the table facet leaves Ptah's comment keys out, a declared attribute under one of them is refused, and the new table of a rebuild takes the comments the read reported, which the host carries on the observed table, its columns and its indexes of a planning request.

Users and groups share YDB's principal namespace. Creation accepts lower-case ASCII letters and digits. Groups have no login or password settings. Membership can name a cluster-managed group, but cannot declare self-membership or make a principal a member of a declared user. Inspection preserves every observed membership, including edges whose principals are outside the managed role list.

A read leaves out of the description the users and groups the cluster made for itself, the database's owner, and every user and group of a dev realm, and names them as roles the server holds outside its scope. Under schema.role-scope-reasons (contract 25.26, ADR 0076) the read also says why, and the host's note gives the reason the host's embedded engine gave: because the cluster made them or they own the database, or the read was of a dev realm, which shares its database's users and groups.

A user's password is set when the user is created, from a YAML, Go or YQL declaration, and is never read back: a declaration that names another password plans nothing, as the host's embedded engine plans it. A YQL document's password reaches the host under source.sql-role-passwords (see SQL schema reading).

The password never enters a plan. Under schema.password-inputs (contract 25.22, ADR 0071) the host sends a declared password, or a password hash, as an execution input, and the statement names the input where the password stands: CREATE USER `app` PASSWORD $PTAH_SECRET_PASSWORD_APP;. YDB refuses that statement as written. DescribeExecutionInputs reports the input, and the provider writes the bound value in as a string literal only when it runs the statement, with HASH for a password hash, and keeps it out of any error the server answers with. The host binds the declared password when it plans from the schema and applies in one run, and the variable's value for a saved plan or a migration file. A host that does not negotiate the feature sends the password itself, and the provider refuses the user rather than write the password into the plan.

schema.common-access (contract 25.5) carries independently covered owners and memberships. An owner's login state is optional: unknown remains unknown, including when the server refuses its principal views. Principal queries read names and login settings without reading passwords or hashes. The host refuses security analysis with incomplete access knowledge. A host that does not negotiate this feature receives uninspected role and grant coverage.

Permissions use canonical YDB names after preparation. Column grants, WITH GRANT OPTION, PUBLIC, and permissions YDB does not recognize are refused. Database grants name the opened database. On older release lines a root-level grant also needs the captured absolute database path; rendering refuses an unbound path. Native grant targets retain the provider's naming, including literal dots in resource-pool names.

Inspection marks as implicit the entries YDB gives an object when it creates it, which no GRANT made, as the host's embedded engine marks them: the entries of the default resource pool that name ydb.granular.select_row and ydb.granular.describe_schema together, and the ydb.granular.describe_schema entry a new secret gets for every subject whose effective permissions on the secret's directory carry that permission.

Mixed plans check captured native state before changing its access. A native alteration precedes grants, revocations, and principal changes. A grant or a revocation on a directory changes the directory's access list and occupies no path, so the native planners leave it out of their path checks; a native creation in that directory, or below it, comes first, because creating the object creates a missing directory and YDB grants nothing on a path that holds nothing. A final native drop accounts for matching revocations by removing the object's access; a temporary external-table drop keeps revocations after restoration. These plans require the negotiated planning.lifecycle feature. Workload operations retain their routing order after principal creation and before principal removal.

Standalone sequences are unsupported. A sequence declaration is refused before rendering.

Migration history

The history service owns its physical YDB storage. Native revision keys are signed 64-bit integers; Atlas revision keys retain exact strings, including an empty identity. Each row contains the key and a deterministic binary SDK revision record. The record retains nanosecond timestamps and durations, direction, progress, diagnostics, checksums, Atlas type bits, and ordered partial hashes.

Creation installs the table layout and a ptah.history ownership attribute in one native request. The marker is stokaro/ptah-ydb-plugin/history-v1/native or stokaro/ptah-ydb-plugin/history-v1/atlas. This is a storage format version, not a stable SDK release. Reads and writes verify the marker, column types, primary key, and absence of automatic behavior such as TTL or changefeeds. Unknown fields, malformed records, altered layouts, and unmarked tables are refused, with one exception: the revision tables Ptah's built-in YDB engine wrote.

That engine kept a database's migrations in tables of its own layout, with no attribute: the native schema_migrations (version, description, applied_at, state, applied, total, error, error_stmt, execution_time_ms, checksum), and the Atlas-compatible atlas_schema_revisions that ptah-compat migrate apply writes. A revision table with no attribute and exactly one of those layouts is read as history, and every revision written to it keeps that layout, so a dirty migration the engine left stays dirty with its progress, a resume continues it, and the engine still reads the table: a down run is suffixed :down in state as the engine wrote it, an empty error is NULL in the native table and an empty string in the Atlas one, and no partial hashes are the JSON null. Those layouts keep the time to the microsecond, the native run time to the millisecond, and no operator version or partial hashes in the native table, so a revision written there holds no finer part, and a native record with one is refused. Under the engine's default names such a table is also left out of a schema read, as the engine left it out, so a plan for its directory does not drop it. A table that is neither the provider's nor one of those layouts is still refused. The provider's own Atlas format preserves SDK revision semantics; it does not reproduce the Atlas SQL column layout or promise interchange with external Atlas tools.

Mutations compare the complete expected record before inserting, replacing, or deleting it. The comparison and write share one native serializable transaction. A caller transaction binds history and application writes to the same commit or rollback. Otherwise the provider opens and finishes a private transaction. A result inside a caller transaction remains provisional until commit succeeds. Conflicts cannot overwrite a committed winner; uncertain outcomes never authorize an automatic retry. Request IDs are correlation identifiers.

Reads distinguish a missing store from an empty store. Namespace traversal is refused. A failed native listing is an error rather than evidence of absence. Snapshots are bounded by the caller's revision limit and a 3 MiB size limit; exceeding either returns no partial history. An unreadable history snapshot or failed dispatched history mutation makes its caller transaction rollback-only. History storage alone does not establish complete migration support. Planning and complete host integration remain pending.

Migration journals and tags

The metadata service stores exact SDK records. A journal key combines the run ID and signed sequence number; duplicate identities are refused even when the payload is identical. A tag key preserves its exact name. Versions remain signed 64-bit values, including zero, and timestamps retain nanoseconds. The host owns event meaning, attempt pairing, tag normalization, and migration recovery.

Creation installs the native layout and the ptah.metadata ownership attribute in one request. Its value is stokaro/ptah-ydb-plugin/metadata-v1/log for journals or stokaro/ptah-ydb-plugin/metadata-v1/tags for tags. Preparation and operations refuse unmarked tables, unknown markers, altered columns or keys, indexes, changefeeds, and automatic deletion settings. Reads also refuse unknown, noncanonical, corrupt, or mismatched record identities. The log and tag tables Ptah's built-in YDB engine wrote, schema_migrations_log (run_id, seq, operation, version, state, actor, actor_source, checksum, logged_at, error) and ptah_migration_tags (tag, version, recorded_at), are read and written in that layout, with no attribute, as its revision tables are; the time is kept to the microsecond, and an empty actor, checksum or error is NULL. Under those default names they are left out of a schema read too.

Log writes use a separate serializable transaction and remain durable when an execution transaction rolls back. Log scopes cannot name that execution transaction. Tag mutations use the selected execution handle, or a private serializable transaction when no handle is active. Preparation inside a transaction only validates an existing store; it never creates schema. Tag replacement and deletion validate any existing record before changing it. Deletion reports existence from an atomic read, not from affected-row counts. A missing tag is distinct from a tag at version zero.

Each dispatched mutation runs once. A lost write or commit acknowledgment is UNKNOWN, without automatic replay. A storage read or mutation error on the selected SQL transaction makes it rollback-only. Lock loss refuses further journal and tag writes, including durable log writes outside an execution transaction.

Opening a reader captures a complete immutable snapshot in one serializable transaction. A count and ordered stream must agree before any page is exposed. The snapshot uses a private temporary file, so memory is bounded by a record and an RPC page rather than journal length. Unix unlinks the file immediately; Windows uses native delete-on-close. Closing the handle or terminating the provider releases the file on both platforms.

Pages contain at most 256 records and 3 MiB including their envelope. Oversized records and incomplete reads are refused, never truncated. Snapshot handles are single-use and bind the exact session, transaction, kind, and store. Page indices advance once; a failed or uncertain page must be closed. Closing an unused ID fences a late open. Each session retains at most 1,024 reader IDs and allows at most 16 unfinished snapshots. Completion and close release snapshot files; transaction finish and session close release their remaining readers.

The live tests cover journal durability, tag commit and rollback, same-transaction snapshot immutability, journals larger than one RPC, storage reservations, corruption, and reader fencing. Metadata unit tests also run on Windows to exercise native file cleanup.

Coordination locks

The lock service uses exclusive ephemeral semaphores on the database-root ptah_locks coordination node. Each acquisition has its own native session. The provider creates a missing node with strict read and attach consistency. Ptah's built-in YDB engine creates the node with neither setting, which YDB reads as relaxed reads, so every database that engine locked holds such a node: the provider alters a node that sets neither mode to strict reads and attachment before it takes a lock there, as a run of that engine took its locks there, and a session of that engine that still holds a lock keeps it. A user who may use the node but not alter it is refused with the right to grant. Any other node must have both settings explicitly strict; the provider refuses one someone set otherwise without changing it. The refusal is a NOT_ACQUIRED receipt that holds nothing, and its reason names the node and its two settings.

YDB's CLI (2.25.0 and 2.29.0, bundled with 25.1.4.7 and 26.2.1.14) and YQL have no command for a coordination node, so making such a node strict goes through the coordination service. With ydb-go-sdk, alter the node in place:

err := driver.Coordination().AlterNode(ctx, path.Join(driver.Name(), "ptah_locks"), coordination.NodeConfig{
	ReadConsistencyMode:   coordination.ConsistencyModeStrict,
	AttachConsistencyMode: coordination.ConsistencyModeStrict,
})

DropNode on the same path removes the node instead, and the next lock creates it strict. Remove it only while no run holds a lock there: dropping the node ends every lock session on it. YDB's default read mode can return stale state during leader changes; see the YDB coordination documentation.

The provider finds the node by describing its path rather than by listing the database root, so a user granted the node takes the lock without the right to list the root. A server that refuses the user the lock is answered with a NOT_ACQUIRED receipt, and its reason names the step and the right: the node is missing and the user may not create it, the user may not use the node, or the server refused the session on the node. The first two are the host's embedded engine's words. For the third that engine reports a timeout, because its SDK retries a refused session without saying why, while the server answers at once with UNAUTHORIZED. The receipt confirms that nothing was acquired, so the host neither fences the session nor sends a cleanup release, and the session takes the lock once the user is granted the node. A host that lists errors.reasons prints the reason after the outcome; see Error reasons.

A TryLock that takes no lock says why in its receipt: another session holds the lock, the session has an active transaction, it lost a lock guarantee, it already holds 16 locks, the server refused this user the lock, or the node is not strict.

Acquisition is nonblocking. Only a confirmed BUSY permits the host to try a new handle under its wait policy. Handles are single-use and bound to the exact session and name. Release of an unused handle fences a late acquisition. A session retains at most 1,024 handles and holds at most 16 active locks. Acquisition during an execution transaction is refused. Read-only lock previews are not represented by this SDK revision.

A native bidirectional stream sends each acquisition once and checks native operation status and request identity. It does not reconnect or reacquire after a transport failure. Keepalives run every two seconds; the native session timeout is ten seconds. Ownership checks run every second and verify the strict node configuration, exact session owner, and exclusive ephemeral semaphore count. A confirmation expires after five seconds. Both monotonic and wall-clock elapsed time are checked, including before refreshing a confirmation after a pause.

Every SQL unit, native coordination operation, history or metadata mutation, and transaction commit verifies guarding locks. Storage creation checks the lease immediately before native dispatch too. Loss or uncertainty cancels guarded calls and permanently fences further writes in that provider session. Rollback, release, and close remain available. A new owner does not repair the old session. Failure after dispatch may have effects; an acknowledgment racing ownership loss is reported as uncertain.

Release requires successful semaphore-release and session-stop acknowledgments. Local stream shutdown is insufficient. Cleanup continues with its own timeout after caller cancellation and retains uncertainty on repetition. Provider death leaves the server to expire the native lease. The coordination node remains for later runs; its ephemeral semaphores disappear after their owners leave.

These are advisory leases, not transactional fencing tokens for SQL or DDL. Fresh checks and cancellation cannot undo a statement already executed or make nontransactional DDL atomic with ownership. Live tests cover competing provider processes, independent names, immediate takeover after confirmed close, process exit, node removal, and refusal of execution/history/commit after loss. Controlled clock tests cover stale confirmations without relying on a scheduled watcher.

Error reasons

The provider lists errors.reasons, so a host that lists it too prints why a call failed after its gRPC code:

provider compare models failed (InvalidArgument): streaming query capability was not established

A reason is a sentence the provider wrote. It names the subject, a capability or a parameter, and never quotes statement text, a value, an address or a credential. Package internal/authored keeps such sentences apart from the errors the provider receives: authored.New marks one, and authored.Status sends it as the reason, with the error's whole text as the status message, which no host prints. The reasons the provider gives are its refusals of a connection address, of a monitoring page it could not read, of a capability the target lacks, a declared external object among them, of a lock the server refuses this user, of a streaming query body changed without allow_state_reset, the refusals whose text is a fixed sentence, such as a closed session or a reached handle limit, and the refusal of a table's declaration by table preparation or the creation projection. That last reason names the table and gives the sentence schema validation reports for the same declaration; the compilation behind it reads no server. A YDB driver's error is never a reason, because YDB can quote a statement and its values, or a credential source: the host prints its code alone.

A declaration a source decoder refuses gives a reason in the words of Ptah's built-in YDB engine: a topic, a changefeed, an external data source or table, a resource pool or classifier, a streaming query, an async replication or a transfer, by its YAML key or its name, the attribute, and what the attribute takes. authored.Declared writes it from the decoder's words around the refusal and the refusal itself without the value it quotes. A refusal another package wrote, such as the YQL splitter's, gives none. A Go directive that leaves out a required attribute is refused as the host's annotation parser refused it, and the host names the directive's place after the reason:

provider decode source failed (InvalidArgument): topic "events": invalid retention_period: takes an ISO 8601 duration such as PT12H or P1D
decode Go directives of engine "ydb": provider decode source failed (InvalidArgument): missing required annotation attribute "rank" on //ptah:schema:resourcepool:classifier at Refused

A failed execution and a commit that did not apply give a reason too, in Execution.diagnostic and TransactionResult.diagnostic (contract 25.18), and a host of that contract prints it after the outcome:

provider execution: NOT_APPLIED (0 units confirmed applied): YDB refused the statement while compiling it (GENERIC_ERROR, line 1, column 27)
provider execution: NOT_APPLIED (0 units confirmed applied): YDB refused the statement: it violates a constraint of the table, such as a key the table holds (PRECONDITION_FAILED, issue 2012)

The reason of a unit YDB answered says what YDB did with it (refused it, refused it while compiling it, aborted its transaction, or answered without showing whether it applied), followed by YDB's status, the most specific issue code and the position YDB named, counted in the unit's own text: on a dev realm, the realm's pragma YDB received first is not counted. For the issue codes 2003, 2012, 2031 and 2032 it also says what the statement did, in the provider's words. A reason never repeats YDB's issue text as a whole: measured on 25.1.4.7 and 26.2.1.14, that text quotes the token YDB refused, a string literal of the statement and the table's path, and a statement can carry a password or a bound secret. The issue text stays in the result's message. A reason carries the parts of it that quote none of those, as the host's embedded engine printed them: the paths of the objects a refused DROP leaves in place, YDB's fixed sentences, and the check a rebuild's copy fails with, which the provider wrote into the statement and which names the table and the column:

provider execution: NOT_APPLIED (0 units confirmed applied): YDB refused the statement: rebuilding table app.items: column n holds a value that does not convert to its new type (PRECONDITION_FAILED)

YDB's fixed sentences follow saying. A name YDB does not take in a path, such as a secret whose name holds a space, is refused on 26.2.1.14 with an issue that quotes the path and the name; the reason carries YDB's sentence without them:

provider execution: NOT_APPLIED (0 units confirmed applied): YDB refused the statement, saying "symbol ' ' is not allowed in the path part" (SCHEME_ERROR, issue 2019)

INTERNAL_ERROR shows nothing about a statement, with one measured exception: 26.2.1.14 fails an UPSERT on a table with a unique index that names no column a synchronous index is keyed on while it plans the query (verification=!hasUniqIndex || !usedIndexes.empty() in kqp_opt_phy_upsert_index.cpp), and writes nothing; 25.1.4.7 runs the same statement. Such a data statement is refused rather than unknown: a unit is NOT_APPLIED with the reason that YDB cannot plan the UPSERT and that naming every column of the table, or UPDATE or INSERT, avoids it, and a statement the host runs for an operator, such as a check or a test assertion, FAILED with YDB's issues. A unit the provider refused before YDB answered, a statement timeout, a lost lock and a commit YDB did not confirm each give a fixed sentence.

Where YDB's issues say a feature flag is off, the reason names the capability and the flag, as the host's embedded engine words such a refusal, around the provider's own reason where the host's engine quotes YDB's text, then what to do. The provider's reason carries YDB's sentence for the flag, without what YDB wrote around it. A session whose address names no monitoring endpoint took the capability from its release line's default, and is told to name the endpoint as well as to turn the flag on; one whose address names it read the flags already, and is told only to turn the flag on. The texts and the flags they name are the host's measured ones (internal/ydbflags in Ptah). Measured on 25.1.4.7 and 26.2.1.14, where EnableResourcePools is off by default, without the endpoint:

provider execution: NOT_APPLIED (0 units confirmed applied): capability resource_pools is off on this YDB cluster, which runs with feature flag EnableResourcePools off: YDB refused the statement, saying "Resource pools are disabled" (UNSUPPORTED, issue 2030). Turn the flag on, or name the cluster's monitoring endpoint in the URL (monitoring=http://host:8765) so Ptah reads the flags before it plans

and with it, the same sentence up to Turn the flag on.

YDB refuses a DROP of an object other objects depend on, such as an external data source an external table reads, with SCHEME_ERROR before it changes anything, and names the dependents inside issue 2019. A reason repeats their paths, as Ptah's built-in YDB engine printed them, because the operator has to drop them first. Measured on 25.1.4.7 and 26.2.1.14:

provider execution: NOT_APPLIED (0 units confirmed applied): YDB refused the statement: other scheme objects depend on what it drops, /local/readers/tbl; drop them first (SCHEME_ERROR, issue 2019)

25.1.4.7 has no unique secondary index, and refuses one with Unknown index type: syncGlobalUnique inside its execution stage, before it proposes a schema operation. The unit is NOT_APPLIED (PARTIAL after an applied prefix), with the reason YDB refused the statement, saying "Unknown index type: syncGlobalUnique": the YDB release the cluster runs has no unique secondary index (GENERIC_ERROR, issue 1060, line 1, column N).

Operational storage reservations

Inspection recognizes history and metadata storage from exact ownership attributes and complete native layout, before extracting indexes or changefeeds. A marked table with an altered layout refuses the entire inspection. An unknown marker version is also refused. An unmarked table remains managed even when its name or layout matches a history table.

Confirmed history, journal, and tag stores are returned as SDK reservations instead of managed tables. A reservation occupies the name for every supported native scheme-object kind and for common tables. Exact UNREPRESENTABLE subject coverage overrides directory completeness at these occupied names, so omission does not assert absence. A comparison leaves a reserved name out. When a plan changes an object a streaming query could read, it counts no query at a reserved name, even where the desired source claims nothing there, as an HCL document without a coverage header does. Filtered reads return reservations only from the exact directories they inspect. Names or layout alone never establish ownership.

The coordination node ptah_locks at the root of the session's database is reserved with purpose LOCK by its name alone. The lock service takes whatever node it finds at that path, after checking its consistency settings, so the name is the reservation; YDB returns SUCCESS for a repeated CreateNode, so no creation receipt could prove more. A read reports the name as taken and describes no node there, so db read and an export leave it out, as the host's embedded engine does. A reservation never changes or drops the node. Every other coordination node, one named ptah_locks in a directory included, is a managed observation.

Operational storage reservations support host collision and reference guards. They do not establish native directory-cascade safety; containment qualification remains pending. Planning orders an object created below a path after the drop of the object there, and refuses any other overlap of directory lifecycles. Markers are storage-format identifiers, not an access-control mechanism against a database administrator.

Schema observations

Inspection reads the native scheme, table, topic, view, and coordination APIs. Common columns preserve YDB type spelling, Decimal precision/scale, nullability, and key membership, and with schema.observed-primary-keys each table's key in key order, so a Go export writes primary_key="b,a" for a table keyed in an order other than its columns'. Provider-owned table properties retain key order, TTL, column families, partitioning, storage settings, attributes, and access metadata. Column properties retain family assignment and typed literal or sequence defaults. The observations carry their complete native field definitions and digests. Normalization works offline after the database session closes.

Schema selectors name exact directories relative to the selected database. Use / for its root. An unfiltered read walks subdirectories. A directory whose name starts with a dot is not read, as the host's embedded engine read none: .sys, .metadata and the other directories YDB keeps for itself, and a user's directory such as .hidden. The read claims such a directory, and a selected path below one, as empty, so a declared object there is created, and nothing there is dropped. Measured on 26.2.1.14, that engine also plans the creation again on the next run, which YDB refuses (path exist); through the provider the next run is refused before the unit is sent. A table whose own name begins with a dot is read like any other. Traversal and snapshot size are bounded. A database nested in the selected one, such as /local/db2 in /local, is another database, and the read leaves it out as the host's embedded engine did: YDB lists it as an entry of type DATABASE, and an unfiltered read neither descends into it nor reads what it holds. A selector that names a nested database, or a path below one, reads as an empty directory, as that engine read it. The read claims the nested database's path as holding nothing of the selected database and says why, and a plan that creates a table, a view or a native object there is refused with a reason that names the nested database. Here the provider is stricter than that engine on purpose: that engine planned the CREATE TABLE, YDB ran it through the outer session into the nested database, and every later run planned it again and failed (path exist). A native write whose directory is at or below a nested database is also refused before anything is sent. Coverage names inspected subjects and records native-object completeness for each directory whose listing and child reads finished. These regions bind the exact database, empty catalog, and exact schema directory; an unfiltered read sends subtree regions instead where the host negotiates them (see below). An empty directory therefore establishes native-object absence there. Unread descendants, excluded system directories, and other namespaces remain unknown. The table and column facet kinds (a row table's settings, indexes and changefeeds, a column table's storage, a column's properties) get the same complete region for each such directory, since the read describes every table there with each facet its kind of table has. Without it those kinds stay narrowed in an export the read otherwise knows completely, and a Go or HCL source has no not-described entry for a facet kind, so ptah introspect of such a read, of the whole database or of a directory beside the database's pools, is refused before it writes (provider encode source failed (Internal)).

A selected directory YDB does not show (it answers SCHEME_ERROR) reads as empty when its parent's listing, proved the same way up to the database root, does not name it. That is the evidence a walk from the root takes for any entry a listing leaves out, and it lets schema apply --schemas create the first object of a new directory. A directory its parent names but the session cannot list is still refused, and so is a path below an object that is not a directory. Execution reads the directory of each object it creates through the same rule; see Execution and transactions.

A selected path that an object holds reads as an empty directory, with nothing below it: YDB needs every path above an object to be a directory. The object itself is read with the directory that lists it. So schema apply --schemas app,app/holder reads the table app/holder in app, and a secret declared in app/holder is planned after the table's drop.

A read with no selector claims more than the directories it lists. With a host that lists schema.subtree-coverage (contract 25.22, ADR 0064 in Ptah), the provider declares / as its schema separator, and the read sends one complete region over the root and every directory below it. The directories the walk does not list, a dot-prefixed one and at the database root the dev realm directory, are covered by it as holding nothing, as the host's embedded engine read them, so an export of the whole database claims every native kind and writes no ptah:schema:notdescribed entry for them. A native object in a directory that does not exist yet is then absent, and schema apply or migrations generate with no --schemas creates it, as the host's embedded engine does. A filtered read keeps one exact region per directory, since a selector names one directory. With an older host, the read claims each directory it lists alone, and a native object in a new directory is left unplanned with a warning, as for a topic in fresh/nested:

Warning: github.com/stokaro/ptah-ydb-plugin/topic fresh/nested.events could not be compared between the database and the desired schema: The current topic or its absence was not established.
No changes planned, but 1 comparison limit remains unresolved.

Name the directory with --schemas to create the object.

Unknown native fields and enum values are refused before JSON conversion, except where the host's embedded engine recorded the object and read the rest: a coordination node whose description holds a mode or a field the provider does not model is reported unrepresentable, and so is a resource pool or a classifier holding a setting outside what YDB keeps. A column store and a sequence, which the provider does not model, are left out and named in unread_tables where the host lists schema.unread-tables. A queue of the older persistent queue kind (PERS_QUEUE_GROUP) is reported uninspected, so a plan neither keeps nor drops it, and a system view outside a dot-directory is the server's and is left out. Any other kind of scheme entry refuses inspection, naming the object and its kind. Shared topic consumers refuse inspection. A table an async replication writes is left out of the tables, and the replication's items name it (see Async replications and transfers). Right after CREATE ASYNC REPLICATION, YDB can list the replica while DescribeTable still answers SCHEME_ERROR. So a row table a listing names and the table service answers SCHEME_ERROR for is described again for up to 10 seconds, and the directory is listed again before each wait. The topic of a changefeed is waited for the same way, with its table described again before each wait: the changefeed the replication adds to its source table can be named before DescribeTopic describes its topic. The read fails if the new listing or description stops naming the object, if it fails, or if the object is still not described when the time runs out; it never reads such an object as absent. A failed traversal returns no snapshot. These limitations preserve data for the remaining extraction work.

Observed property models use YDB's typed protocol definitions inside this provider. The neutral SDK carries their envelopes. Common fields and runtime statistics are excluded from property payloads; they cannot become a second writable representation. Index observations preserve sync, async, unique, vector, prefixed vector, and fulltext variants. The provider reads every kind-specific implementation table, including its columns, keys, and partition settings; parent table metadata alone does not expose those settings. Missing implementation tables or non-ready indexes refuse inspection. Internal sequence configuration is retained; its mutable position is excluded after unknown-field validation. An empty index facet records the absence of indexes on that inspected table.

Changefeed observations combine the table API description with a topic API read for each child stream. They retain modes, formats, state, retention, consumers, access metadata, and attributes, including replication bindings. Initial scan presence is retained after completion; progress counters are excluded. A changefeed in STATE_INITIAL_SCAN asked for a scan even before YDB reports its first progress, as for the changefeed CREATE ASYNC REPLICATION adds to a source table. An empty changefeed facet records absence on its inspected table. Standalone topics use the same reader and normalization rules, with their own object identities.

Topic observations preserve partition settings and quotas, retention, codecs, consumer options, and attributes. Live partition topology and topic/consumer statistics are excluded. A missing child description refuses the entire read.

Indexes, implementation tables, changefeeds, and the consumers of a changefeed's topic are read in name order; a topic's own consumers keep their order (see the topic section). Column and key arrays retain their observed order. Desired coordination-node declarations are available as described below. Every observed model has a separate desired contract and an offline conversion. Comparison and planning remain limited to coordination nodes, the TTL, partitioning settings and column families of row tables, and the storage of column tables; source adapters for the other families remain pending, as described under row-table declarations below.

Views retain the query returned by the view service and the attributes read through the table service on the same path. This includes comments and custom attributes. Coordination nodes retain native consistency modes, timing, and counter configuration; the envelope owns the node path. External data sources retain source type, location, authentication references, and native properties. External tables retain source references, location, ordered typed columns, and content options without rewriting their native keys or JSON values.

Object counts

For a host that lists models.report-counts, the provider reports count metrics for its topic, changefeed, coordination node, streaming query, external data source, external table, async replication, transfer, secret, resource pool and resource pool classifier models, and for its row-table properties, column storage, index set and index settings models, desired and observed, under the metric names and help texts of the host's built-in YDB engine. It counts each value offline: a topic is one topic and its consumers, a table's changefeed set its feeds and the consumers across them, an external table one table and its columns, and every other object one. ptah schema stats then writes these lines through the provider as through the built-in engine.

The table setting families count as the built-in engine counted them:

Metric	Counts
ydb_ttl_tables	Row tables with a TTL, and column tables whose TTL is one tier that deletes rows.
ydb_column_families	The column families of each row table, without a default family that has no storage pool of its own, no compression, the regular cache and keep_in_memory off.
ydb_table_partitioning	Row tables whose partitioning, read replicas or key bloom filter differ from what YDB gives a new table, and declarations that state a starting layout.
ydb_index_partitioning	Global sync, async and unique indexes whose partitioning or read replicas differ from what YDB gives a new index.
ydb_column_tables	Column tables.
ydb_tiered_ttl_tables	Column tables whose TTL has any other tiers, such as one that moves rows to an external data source.
ydb_vector_indexes	Vector indexes.

The host counts the global indexes it reads as common indexes in indexes. The index set and the column storage models add the indexes they hold under the same metric and help, Indexes: vector and full-text indexes, and a column table's local indexes, which the built-in engine counted among the indexes too. A host adds them to its count with stokaro/ptah#5378; a host without it refuses schema stats when a provider reports the metric.

What YDB gives a new table or global index is automatic partitioning by size at 2048 MB, none by load, at least one partition, no maximum, no read replicas and no key bloom filter; an unspecified setting is that value. A read holds every setting the server describes and is counted against these values, as is the declaration the host converts it to before it counts.

A host that reads global indexes as common indexes (schema.observed-indexes) leaves them out of a table's index set, so ydb_index_partitioning counts each such index from its own settings, the index settings model. A host reads those when it negotiates schema.observed-index-facets; through a host that does not, the count leaves the common indexes out.

Offline coordination declarations

The source service accepts the captured YAML document key coordination_nodes. The host parses YAML and supplies typed values and original byte spans; the provider interprets this grammar without reading files or opening a session.

coordination_nodes:
  workers:
    schema: app
    self_check_period: PT2S
    session_grace_period: PT5S
    read_consistency_mode: strict
    attach_consistency_mode: relaxed
    rate_limiter_counters_mode: detailed

Each entry is a mapping. Its key supplies the node name unless name overrides it. schema is an exact directory relative to the database; struct_name retains source provenance. These fields and every setting require text. Unknown keys, explicit nulls, numeric coercion, repeated fields, duplicate object identities, and collisions with declared tables in the same captured namespace refuse the whole source batch. Root ptah_locks is reserved for runtime use; the same leaf name in a user directory is allowed. Inspection still returns root coordination nodes as ordinary observations.

Periods use positive ISO8601 durations and must preserve whole milliseconds. The self-check interval must be 500–10,000 milliseconds. The grace interval must be at least the effective self-check interval plus 1,000 milliseconds and at most 30,000 milliseconds. The provider refuses configurations the server would clamp. Mode strings are case-insensitive in source and stored lowercase.

Omitted settings remain absent in the desired model. They request server defaults when the node is created or compared: a 1,000-millisecond self-check, 10,000-millisecond grace interval, relaxed reads, strict attachment, and aggregated counters. Explicit settings equal to defaults remain explicit. Offline normalization does not assert desired/observed equivalence.

The decoder returns native objects with source traces and coverage limited to the models enrolled by each document. An empty mapping declares no nodes within that scope; an empty model scope stays empty. No successful prefix survives an invalid entry or cancellation. Desired and observed models have distinct encodings and definition digests; observation coverage contains only observed models.

The same service decodes the host's Go directive ptah:schema:coordinationnode, written on a struct of its own, one node per directive:

//ptah:schema:coordinationnode name="limits" schema="app" self_check_period="PT2S" read_consistency_mode="strict"
type Limits struct{}

name is required, and the other attributes are the YAML settings above, with the same rules; struct_name has no attribute, since the host names the struct. A YAML document that writes coordination_nodes, and every Go source, describes the nodes completely, as they do for the host's embedded engine: a node the source does not declare is one the plan removes from the directories it reads.

A host that lists source.hcl (contract 25.3, ADR 0036 in Ptah) reads the top-level coordination_node block the engine claims, as the host's bundled YDB source reads it: labeled with the node's name, or its directory and its name, its settings string attributes spelled as above, and named coordination_node by a permission's for.

coordination_node "app" "limits" {
  self_check_period     = "PT2S"
  read_consistency_mode = "strict"
}

It also claims, at the index-attributes site, a global index's partitioning and read replicas inside the frontend's index block, spelled as the index directive spells them (see Index settings from a source): a switch and the read replicas as strings, a count as a number.

index "by_body" {
  auto_partitioning_by_load              = "ENABLED"
  auto_partitioning_min_partitions_count = 2
  columns                                = [column.body]
}

A ptah:not-described coordination_node header entry, with a node's path or none, is the one header kind the engine declares, and the provider refuses a name no node can take with the host's words. With source.limits negotiated, the document's claim leaves the node it names, or every node, unmanaged. A host that does not list source.hcl reads no such block.

An HCL document is an Atlas file, which names none of the nodes it leaves out, so it claims the nodes only through its header, as the host's embedded engine reads it: a coordination_node entry, or the captured ptah:feature-coverage header an export writes, which the host reads itself. A document with neither claims nothing of them, and a plan from it creates and alters the nodes it declares and drops none. The provider cannot keep this rule alone: the SDK lets a decoding claim less than complete only through header limits, so its reply for a document without them claims every node. A host that takes an HCL document's claim only from its header asks the provider about no other document; one that asks about every document plans to drop each node the document does not declare, and writes its HCL exports without the captured header. TestHostKeepsTheNodesAnHCLDocumentDoesNotName measures the first kind of host and skips on the second.

An HCL file the host's embedded engine exported names that engine's coordination-node model in its captured header: owner ptah.run/ydb, kind ptah.run/ydb/coordination-node, schema version 1. The provider names that identity as a predecessor of its own coordination-node model (models.predecessors, contract 25.25), so a host that lists the feature reads such a file with the claim it was written with. Applied back, it plans nothing, and a node whose block is removed is dropped, as the embedded engine dropped it. A file the host cuts the document into carries the provider's identity. A host without the feature refuses the file with unknown feature codec: "ptah.run/ydb/coordination-node"/desired. TestHostReadsTheBuiltinEnginesHCLExports_HappyPath applies exports recorded from the embedded engine (internal/backend/testdata/builtin-hcl).

A captured header the host writes for this provider's models names each model by the identity the provider describes: the provider as the owner, the model's kind and representation, its schema version and its definition digest. The release version, the features a host negotiated and the engine's other declarations do not enter it, so an HCL export one release of this provider wrote reads with the next one while no definition text changed. When a text changes without changing what a claim of the model means, the identity it replaces is named a predecessor of the model itself (models.own-predecessors), and an earlier export still reads; a header that names a definition the provider neither describes nor names is refused: incompatible feature codec: coverage model "github.com/stokaro/ptah-ydb-plugin/coordination-node"/desired. TestModelIdentitiesAreDescribedOrDeclaredPredecessors pins the identity of every desired model, so a definition text cannot change unseen, and TestHostReadsAnEarlierProviderHCLExport_HappyPath applies an export an earlier build recorded (internal/backend/testdata/provider-hcl). A Go export, and an HCL export from a host that writes no captured header for a provider, name no model identity.

A Go source carries the same entries as ptah:schema:notdescribed declarations, under source.limits, of the kinds the engine lists in go_header_kinds: coordination_node, streaming_query, resource_pool, resource_pool_classifier, secret, topic, external_data_source, external_table, replication and transfer. An entry without a name leaves the kind unmanaged; one with a name, written as a YQL header names it, leaves that object unmanaged. A source that declares an object it also leaves unmanaged is refused.

//ptah:schema:notdescribed kind="topic" reason="not-inspected" provenance="observed"
type PtahSchemaObjects struct{}

EncodeSource (source.encode, ADR 0037, and source.encode-groups, ADR 0040) writes desired values back as the keys, directives and blocks this provider's decoders read, which decode to the same declarations:

Value	YAML	Go	HCL
Coordination node	entry of coordination_nodes	ptah:schema:coordinationnode	coordination_node block after the tables
Topic and its consumers	entry of topics	ptah:schema:topic, ptah:schema:topic:consumer	loss
Secret, by the variable its value comes from	entry of secrets	ptah:schema:secret	loss
External data source and external table	entries of their keys	their directives	loss
Resource pool and classifier	entries of their keys	their directives	loss
Streaming query	entry of streaming_queries	ptah:schema:streamingquery	loss
Row-table TTL	platform.ydb	attributes of ptah:schema:table	the table's platform "ydb" block
Row-table partitioning, read replicas, key Bloom filter, starting layout	table keys	attributes of ptah:schema:table	loss
Column families and the columns they hold	column_families	ptah:schema:columnfamily	loss
Column-table storage, hash key, shards, TTL	column_store	attributes of ptah:schema:table	a TTL that only deletes in the table's platform "ydb" block; the rest is a loss
Changefeeds and their consumers	changefeeds	ptah:schema:changefeed, ptah:schema:changefeed:consumer	loss
A global index's partitioning and read replicas	keys of its index entry	attributes of ptah:schema:index	attributes of its index block

A value is written as the declaration the provider compares it as: a topic or a changefeed as its declared settings, without the attributes YDB keeps on it; a table's settings without the primary key the common columns carry; a column default and a Serial's sequence not at all, since the common column carries them: the read reports a sequence's start and increment as the column's identity_start and identity_increment beside auto_increment, and refuses a sequence with any other setting. A facet a source states by writing nothing, such as an empty changefeed set, an empty native index set or a column of the default family, is written by the table's own part. The default family every row table holds is written only when it has a setting of its own, such as a compression or a storage pool, and a declaration that leaves it out keeps what the table holds. A Go export writes the table directive's attributes in the host's embedded engine's order: a column table's storage, with store="column", then the TTL, then the partitioning and the starting layout. It also gives each column the Go type of its YDB type, as that engine does, and refuses a nullable key column, which no declaration can ask for.

A Go export writes the directives of the standalone objects in the order the host's embedded engine wrote them, each with its attributes in the order its claim lists them: topics and their consumers, secrets, external data sources, external tables and streaming queries after the views, then resource pools, their classifiers and coordination nodes. Under source.go-placements (contract 25.26, ADR 0077) those last three stand after the default privileges, below the grants, where the embedded engine wrote them; a host that does not list the feature writes them after the views.

What no declaration states is a loss for its value, in one line that names it: an access requirement, table attributes, storage settings, tiering, a temporary table, a family kept in memory, a TTL run interval, a disabled changefeed or the stream of an async replication, a native index, a column's compression or encoding, a native view (the host writes views from the common wire), and an object of another database than the one it was read in. A secret's value is never written. HCL declares coordination nodes, a global index's settings, and a table's TTL in its platform "ydb" block, as the host's embedded engine does; its other YDB settings are losses, which the host reports at the table in the embedded engine's words, such as YDB table settings (AUTO_PARTITIONING_MIN_PARTITIONS_COUNT = 3) are not represented in HCL. A facet whose TTL the block writes and that holds such a setting too is written in part, and the loss names the setting. The settings a read reports for every table, YDB's documented defaults, are no loss, and neither are the empty changefeed and native index sets a read reports for a table without either, which the table's platform "ydb" part writes by writing nothing; a changefeed or a native index the table holds is a loss by name (changefeed updates is not represented in HCL). The parts are decoded again before they are returned, and a value they do not give back is a loss. A value that needs several declarations is a loss for a host that does not negotiate source.encode-groups. struct_name is written in YAML alone.

A source claims every model its format lists, completely. An export of a read is complete for the selection it is applied to, as the host's embedded YDB engine writes it: a host that writes a read narrowed by --schemas sends each kind the read knew in every directory it listed as complete. Applied back over that directory, the export plans nothing, and an object whose declaration is removed from it is dropped; applied over more directories, it drops their objects too. What the request does not know is written as not-described entries, so that the source read back claims no more than that: a kind no directory of the read knew, or one the read knew less in some directory, is left unmanaged as a whole, since a header cannot name a directory, with reason="not-inspected" provenance="observed". An object whose own knowledge is not complete gets an entry that names it. A facet is known where its table is, so a table the export declares carries its facets as the read holds them, and a declared table whose facet knowledge is not complete refuses the export. Knowledge left unmanaged by a source's own entry is passed on with provenance="declared". YAML writes no header, and neither does a format for a host that does not negotiate source.limits, so there such knowledge refuses the export, as does a kind no entry names. A host that writes an HCL export's account itself, as its captured ptah:feature-coverage header, sends the provider no coverage for HCL, and the provider writes no entry there.

The subprocess tests exercise decoding, encoding, normalization, and representation conversion without a database address; a YQL schema declares coordination nodes through SQL schema reading.

Offline row-table declarations

The same source service decodes the table-scope settings that the host's bundled YDB source reads from YAML and from Go annotations, so a table written the way users write tables reaches the provider. The host lexes the source and sends the values of the keys, directives and attributes the provider claimed; the provider owns their meaning. The host refuses a YAML key the provider does not claim. It reads a Go comment that no provider claims as no declaration, and refuses an attribute of its own directives that no provider claims, so the provider claims the Go directives and attributes it cannot carry and refuses them by name. Nothing is dropped in silence.

A YAML table entry takes these keys, spelled as the YDB settings are:

tables:
  documents:
    auto_partitioning_by_size: ENABLED          # ENABLED or DISABLED, in either case
    auto_partitioning_partition_size_mb: 2048   # a whole number of at least 1
    auto_partitioning_by_load: ENABLED
    auto_partitioning_min_partitions_count: 2
    auto_partitioning_max_partitions_count: 16
    read_replicas_settings: ANY_AZ:2            # PER_AZ:<n> or ANY_AZ:<n>; a count of 0 removes replicas
    key_bloom_filter: ENABLED
    column_families:
      default: {compression: lz4}               # off or lz4
      cold: {data: hdd, compression: lz4, fields: [body]} # data is a storage pool kind, kept as written
      hot: {cache_mode: in_memory}              # regular or in_memory, in either case
    platform:
      ydb:                                      # the TTL, spelled as the Go platform properties without the prefix
        row_deletion_column: created_at
        row_deletion_interval: P30D

In YAML, a family's fields lists its column names as a sequence or a comma-separated scalar. Each name must match a column in that table. A column can belong to only one family, and the default family lists no columns.

The provider claims the ydb group of the table's platform key as platform.ydb, and the host keeps no copy of the group. The provider therefore refuses every key of the group other than row_deletion_column, row_deletion_interval and row_deletion_unit. The host refuses a group that holds a list or a mapping before it reaches the provider; scalar values arrive as written.

In Go, the same settings are attributes of the host's own ptah:schema:table directive, spelled as the YAML keys are, and a TTL is written as the platform properties of the ydb engine. A family is a directive of its own, //ptah:schema:columnfamily, on the table's struct, with the attributes name, data, compression, cache_mode and fields. The table attribute names another table of the file (name or schema.name) and wins over the struct's own table. fields lists the columns the family holds, as a comma-separated list of the column names (fields="body, notes"). The provider returns one column facet for each name, owned by the column under the table, and the host attaches it to the field of the struct the directive is written on. A family that has fields and a table attribute for another table is refused, because the host places only the columns of the directive's own struct. The default family lists no column, a column sits in one family, and a name the struct has no field for is refused by the host.

//ptah:schema:table name="documents" auto_partitioning_by_load="ENABLED" read_replicas_settings="ANY_AZ:2" platform.ydb.row_deletion_column="created_at" platform.ydb.row_deletion_interval="P30D"
//ptah:schema:columnfamily name="default" compression="lz4"
//ptah:schema:columnfamily name="cold" data="hdd" compression="lz4" fields="body"
//ptah:schema:columnfamily name="hot" cache_mode="in_memory"
type Document struct { /* ... */ }

A family's cache mode, regular or in_memory, needs a target that takes one: YDB 25.4 and later, or a cluster that turned EnableTableCacheModes on (see Feature flags). A plan that declares one for any other target is refused with the flag's name, regular included, as the host's embedded engine refuses it.

In HCL, the TTL is the platform "ydb" block of the table, with the same three properties as override blocks. ptah and ptah-compat read it and schema inspect writes it:

table "events" {
  schema = schema.app
  platform "ydb" {
    override "row_deletion_column" {
      value = "created_at"
    }
    override "row_deletion_interval" {
      value = "P30D"
    }
  }
  column "id" {
    type = Uint64
  }
  column "created_at" {
    type = Timestamp
    null = true
  }
  primary_key {
    columns = [column.id]
  }
}

The provider claims the block (source.hcl-platform, ADR 0072), so it needs a host that reads it; a host that does not refuses the document. The host refuses an override other than the three, naming them. The block's label may be the alias ydbs. A table that writes no block states nothing about its TTL, and keeps the TTL it holds, where a YAML or Go source that states no TTL asks for none. HCL cannot say a table is a column table, so the block states the TTL of a column table the database holds, which keeps its storage and takes the TTL as one tier that deletes.

A TTL needs row_deletion_column and row_deletion_interval. The interval is an ISO 8601 duration of weeks, days, hours, minutes and seconds (P30D, P2W3D, PT1H30M), and YDB keeps whole seconds, so a fraction of a second, months and years are refused. A column that counts since the Unix epoch also needs row_deletion_unit, one of SECONDS, MILLISECONDS, MICROSECONDS and NANOSECONDS in any case. Which column types fit the unit is checked when the plan is made, where the table's columns are known. A refused interval or unit says why in the host's output, in the words of the host's embedded engine without the value the source wrote, which a provider's reason never quotes: table "items": platform.ydb.row_deletion_interval: the interval has a fraction of a second, and YDB keeps whole seconds only (PT1.5S reads back as PT1S). A tier of a column table's TTL is refused the same way.

All the keys, attributes and directives of one table become one desired row-table facet, because a document may hold only one facet of a kind for a table. The facet holds only what the source states. It has no primary key, which comes from the table's common columns; a stated key must still be distinct and ordered. A partitioning setting a source leaves out keeps what the table holds. A table that declares nothing, such as an empty column_families mapping or store="ROW" alone, gets no facet.

The values are read as the text the author wrote, so 3 and "3" are the same count, and a boolean where ENABLED is due is refused. The shared frontend normalizes the spelling of a YAML number before the provider sees it (1_000 arrives as 1000, 0x10 as 16); the bundled decoder reads the written text and refuses both.

A starting layout is the partitions a new table is created with, and YDB takes it only when it creates the table. uniform_partitions takes a count of at least 1, and partition_at_keys takes split points in the host's grammar: a comma separated list whose items are a value, for a split on the first key column, or a parenthesized list of values for the leading key columns, such as 10, 20 or (10, 'a'), (20). A value is a bare run of characters or a string in single or double quotes with backslash escapes. Both keys are claimed in YAML and in the Go table directive, and a table that names both is refused where it is decoded; the host accepts the pair when it decodes it and refuses the table when it plans it. The key column types are checked when the table is created (see "Table creation"). The corpus holds the inputs and the host's answers.

Not read yet, and listed so that nobody mistakes the silence for support:

Setting	Why
fields with a table attribute for another table in Go	The host places the columns of the directive's own struct only. Claimed and refused by name.
Local, full-text and vector index options on an index of another type	A host that lists source.index-kinds sends an index of a vector, full-text or local type whole (see Index declarations of the provider's types). On an index of any other type, and on every index for a host without the feature, these attributes of ptah:schema:index and the same keys of a YAML index entry are claimed and refused by name. A global index's partitioning and read replicas are read (see Index settings from a source).

A table is managed for its TTL when its source states table properties for it, or when the host reports complete knowledge of the table properties of the source (the knowledge a Go or YAML source carries when its format describes them). A comparison that finds a TTL on such a table and none declared reports a change, and the planner removes the TTL, as the bundled source does: measured with the host at the commit named in the corpus, schema apply over a Go file with no TTL plans ALTER TABLE ... RESET (TTL) for every table of the file. A source that could not describe the properties, or a table whose current properties were not read, leaves the TTL alone.

Column tables

A column table is declared with store="COLUMN" in the Go table directive (in either case; an export writes store="column", as the host's embedded engine did), or with the column_store mapping in YAML. The Go attributes partition_by_hash (a comma separated list of key columns), column_shards (a positive 32-bit count) and column_ttl, and the YAML keys hash_columns, partitions and ttl, state the hash key, the shard count and a tiered TTL. A tiered TTL names a column, an optional epoch unit and ordered tiers; only the last tier may delete rows, and every other one moves them to an absolute external data source path. A TTL that only deletes is written as the row TTL (platform.ydb.row_deletion_* in Go, the row_deletion_* keys of the YAML ydb group) and becomes the column table's one deleting tier. The declaration becomes one column-storage facet, and what it leaves out stays the server's choice. store="ROW" declares nothing.

The corpus holds each spelling and each refusal with the host's answer (*-store-column-* rows). The host's bundled decoders also accept a column table beside row-table settings (partitioning, read replicas, a key bloom filter, a starting layout, column families) and beside both a tiered TTL and a row TTL, and its renderer refuses the table when it plans it. The provider refuses such a declaration where it is decoded, with the renderer's words: row-table partitioning, column families and changefeeds cannot be applied to a column table, and declare either tiered TTL or a TTL that deletes rows, not both. It is stricter in one place: the host reads a fractional YAML partitions as its whole part, and the provider refuses it.

Inspection reads a column table, a COLUMN_TABLE directory entry, from the table service and from the monitoring endpoint's scheme description (/viewer/json/describe). The table service's description carries the columns, the key, the hash key (partitioning_settings.partition_by) and the TTL, and leaves out the shard count and the local indexes (measured on 25.1.4.7 and 26.2.1.14 for a table created with AUTO_PARTITIONING_MIN_PARTITIONS_COUNT = 3); the monitoring page carries both. The two are read together and a disagreement refuses the inspection. The table service writes a tiered TTL in a field the pinned ydb-go-genproto lacks, so the provider reads a column table's description with definitions of its own (ColumnTableDescribeResult, adapted from upstream with its wire numbers). The table reads back as a common table with one column-storage facet (hash key, shard count, data channel count, local indexes and the TTL with its tiers, a TTL that only deletes included) and a column facet for each column; it has no row-table, index or changefeed facet. Its access metadata is not observed. A column table with a column family, a column default, attributes or row-table settings is refused. The hash function has to be the one YQL's PARTITION BY HASH writes, HASH_FUNCTION_CONSISTENCY_64: the native CreateTable call writes HASH_FUNCTION_MODULO_N instead, and the host's embedded engine reads no other.

Without the monitoring parameter in the address, an inspection that meets a column table is refused with column table <path> requires monitoring=http(s)://host:port in the YDB address, and the column table is never read as a row table. The SDK client passes the host the status code of a refused call and not its message, so the host reports provider inspect failed (FailedPrecondition).

Creation. The planner creates a declared column table with one YQL CREATE TABLE ... PARTITION BY HASH (...) WITH (STORE = COLUMN, ...) through the query service, once, outside a transaction, with the shard count as AUTO_PARTITIONING_MIN_PARTITIONS_COUNT and the TTL as YQL writes it. YQL is used rather than the native CreateTable call because the native call writes HASH_FUNCTION_MODULO_N, which the host's embedded engine does not read. Every key column is written NOT NULL, as the host writes it: YDB refuses a nullable key column in a column table, and a Go field marked primary is nullable unless it says not_null. The table is read back from the table service and the monitoring endpoint and must hold the declared columns and key, the declared hash key or, without one, the whole key, the declared shard count, the declared TTL and the declared local indexes. A creation in a session whose address names no monitoring endpoint is refused before anything is written. A tier that moves rows needs the tiered_ttl capability, which a session reads from EnableTieringInColumnShard; without it the plan is refused with the reason. Column defaults, Serial columns, column families, global indexes and a data channel count other than the server's 64 are refused. A TTL column has to be first in the key: the host's embedded engine also takes one later in the key that a min_max local index reads, which 26.2.1.14 refuses with the index in place (Haven't MAX-index for TTL column and TTL column is not first column in primary key), so the provider refuses it before a statement is sent.

Local indexes. The column-storage facet states a column table's local indexes as its read describes them: a bloom (bloom_filter), an ngram (bloom_ngram_filter) or a minMax (min_max) index, each over one column, with its settings. A setting the declaration leaves out is the server's default: a probability of 0.1, an n-gram size of 3 and case-sensitive matching. The storage YDB places an index in is not part of a declaration, and the comparison leaves it out, since YDB describes an index created with its table in other storage than one added later. An index reads one column (26.2.1.14 answers Local bloom index on column tables requires exactly one index column, where the host's embedded engine takes several for a bloom index), an n-gram index a Utf8 one, an n-gram size is 3 to 8, and a probability gives at most 8 n-gram hashes, as that engine requires. Each kind needs its capability (see Feature flags); 25.1 has no local index grammar at all (local: alternative is not implemented yet). A new table's local indexes are written inside its CREATE TABLE, as INDEX `bf` LOCAL USING bloom_filter ON (`body`) WITH (false_positive_probability = 0.01), every setting named. A Go or YAML source declares a local index as an index directive or entry of its type, which a host that lists source.index-kinds sends whole (see Index declarations of the provider's types); an HCL document declares none, since HCL cannot state the column table it belongs to.

Export. A Go or YAML export of a column table writes its hash key, its shard count and its TTL, and on a host that lists source.index-kinds each local index as a declaration of its type with its settings. A source writes no data channel count, and the 64 YDB gives every column table the provider creates is what a declaration without one creates, so a read's count of 64 is no loss; any other count is. The storage and the encodings YDB gives a local index are not part of a declaration either.

Comparison and planning. The comparison ydb-row-table-properties compares the column-storage facet of a table that exists on both sides, and reports a github.com/stokaro/ptah-ydb-plugin/column-storage-change when the hash key, the shard count, the TTL or the local indexes differ, or when the table is a column table on one side and a row table on the other. A declaration that leaves out the hash key or the shard count takes what the table holds. Under complete coverage, a table the source states no column storage for asks for a row table. YDB fixes a table's storage kind, hash key and shard count when it creates the table and copies no column table into a new one, so the planner refuses each of these changes with the host's words (changing between row and column storage requires an explicit data migration, changing a column table's hash key or shard count requires an explicit data migration, with both values; column-table rebuilds require an explicit data migration). A column step of a column table is refused as not planned yet.

The local indexes are managed when the declaration states that it lists every one, as a read does and as a Go or YAML source of a host that lists source.index-kinds does (localIndexesStated), or when it states at least one: an index the table holds and the declaration leaves out is dropped. Any other declaration that states none leaves the table's own alone, since its source cannot declare one. Each change is one step in the lifecycle phase, column-table-add-local-index/N, column-table-drop-local-index/N or column-table-rename-local-index/N, of the operation kind column-table-local-index-operation, running ALTER TABLE ... ADD INDEX ... LOCAL USING ..., ALTER TABLE ... DROP INDEX ... or ALTER TABLE ... RENAME INDEX ... TO ... through the query service. An index whose kind, column or settings differ is dropped and added again under its name, the removal first. An index whose name alone changes is renamed in place on a target with index_rename, and dropped under its old name and added under its new one on a target without it, as the host's embedded engine plans both. The host compares no local index, so the provider pairs them as the host's comparison pairs a renamed index: each new name, in name order, takes the first old name in name order whose index has the same column, kind and settings. The session reports index_rename from EnableMoveIndex, and 26.2.1.14 renames a local index with the flag off too, so a plan without it rebuilds an index the server could rename. 26.2.1.14 also changes a bloom index's probability in place (ALTER INDEX ... SET (false_positive_probability = ...)), which neither engine plans. Only an index a plan adds needs its capability. Execution reads the table from the table service and the monitoring endpoint and refuses before it writes when the table holds an index of the name to add or to rename to, lacks its column or holds it with another type than an n-gram index reads, or holds the index to drop or rename otherwise than the plan captured it; after the call it requires the added or renamed index to read back as requested under its name, the dropped or renamed one to be gone under its old name, and every other local index, the layout and the TTL to be unchanged.

The TTL changes in place, as the host's embedded engine changes it. ALTER TABLE ... SET (TTL = ...) replaces the TTL a column table holds, whatever either one moves rows to (measured on 25.1.4.7 and 26.2.1.14), so a held TTL that moves no row is replaced by the SET alone (column-table-ttl/<n>/set). A held TTL that moves rows to an external data source is removed first with ALTER TABLE ... RESET (TTL) (column-table-ttl/<n>/reset, placed early and ordered before the SET): YDB refuses to drop or replace a data source a TTL names (Other entities depend on this data source), so the held TTL lets go of its sources before their owner drops or replaces them, and the declared one is set once they exist. Each step reads the data sources its TTL names, which orders it against their owner. A table without a TTL takes the SET alone, and a declaration without one the RESET alone. A lone RESET drops the table's TTL setting and a lone SET creates it; any other step alters it, because the host refuses a provider step that creates what another provider step drops (it runs every creation of a phase before every drop), and the dependency orders a pair. The declaration is held to the rules of a creation, and a tier that moves rows needs tiered_ttl. Each step reads the table from the table service and the monitoring endpoint, refuses before it writes when the table holds another TTL than the plan saw or when the TTL to set reads a column that is absent, not first in the key or of a type its mode does not read, runs its statement once through the query service and reads the TTL back.

internal/tablesource/testdata/corpus holds the inputs and the answers of the host's bundled decoder for them, and the differential test in internal/tablesource requires the provider to mean the same as the host for every input the host accepts, to refuse every input the host refuses, and to name every difference in the table above. testdata/oracle explains how the answers are regenerated against a host checkout.

Coordination comparison

The offline models.compare service compares desired coordination declarations with captured native observations. Its context must establish the coordination_nodes capability, which session receipts advertise for the native implementation. Omitted settings compare using server defaults; source holder names do not change native configuration. Removing an explicit nondefault setting produces a change back to the default. Raw stored values that the server clamps refuse comparison.

The github.com/stokaro/ptah-ydb-plugin/coordination-node-change model carries complete before and after operands. The native before retains access metadata. A null operand means established absence. Normalization reconstructs impact, removal, and access facts from these operands. Creation has unknown inherited access policy; alteration preserves access; dropping a node removes its runtime state and access. Planning lowers these changes to the github.com/stokaro/ptah-ydb-plugin/coordination-node-operation model, which rendering preserves as a native execution envelope.

An unread object does not establish absence. Independent coverage sources must all establish completeness; overlapping unknown regions override complete regions, and explicit subject claims override namespace claims. Namespace components, target scopes, and exact empty paths retain their boundaries. An incomplete desired source preserves known current configuration. Every decision is made for one node, so a namespace the read left out produces no diagnostic of its own.

The host names a declared node without a database, because it lives in the session's, while inspection names that database in every owner and region it reads. The comparison reads both in the host's terms, so the declared node and the node read at its path are one node. A change names the node as the host sent it: by the declared identity, or by the read one for a removal. A comparison whose read names two databases is refused.

Coordination exports retain owner, direct grants, and effective-permission requirements in an optional access field. Its JSON field names use snake case. Omission leaves access unmanaged. The YAML source grammar above does not accept access declarations. Comparison reports unsatisfied access requirements as undecided; change/operation normalization refuses them. Existing nodes can change configuration while satisfying unchanged access requirements. Creation with explicit access requirements is refused because the coordination service cannot promise inherited permissions or apply grants.

The comparison refuses contradictory identities, forged normalization receipts, and collisions with declared relations. Required nodes are not removed. Exclusions for this model match exact relative paths such as app/workers, without case folding or trimming. Inspection reserves the database-root ptah_locks node (see the reservations above), and the comparison leaves that path out too: it plans no removal and reports nothing about it. A declaration at that path is the one case it reports, as undecided; YAML and Go declarations and change normalization reject it. This changes the desired model definition; pre-v1, no compatibility with the earlier experimental definition is owed.

Offline subprocess tests cover decode, compare, and change normalization without opening a session. Live tests compare native observations after closing their session on both qualified YDB lines. TestHostAppliesACoordinationNode_HappyPath drives ptah schema apply of the current host through this provider from a YAML, a Go and a YQL source: it creates a node, finds nothing to change on the next run, changes its settings in place, and removes it once the source no longer declares it, reading the node back after each run.

Comparison counts

The provider offers models.compare-counts, so ptah schema drift --format json lists its native kinds in the report's inventory. Each object comparison counts, under its change kind, the objects it decides: an object both sides hold counts on both, whether or not it differs and also when the plan drops and creates it again, an object only the source declares counts as desired, and an object only the server holds counts as observed. An object the comparison excludes, reserves, keeps because the source does not describe it, or leaves undecided counts on neither side.

The counted kinds are secrets, streaming queries, coordination nodes, topics, external data sources, external tables, resource pools, resource pool classifiers, async replications and transfers. A service counts every kind it owns, with zeros for a kind a batch holds nothing of, because the host reports a kind only when every service that ran counted it.

The row-table facet comparison counts nothing, so its two change kinds have no inventory entry. A table's TTL and partitioning are settings that stand for no object. A table's changefeeds change as one set, with one change for the table, and the contract refuses more additions and removals than a kind has changes, so a set that gains two changefeeds cannot count them.

A host that does not list the feature receives no counts and reports none of these kinds, never a zero.

Offline column declarations

The values.convert service converts column properties between distinct typed observed and desired models. It preserves family assignment, typed literal defaults, sequence configuration, compression settings, and encoding order. Identity, storage type, and nullability remain in the common column contract. An omitted default requests the native no-default setting and stays omitted during normalization; dropDefault: true expresses removal explicitly. Unknown fields, unknown enums, duplicate keys, null settings, and overlapping defaults are refused. Explicit sequence configuration retains optional values, including cycle: false.

Sequence positions are mutable data. An observation containing setVal refuses conversion rather than exporting a request to reset the sequence. Conversion validates the source payload and its captured normalization, retains exact owner identity and ordered records, and returns no prefix on failure or cancellation. It opens no connection and establishes no inspection coverage.

The column-property models complement the common column service and preserve native settings through conversion and JSON schema export.

Offline row-table declarations

The row-table model preserves ordered primary keys, TTL, storage pools, column families, attributes, partitioning policy, bloom-filter settings, read replicas, tiering, and temporary lifetime. Unknown settings, duplicate keys or families, invalid TTL modes, and null settings are refused. Attribute values remain exact, including empty strings. The row-table reader interprets an unspecified native storage kind as row storage; column storage retains its separate model.

Access declarations keep the owner and direct grants separate from requirements on effective permissions. Effective permissions include inherited access under YDB's permission model. A present access declaration specifies the complete direct-grant set and the required effective-permission set. These requirements do not add direct grants. Conversion neither infers an inheritance switch nor changes permissions. The provider does not yet advertise table or access comparison/planning.

Physical shard boundaries are runtime topology. A captured value containing them refuses conversion rather than turning the current split points into creation settings. A starting layout (uniformPartitions, or partitionAtKeys as lists of source text values) is a creation setting of the desired model only: a read finds none, and converting a desired table to the observed representation leaves it out. Partitioning policy remains declarative. Statistics and transient object identity cannot enter the property model. Conversion preserves a complete mixed table/column batch, works after the session closes, and establishes no inspection coverage.

JSON schema export includes table and column facets together with native objects and their captured coverage. Unsupported input refuses the whole conversion.

Offline index and stream declarations

The values.convert service also converts index, changefeed, and standalone topic properties between separate typed desired and observed models. It works without an open session, retains exact owner identity, and returns no prefix when any value in a mixed batch fails. Conversion establishes no inspection coverage. The index model advertises comparison and planning, for adding and dropping global secondary indexes of an existing row table (see Index alteration), and so do the topic model (see Topics) and the changefeed model (see Changefeeds).

Index declarations retain the native index kind, ordered index/data columns, and kind-specific settings. Each declaration includes complete requirements on the server-owned implementation tables: ordered columns and keys, column defaults, storage, partitioning policy, and access constraints. These are requirements on the index implementation, not independent tables to create. Missing implementation tables or columns refuse conversion. Physical shard boundaries also refuse conversion; current splits are not creation settings. The observed index model canonicalizes unspecified implementation storage as row storage and excludes internal sequence positions. This changes its definition digest; this provider is pre-v1 and carries no reader for the previous shape.

Topic declarations preserve partitioning policy, retention, codecs, write/read quotas, attributes, metrics settings, and consumer configuration. Consumer settings retain read timestamps, availability periods, codec preferences, attributes, and streaming-consumer presence. Access uses the same direct-grant and effective-permission requirements as row tables. Runtime partition topology and statistics cannot enter a declaration.

Changefeed declarations include the child topic's settings as well as mode, format, virtual/resolved timestamps, attributes, AWS region, schema-change reporting, and user SIDs. disabled and initialScan are independent settings. An observation in STATE_INITIAL_SCAN converts to enabled intent with initialScan: true. Converting that declaration back predicts STATE_ENABLED; it does not prove that a server finished scanning. Scan progress counters remain excluded.

Unknown fields/enums, runtime state in declarations, null settings, duplicate names, and incomplete child descriptions are refused. Empty index/changefeed sets keep their own desired model definitions. Indexes, implementation tables and changefeeds normalize by name; consumers keep the order the declaration lists them in, and column and key order is retained.

Offline native-object declarations

View declarations retain query text and table-service attributes, including comments. The view and table APIs must agree about access metadata. Conflicting access or additional table settings refuse conversion rather than creating competing declarations for one view. The table API returns disabled partitioning switches for views on both tested YDB lines. This exact placeholder is excluded; other partition settings are refused.

External-source declarations preserve source type, location, and exact native property keys and values, including authentication references. External-table declarations preserve the data-source path, location, content options, ordered column names and types, nullability, and typed column settings. Conversion does not contact an external service or resolve a secret.

Coordination conversion retains optional timing and mode settings together with access requirements. Native unspecified settings remain omitted. A desired struct_name cannot convert to an observation because it has no server counterpart. A runtime-reserved coordination identity is not exportable as a user declaration.

Column-storage declarations retain hash-column order, partition and data-channel counts, local-index settings, and TTL tiers. Optional false values, disabled TTL, and omitted TTL stay distinct. The same validation applies to both representations; native IDs and shard addresses have no declaration field.

The native models use distinct desired definitions. This changes experimental definition digests and the coordination change/operation contracts. This provider is pre-v1 and has no reader for the previous shapes.

The live CI jobs build a pinned Ptah host in a separate checkout and run schema inspect --format json with an explicitly trusted provider manifest. They compare every exported model with SDK conversion and check that coverage keeps every inspected model family. These tests exercise row tables, columns, views, external objects, and coordination nodes. Local runs enable the same checks with PTAH_HOST_TEST_BIN pointing to that host binary.

Offline column-storage observations

The column-storage decoder accepts captured monitoring JSON together with an independent table-service description. It checks table identity, ordered columns and primary keys, primitive types, nullability, and reported hash keys. It preserves hash-column order, partition and data-channel counts, local bloom/ngram/min-max indexes, index storage settings, and TTL tiers. Omitted index options remain absent. Explicit false values remain explicit.

Unknown schema fields, repeated fields or aliases, inconsistent descriptions, unsupported type extensions, and configured column families are refused. Native IDs, schema versions, and shard addresses are excluded from the model. Local indexes are sorted by name; hash columns and TTL tiers retain order.

Fixture replay covers YDB 25.1 and 26.2 captures: monitoring pages copied from the host, and pairs of a monitoring page and a table-service description read from one live table each (internal/columnstore/testdata/README.md). A subprocess test checks SDK normalization and conversion without opening a database session, and live tests inspect column tables on both lines.

Native protocol generation

The upstream Go bindings lack some native index, sequence, changefeed, consumer and column family fields, such as a family's cache mode. The provider's typed definitions preserve their native wire numbers; attribution and the pinned upstream source are in third_party/ydb/NOTICE.md. Unsupported fields remain errors, a family's compression level among them. A table service request is built with the pinned bindings and carries the fields they lack as unknown fields, the only ones the provider accepts: a family's cache mode as field 6 of the family, and a new table's vector or full-text index as field 7, 8 or 9 of its index, the upstream arm. Runtime statistics are omitted from schema observations.

With protoc 35.1 on PATH, regenerate with:

sh scripts/generate-native-schema.sh

The Go generator is pinned in go.mod. CI regenerates these definitions and requires unchanged generated files.

Native coordination planning and execution

The offline planner preserves complete before and after operands. Each step has an object effect, a shared ydb.scheme-path effect, and a forbidden SQL transaction requirement. The path effect names no database, as the host names a path of the session's database. Table-to-node and node-to-table replacements require an explicit common path handoff. A view the host creates or drops occupies its path as a table does: a node beside it needs no order, and a node that takes the path of a dropped view comes after the drop. A grant or a revoke on a table or a view, and a comment, change a resource of the object's own and occupy no path; the step's read of the object carries the path. YDB needs every path above an object to be a directory, so a node created below a path follows the step that drops the object there, and a node dropped at a path precedes a step that creates an object below it. That holds whichever planning service plans the other step: a drop records a ydb.scheme-directory creation at its path, a creation reads each directory above its own, and the host runs a creation before another service's read. A drop below a path where the plan creates an object is refused before anything runs, since YDB keeps a directory after its last object goes and YQL has no statement that removes one. The refusal is the host's embedded engine's, this plan drops app/holder/events and creates an object at app/holder, which is a directory in the database, also where two planning services plan the two objects, and where both are tables. A service sees only its own changes, so the pair of two services is refused in the streaming query service's call, which the host asks on every plan with the desired and current objects of every kind and the subject of every change. Any other overlap of parent and descendant paths, and an unsupported common footprint, is refused. The host's window markers (such as common/feature-creations) and the column of a column step occupy no path, so they order nothing; the host places the step between its markers. A common step that names another database than the change is refused. A parent directory that does not exist holds no node, and CreateNode creates it with the node; execution never deletes a directory.

Rendering validates the complete operation receipt and emits a native envelope, or, in the TEXT form, the coordination node statement the host's embedded engine writes (see Text units). Execution uses native CreateNode, AlterNode, and DropNode requests. A nonempty database identity must equal the selected session database. The provider validates every envelope and transaction constraint before executing any unit in the batch.

Before each native write, execution lists the parent directory, or proves that it does not exist, and reads the node. The current configuration and access metadata must match the captured observation, or creation must have established absence. A stale operation is refused before dispatch. Effective settings are sent explicitly, including omitted settings that reset to defaults. Alteration preserves the existing node. After a successful reply, execution reads back the configuration or confirms absence; alteration also checks that access metadata did not change.

A refusal before dispatch reports NOT_STARTED, or PARTIAL after an applied prefix. A failed or unconfirmed dispatched request reports UNKNOWN and is never retried. Native operations cannot join SQL transactions. Lease checks run before dispatch and guard the call, as they do for SQL execution.

These checks are not an atomic compare-and-set. Another actor can change a node between validation and dispatch; the native API has no conditional schema-write contract. Successful creation does not establish exclusive ownership. The SDK subprocess tests cover create, alter, reset, drop, stale plans, mixed SQL/native batch refusal, and lease loss on both tested YDB release lines. The host routes a coordination node's comparison, planning and execution to these services (see Coordination comparison); other object lifecycles remain pending.

Topics

The provider declares, compares, plans and applies standalone topics and their consumers, with the syntax, the rules and the statements of the host's embedded YDB engine (dialect/ydb/ydbtopic).

A YAML document declares topics under the document key topics, a Go file with the directives ptah:schema:topic and ptah:schema:topic:consumer, and a YQL schema with CREATE TOPIC and ALTER TOPIC ... ADD CONSUMER:

topics:
  order_events:
    schema: app
    min_active_partitions: 2
    retention_period: PT36H
    supported_codecs: [raw, gzip]
    consumers:
      billing:
        important: true
      audit:
        read_from: "2026-01-01T00:00:00Z"

//ptah:schema:topic name="order_events" schema="app" min_active_partitions="2" retention_period="PT36H" supported_codecs="raw,gzip"
//ptah:schema:topic:consumer name="billing" topic="order_events" schema="app" important="true"
//ptah:schema:topic:consumer name="audit" topic="order_events" schema="app" read_from="2026-01-01T00:00:00Z"
type Events struct{}

The settings are the host's: min_active_partitions, max_active_partitions, auto_partitioning_strategy, the up and down utilization percentages and the stabilization window, retention_period, partition_write_speed_bytes_per_second, partition_write_burst_bytes and supported_codecs; a consumer takes important, read_from, supported_codecs and availability_period. Each is parsed and refused with the host's words. A Go consumer names a topic the same file declares. Each source describes the topics of the directories it reads completely, so a topic it leaves out is dropped; a ptah:not-described topic header entry of a YQL document keeps one. HCL has no topic block. The model keeps the consumers of a declared topic in the order the declaration lists them, and those of a topic it reads in the order YDB describes them, which is the order they were added in, so a statement Ptah writes for a topic lists them as Ptah's built-in YDB engine did.

A setting a declaration leaves out takes the value YDB gives a new topic, and the comparison reads both sides that way: one partition, a day of retention, a write speed of 1 MiB/s and a burst of the speed, and with auto-partitioning on, the maximum at the minimum, 90% and 30% thresholds and a five-minute window. Consumers compare by name. YDB's own attributes and the access metadata are not compared. A topic the read finds holding a setting no statement sets (retention_storage_mb, partition_count_limit, a metering mode, the read speeds or a metrics level), or a consumer whose codec list names a codec no declaration names, refuses the comparison as the host refuses to read it, whatever the source says of the topic: YDB topic <path>: <why>, the path from the database root.

The planner writes one step per topic, outside a transaction. It refuses a change YDB cannot make in place: fewer partitions than the topic has, and switching auto-partitioning off once it is on (declare paused instead). It refuses a consumer's availability_period on a server before 25.4, where the session reports topic_consumer_availability_period false. The operation runs the host's statements: one CREATE TOPIC naming what the declaration names, one DROP TOPIC, or one ALTER TOPIC that names every setting at its resolved value once any differs, drops, alters and adds consumers, and a second ALTER TOPIC that adds again a consumer whose codec list the declaration empties, which YDB changes only by dropping the consumer. A drop of a topic or of a consumer is destructive.

Execution reads the topic before the first statement and refuses one that changed since the plan read it, or a path that holds something else; it runs each statement once and reads the topic back, requiring the declared settings and consumers, the access metadata an alteration found, or the topic's absence after a drop. An unconfirmed result is UNKNOWN and is never retried.

TestTopicLifecycleSubprocess creates a topic, restarts a consumer, refuses a change planned before another writer changed the topic, and drops it, reading the topic back from the server after each step. TestHostAppliesATopic_HappyPath drives ptah schema apply from a YAML, a Go and a YQL source: it creates a topic with its consumers, finds nothing to change, changes the retention and the consumers, finds nothing to change again, and drops the topic.

Streaming queries

The provider reads, compares, plans and applies streaming queries from YAML, Go and SQL sources. A session enables streaming_queries only when the target reports both EnableStreamingQueries and EnableExternalDataSources through its monitoring endpoint. A target without that evidence refuses a streaming operation. A listed query whose system view cannot establish every setting is uninspected, so an incomplete reading cannot authorize its removal.

streaming_queries:
  copy_events:
    schema: app
    text: INSERT INTO `app/sink` SELECT * FROM `app/source`;
    run: true
    resource_pool: default
    allow_state_reset: false

The Go directive is ptah:schema:streamingquery, with the same attributes and an explicit name. SQL accepts CREATE STREAMING QUERY ... DO BEGIN ... END DO, including IF NOT EXISTS and OR REPLACE. OR REPLACE grants the same state-reset permission as allow_state_reset: true. HCL has no streaming query block. A complete source drops a query it omits; the SQL header ptah:not-described streaming_query preserves queries outside its coverage.

An omitted run means true and an omitted resource pool means default. Comparison ignores query whitespace and comments but preserves literal values. A changed body requires explicit state-reset permission because it discards checkpoints. Stopping or changing the resource pool keeps topic offsets. The planner stops a running query before changing its inputs and restarts it after its new inputs exist; a change that only stops the query emits one statement. A creation, a change or a restart runs after the host's common steps, and a grant on the query after it, as the YDB engine built into the host at 74de0b65b placed them: measured on 26.2.1.14, a query between two topics beside a group, a table and grants on the table, a third topic and the query plans the topics, the group, the table, the grants on the table and the topic, the query and then the grant on it. A principal change after a grant on the query waits for the grant, so the query runs ahead of that change. A stop and a drop run early. This assessment also runs when the query declaration is unchanged. Captured query values require planning.object-context (contract 25.7), and each query receives an assessment receipt. A changed input restarts a running query only when the desired capture establishes every surviving input and resource pool. A resource pool altered in place is no changed input: YDB runs ALTER RESOURCE POOL beside a query running in the pool and the query keeps running, measured on 26.2.1.14, so the plan alters the pool alone, as the host's embedded engine did. A plan that drops the pool a running query names is refused. Missing definitions, conflicting occupants, and foreign database inputs refuse the complete plan. An unscoped inspection walks every directory for streaming queries, including dot-prefixed user directories. A failed directory listing refuses the read; an unread query keeps explicit unknown coverage. Scoped inspection and reads that exclude other dev realms cannot establish the full dependent set, so input mutations refuse when that inventory is incomplete. A disabled query reader does not prove absence: directory listings do, and a listed unread query remains unknown. Stopped queries retain their stopped state. Native and text rendering carry the same operations and preconditions.

Input dependencies use physical scheme paths, including absolute paths inside the selected database or dev realm. A constant PRAGMA TablePathPrefix applies to subsequent relative paths. A foreign path or an unresolved prefix refuses planning. Constant named paths, named subqueries, AS_TABLE, static CONCAT and EACH inputs retain their dependencies. A qualified external reference retains the local external data source as its dependency. Unresolved named paths, USE, and dynamic table functions such as RANGE refuse planning: their complete physical inputs cannot be proven from the captured body. Ordinary strings outside a table expression create no path dependency. A named resource pool remains a database-wide dependency, including a literal dot in its name, and a query that names none depends on the pool default. Scope selection keeps that pool when it keeps the query, unless the pool is explicitly excluded, so an --include of a query plans the change of the pool it runs in, as the host's embedded engine did. Access changes follow a query's stop and restart; a final drop accounts for revoking that query's access.

Execution verifies the stored query settings before dispatch, checks cancellation and held locks before each statement, and reads back the result. This precondition contains no ACL or principal state; access ordering remains explicit in the complete plan. Each statement runs once. A dispatched operation whose result cannot be confirmed is UNKNOWN. The dev-realm guard checks the query body as well as the query's own path.

TestHostStreamingQueryGraph exercises the combined host graph on YDB 26.2: it stops a query, removes its old source, replaces a table with the new source topic at the same path, restarts the query, verifies a synced second apply, and stops it without changing its body.

Async replications and transfers

The provider reads, declares, compares, plans and applies YDB async replications and transfers, with the rules and the statements of the host's embedded YDB engine (dialect/ydb/ydbreplication). The domain rules live in internal/replication.

A YAML document declares them under the document keys async_replications and transfers, and a Go file with the directives ptah:schema:async_replication, ptah:schema:async_replication:item and ptah:schema:transfer, with the host's spelling. In YAML a name key overrides the entry's key; slashes around schema are dropped:

async_replications:
  mirror:
    schema: replicas
    connection_string: grpcs://primary:2135/?database=/prod
    user: replicator
    password_secret_name: password
    consistency_level: global
    commit_interval: PT30S
    items:
      - source: accounts
        target: replica/accounts
transfers:
  ingest:
    source: orders/feed
    target: order_log
    using: "($m) -> { return [<| id: $m._offset |>]; }"
    flush_interval: PT10S

//ptah:schema:async_replication name="mirror" schema="replicas" connection_string="grpcs://primary:2135/?database=/prod" user="replicator" password_secret_name="password"
//ptah:schema:async_replication:item replication="mirror" schema="replicas" source="accounts" target="replica/accounts"
//ptah:schema:transfer name="ingest" source="orders/feed" target="order_log" using="($m) -> { return [<| id: $m._offset |>]; }"
type Replication struct{}

A replication takes connection_string, one credential (token_secret_name, token_secret_path, or user with password_secret_name or password_secret_path), consistency_level, commit_interval and at least one item. A transfer takes source, target, using, the connection settings, consumer, batch_size_bytes and flush_interval. Each is parsed and refused with the host's words. A Go item joins the replication of its name and directory in the same file. A YQL schema declares both through SQL schema reading. An HCL file declares neither yet.

Inspection describes each REPLICATION and TRANSFER entry through the replication service: the connection string rebuilt from its parts, the credential by the secret it names, the items with their sources relative to the source database and their targets relative to the database root, the consistency and batch settings where they differ from YDB's defaults, a transfer's lambda as written, and the state (running, paused, done or error). local-ydb leaves the replication API out of its gRPC services unless YDB_GRPC_SERVICES=replication names it; on such a cluster, and for a description the provider cannot model, the object is reported uninspected with the reason, and the rest of the snapshot is read. A table an async replication writes (attribute __async_replica) is left out of the tables: YDB refuses every write and ALTER on it, and a plan that read it as an ordinary table would drop or alter it.

A replica no replication writes any more, left behind by a replication dropped without CASCADE, stays read-only for good, and YDB takes no table at its path. Where the host lists schema.unread-tables, the read names each such table in unread_tables with the reason the host's embedded engine gives, so the host refuses a schema that declares a table there before any step and prints a note on db read and schema inspect. A replica that a replication of the read writes is not named; the planner refuses a table declared there. A filtered read lists the database's replications to tell the two apart when its directories hold a replica, and a replication left uninspected makes the reason say only that the read cannot describe the replication that writes it. Without the feature, such a table is left out silently, and a declared table there is refused when its creation runs (NOT_STARTED). Either way the replica stays until an operator drops it by hand with DROP TABLE, which YDB accepts; a schema that leaves it out plans nothing for it.

The comparison resolves both sides as YDB keeps them: the connection string in its canonical form, the consistency level row and a global commit interval of ten seconds where none is named, the batch size 8 MiB and the flush interval a minute, a transfer's consumer as any where the declaration names none, and a replication's items by the tables they replicate. An object the desired source does not describe is kept; a replication YDB reports with no item, one that failed before it resolved its tables, has no declaration that keeps it and is left undecided. Session capabilities: async_replication on every line (EnableReplication on 26.2), transfers from 25.2 (EnableTopicTransfer), and replication_secret_paths from 25.4.

The planner writes one step per object, outside a transaction, and refuses what the host refuses: a change of a replication's items, consistency level or commit interval, or of a transfer's source, target or consumer, which YDB makes in no object in place; a connection or credential change of an object that is not paused, or of a replication that was failed over; a credential taken away; a secret path before 25.4. It refuses a drop of a running replication whose replica the plan declares as a table, a creation where a table stands at a target, and a creation whose target lies at, above or below the target of another replication the plan keeps or creates: YDB creates the replication over the other's replica and both write it, and a later drop of either with CASCADE drops the replica while the other runs on without it, measured on 25.1.4.7 and 26.2.1.14. Where the host negotiates planning.object-context, the planner also assesses every replication the plan keeps, and refuses a table the plan creates at the path of a live replica, and a failed-over replica the plan drops while the schema keeps the replication. A creation names its replica paths as created and a drop of a running replication with CASCADE names them as dropped, so each step is ordered against the common steps at those paths, and a creation comes before a view the plan creates over one of its replicas. A replication of this database also reads its source tables, so it is created after a table the plan creates and dropped before one the plan drops: a replication created before its source creates no replica. A transfer reads its table and, at a local source, the topic or the changefeed of a table. It names the changefeed as the changefeed planner does, so the host creates it after a changefeed the plan adds; a dropped transfer names the topic by the topic's identity rather than its path, so the drop comes before the topic's even where the plan creates another object at that path. A creation or a change of either object reads the secret its connection names by path, so the plan creates the secret first, and a source that drops the secret while it declares the object is refused. A drop of a failed-over replication runs without CASCADE and keeps its tables. The provider never pauses, resumes or fails over an object.

A grant names a replication or a transfer by its directory and name (on_async_replication, on_transfer), and GRANT and REVOKE name it by its path. The planner puts a GRANT or a REVOKE after the creation or the change of its object, and a drop replaces the REVOKE the host planned, since YDB drops an object's access list with it and a REVOKE after the drop would name a path that is gone. A change in place, such as a new lambda, keeps the entries.

Execution reads the object before the statement and refuses one that changed since the plan read it, runs the statement once, and reads the object back. YDB creates a replication's replica tables some time after the statement returns, and refuses a view over a table that is not there yet. So a creation whose replica a later step of the plan reads, such as a view over it, then waits until each of its replica tables in this database exists, at most two minutes. A table still missing then stops the run after the replication, with the diagnostic ydb.replica-tables-not-created. A replication nothing in the plan reads goes on at once. The statement names the object, the replica targets, the transfer's table and a secret path absolutely; a transfer's topic keeps its relative spelling, because 25.1.4.7 cannot find a topic of its own database named absolutely (Discovery error: ... SCHEME_ERROR). The TEXT form writes the host's statements, with paths relative to the database root. A dev realm refuses both objects, which reach past it.

On 25.1.4.7 with EnableTopicTransfer on, dropping a table a transfer wrote into, after the transfer is gone, stops the server on a schemeshard VERIFY (AliveChildrenCount >= 1); 26.2.1.14 drops it. The provider's CI therefore keeps the flag at its default on 25.1 and checks the refusal there.

A consumer a transfer reads through is the transfer's. YDB makes one with a generated name for a transfer that names none, and a declaration of the changefeed or the topic cannot name it. The read marks each consumer a transfer of the same read reads through (Consumer.transfer in the topic description, which YDB never sets), and the comparison of the topic or the changefeed keeps a marked consumer the declaration leaves out, as the host's embedded engine adopts it: the run after a transfer's creation plans nothing, and a change of the topic or the changefeed keeps the consumer. A topic operation neither checks nor confirms a marked consumer, since the transfer's drop, which the plan may run first, takes the one YDB made. A transfer read outside the selected directories marks nothing.

A declared transfer whose table or topic the plan would not leave is refused before anything is planned, in the host's embedded engine's words. The table has to be declared, where the host lists every declared table among a comparison's declared relations (models.declared-relations, contract 25.32): YDB creates no transfer into a missing table (Path does not exist). A topic of the transfer's own database has to be declared as a topic, where the host also sends the transfer comparison the declared topics (models.declared-objects, contract 25.33, ADR 0109 in Ptah), or be a changefeed of a declared table: YDB accepts a transfer of a missing topic and then stops it (Discovery error). A table's changefeeds are a facet the row-table comparison owns; a host that sends the transfer comparison the declared changefeeds of the tables (models.declared-facets, contract 25.37, ADR 0119 in Ptah) has a stream of a declared table accepted only when it is one of the table's declared changefeeds, as the host's embedded engine did. A host that does not send them leaves the comparison to take any stream of a declared table for a changefeed. A topic or a changefeed the read recorded without describing it is one the plan neither drops nor changes, so a transfer that reads it is compared, as the embedded engine compared it: a topic the read lists and does not read, such as a queue of the older persistent queue kind, and a stream of a declared table that holds a changefeed no declaration can describe, such as one whose topic has a consumer with a codec Ptah does not declare. The read leaves out the whole changefeed set of such a table, so the comparison takes any stream of it for one of its changefeeds. Only a host that sends the transfer comparison what the read holds (models.current-context, contract 25.40, ADR 0122 in Ptah) says so; without it such a transfer is refused like one of a missing topic. A target without transfers refuses the transfer by the capability first, as the embedded engine did.

Not modeled yet: HCL blocks.

TestReplicationLifecycleSubprocess creates a replication of the server's own table, finds nothing to change, refuses a connection change while it runs, makes it once the replication is paused, and drops it with its replica. TestReplicationFailedOverSubprocess drops a failed-over replication and writes to the table it kept. TestTransferLifecycleSubprocess creates a transfer from a changefeed into a table, moves a row through it, changes its lambda and drops it, and TestTransferRefusedWithoutTheCapabilitySubprocess checks the refusal on a line without transfers. TestHostAppliesAnAsyncReplication_HappyPath and TestHostAppliesATransfer_HappyPath drive ptah schema apply from YAML through the same steps. TestHostGrantsOnAnAsyncReplication_HappyPath and TestHostGrantsOnATransfer_HappyPath grant a user a permission in the plan that creates the object, keep it through a change in place and drop the object without a REVOKE. Each needs a server whose gRPC services include replication.

Resource pools and classifiers

The provider declares, compares, plans and applies resource pools and resource pool classifiers, with the syntax, the rules and the statements of the host's embedded YDB engine (dialect/ydb/ydbworkload). Both belong to the whole database rather than to a directory, and YDB keeps them behind the EnableResourcePools flag, off by default on every line: a session reports resource_pools from the cluster's flags, so the address must name the monitoring endpoint (see Feature flags).

A YAML document declares them under the document keys resource_pools and resource_pool_classifiers, a Go file with the directives ptah:schema:resourcepool and ptah:schema:resourcepool:classifier, and a YQL schema with CREATE RESOURCE POOL, CREATE RESOURCE POOL CLASSIFIER and, for the pool YDB creates with the database, ALTER RESOURCE POOL default SET:

resource_pools:
  reporting:
    concurrent_query_limit: 10
    queue_size: 20
    query_memory_limit_percent_per_node: 25.5
  default:
    resource_weight: 30
resource_pool_classifiers:
  reporters:
    resource_pool: reporting
    member_name: analysts
    rank: 100

//ptah:schema:resourcepool name="reporting" concurrent_query_limit="10" queue_size="20" query_memory_limit_percent_per_node="25.5"
//ptah:schema:resourcepool:classifier name="reporters" resource_pool="reporting" member_name="analysts" rank="100"
type Pools struct{}

A pool takes concurrent_query_limit and queue_size as whole numbers from 0, and database_load_cpu_threshold, the three *_percent_per_node limits and resource_weight as percentages from 0 to 100; a setting left out has no limit, and zero is a limit. A queue needs a query limit or a load threshold beside it, and the pool default takes neither of those. A classifier names its pool and its rank, and its member when it matches one user or group. Planning declarations without a read of the database, as schema render does, refuses a classifier whose pool the declarations neither hold nor name default: YDB takes such a classifier and runs the member's queries in default without a word. A migration keeps a classifier routing to a pool the plan drops, which YDB takes too. Each rule refuses with the host's words.

A source declares the pools and classifiers it manages, not every one of the database, so the comparison keeps one a source leaves out, as the host does; an absence a source claims for one object is left undecided. A pool compares setting by setting and a classifier as a whole. The default pool must be read before its settings change, and it is never created or dropped. A schema written as SQL, such as schema inspect --format sql, writes it as the ALTER RESOURCE POOL default SET of the settings it names, and writes nothing for it when it names none, as the host's engine did.

The planner writes each workload statement of a plan as a step of its own, outside a transaction, ordered after the steps that create or change a role and before those that drop one, since a classifier names one. The steps are chained in the host's order: classifier removals, pool removals, pool creations and changes, classifier changes, then classifier creations. YDB keeps one classifier per rank and checks it at every statement, so a classifier that moves to a rank another changed classifier leaves is dropped and created again around the move, in two steps. Each step runs after the common steps that need neither it nor a later workload step, which is where the YDB engine built into the host at 74de0b65b ran its workload statements: a pool right before the first grant on it, and a classifier after every common step. Measured on 25.1.4.7 and 26.2.1.14 with a group, a table, a topic, a pool, its classifier and grants on the table, the topic and the pool, schema apply and migrations generate create the pool after the table, then the grants, then the classifier, and the down file drops the topic, the table, the classifier, the pool and the group. In a migration file each step is one statement.

Execution of a step reads .sys/resource_pools and .sys/resource_pool_classifiers before its statement and refuses an object that changed since the plan read it; it runs the statement once and reads both views back. The workload manager reads its classifiers from a cache, so for a while after a classifier and its pool are dropped it can still route a query to the pool, and YDB refuses that query with Failed to resolve pool id before it runs it. A read of the views refused so is sent again, for up to 20 seconds, for an inspection and for an execution's check and confirmation alike; a statement is never sent twice. A classifier with no member name routes every query of the database, the server's own included. Measured on 25.1.4.7: while such a classifier routed every query to a pool that admits none, a read of .sys/resource_pool_classifiers sent to the default pool ended in System view: timeout. Inspection reads the views on a target with resource pools, for a read of the whole database, one that selects the root, /, and one filtered to other directories where the host lists schema.database-kinds: the engine names both kinds database kinds, so a filtered read reports them beside its scope, named at the database (ADR 0065). So schema apply --schemas app creates a pool its source declares, and keeps the database's pools when the source declares none, as a run of the whole database does. With a host that does not list the feature, a read filtered to other directories cannot report what belongs to the database, so it leaves both kinds uninspected; selecting the root as well (--schemas app,/) reads them. A dev realm holds none of its own.

A complete workload read records absence inside the opened database's root namespace, including when no pools or classifiers were returned. It establishes no absence in another database or a directory namespace. The host preserves that boundary when it compares a new declaration against the inspection.

TestHostAppliesAResourcePoolInARunOfADirectory_HappyPath drives ptah schema apply --schemas the fixture directory alone: it creates a declared pool, reads it as synced, keeps it when the source leaves it out, and changes it in place.

TestWorkloadLifecycleSubprocess creates a pool and two classifiers, swaps their ranks from the text of a migration file, refuses a change planned before another writer changed the pool, and removes them, reading the views back after each step. TestHostAppliesResourcePools_HappyPath drives ptah schema apply over the fixture directory and the root from a YAML, a Go and a YQL source: it creates them, finds nothing to change, changes a pool and swaps two ranks, finds nothing to change again, and keeps them when the source leaves them out. TestHostCreatesAClassifierAfterTheGrants_HappyPath pins the order of schema apply --dry-run and of the up and down files of migrations generate.

Changefeeds

The provider declares, compares, plans and applies the changefeeds of row tables and the consumers of their topics, with the syntax, the rules and the statements of the host's embedded YDB engine (internal/ydbchangefeed).

A YAML table declares its changefeeds under the table key changefeeds, and a Go file with the directives ptah:schema:changefeed and ptah:schema:changefeed:consumer on the table's struct, or on another struct with the table attribute:

tables:
  orders:
    columns:
      id: {type: Uint64, primary: true}
    changefeeds:
      updates:
        mode: NEW_AND_OLD_IMAGES
        format: JSON
        retention_period: PT12H
        consumers:
          billing:
            important: true
          search: {}

//ptah:schema:table name="orders"
//ptah:schema:changefeed name="updates" mode="NEW_AND_OLD_IMAGES" format="JSON" retention_period="PT12H"
//ptah:schema:changefeed:consumer name="billing" changefeed="updates" important="true"
//ptah:schema:changefeed:consumer name="search" changefeed="updates"
type Order struct {
	//ptah:schema:field name="id" type="Uint64" primary="true"
	ID uint64
}

The options are the host's: mode and format, which YDB requires, virtual_timestamps, resolved_timestamps, initial_scan, user_sids, schema_changes, topic_min_active_partitions, topic_auto_partitioning and retention_period; a consumer takes important, read_from, supported_codecs and availability_period. Each is parsed and refused with the host's words, and a Go consumer names a changefeed its table declares. A YAML or Go source describes the changefeeds of every table it declares, so a table it declares with none is declared to have none and the changefeeds it holds are dropped. A YQL source cannot declare one (see SQL schema reading) and leaves them as they are. A column table takes no changefeed.

The comparison reads both sides as the host's specification: a mode, a format and a codec in any case, an interval by the seconds it denotes, YDB's 24 hours for a retention left out, consumers by name and their codecs as a set, and a topic's starting partition count only where both sides name one. The statements of a change list the consumers as Ptah's built-in YDB engine did: those it drops in the order the read lists them, which is name order, and those it adds in the order the declaration lists them, so the rollback of a change drops the consumers it added in the order it added them. A changefeed the read cannot represent (an AWS region, attributes, a fraction of a second, an auto-partitioning strategy no changefeed declares, a consumer attribute other than YDB's own) leaves the table's changefeeds undecided, and so does a change to a stream an asynchronous replication controller owns, or a declaration of such a stream that does not retain the controller's binding, as a desired side converted from a read retains it; such a stream the declaration leaves out is kept. A binding is read only with every field, an absolute replica path, a canonical positive item number and no other key; any other is an attribute the read cannot represent. YDB drops a table whose stream a controller owns and leaves the replication failing (Discovery for all topics failed), so such a table is dropped only with a replication of this database the plan drops too. The table planner refuses the drop where the binding names a replica outside the database, in a database nested in it, or at a path where this database holds no table, which is a database of the same name on another server; the read lists the replica's path to tell. The replication planner refuses it where the plan keeps the replication.

The table lifecycle service plans every changefeed change of a table as one step, outside a transaction: the host runs a step that drops an object after every step that creates one, and YDB keeps a limit on the changefeeds of one table, so a swap at the limit and a changefeed dropped and added again need one step. The operation runs the host's statements: ALTER TABLE ... DROP CHANGEFEED for every removal, then ALTER TABLE ... ADD CHANGEFEED with an ALTER TOPIC ... ADD CONSUMER per consumer for every addition, then ALTER TOPIC for a retention or a consumer that changes in place. YDB changes no option of a changefeed in place (MODE alter is not supported), so such a change drops the changefeed and adds it again, and a consumer whose codec list the declaration empties is dropped and added again. A disabled changefeed the declaration states disabled, as one converted from a read does, keeps its state: its topic changes in place, since ALTER TOPIC disables nothing, and no step adds a disabled changefeed, since no statement disables one. A new table's changefeeds follow its creation. The planner refuses an option the session lacks with the host's capability key: changefeed_user_sids before 26.1, changefeed_schema_changes before 25.3, changefeed_topic_auto_partitioning before 25.2 or with the feature flag off, and a consumer's availability_period before 25.4. It refuses the shapes YDB refuses on every line, a topic of several partitions on a first key column other than Uint32 or Uint64, a changefeed named after an index of its table, and a changefeed and an index that trade a name in one plan. A removal or a recreation, and a consumer dropped or restarted, is destructive.

Execution reads the table and each changefeed's topic before the first statement and refuses a changefeed that changed since the plan read it; it runs each statement once and reads the table back, requiring each declared changefeed with its topic's retention and consumers, or a removed one's absence. The one statement that runs again is one YDB refuses because another schema operation holds the table's path (OVERLOADED, path ... has been locked by tx or path is under operation), which changed nothing: a changefeed added with an initial scan holds the path until the scan has read every row, so the next statement waits, 50 ms at first and up to a second between attempts, until it runs or the request's context ends. Any other unconfirmed result is UNKNOWN and is never retried.

In the TEXT form a changefeed operation is the statements the host's embedded engine writes into a migration file for the same change, one to a line, in its order: the removals, a changed changefeed among them, then the additions, each with the consumers of its topic, then each topic's change in place. A migration runner runs each statement alone, so an ALTER TOPIC finds the topic the ADD CHANGEFEED above it created. A migration file then carries a changefeed change, and a rebuild of a table that holds changefeeds.

TestChangefeedLifecycleSubprocess adds a changefeed with an initial scan and two consumers, restarts a consumer, refuses a change planned before another writer changed the topic, drops and adds the changefeed for a mode change, and drops it, reading the server back after each step. TestHostAppliesChangefeeds_HappyPath drives ptah schema apply from a YAML and a Go source: it creates a table with two changefeeds, finds nothing to change, changes a topic in place and recreates a changefeed, finds nothing to change again, and drops both.

External data sources and external tables

The provider declares, compares, plans and applies YDB external data sources and the external tables over them, with the syntax, the rules and the statements of the host's embedded YDB engine (dialect/ydb/ydbexternal). One comparison service and one planner own both kinds, because a data source the plan drops and creates again takes the tables over it along.

A YAML document declares them under the document keys external_data_sources and external_tables, a Go file with the directives ptah:schema:externaldatasource and ptah:schema:externaltable, and a YQL schema with CREATE [OR REPLACE] EXTERNAL DATA SOURCE and CREATE [OR REPLACE] EXTERNAL TABLE; OR REPLACE is not part of the declaration, because whether a plan replaces an object is the plan's decision:

external_data_sources:
  events_bucket:
    schema: ext
    source_type: ObjectStorage
    location: https://storage.example.test/events/
    auth_method: NONE
external_tables:
  events:
    schema: ext
    data_source: ext/events_bucket
    location: 2026/
    columns:
      - {name: id, type: Int64, not_null: true}
      - {name: kind, type: Utf8}
    options:
      FORMAT: csv_with_names
      CSV_DELIMITER: ";"

//ptah:schema:externaldatasource name="events_bucket" schema="ext" source_type="ObjectStorage" location="https://storage.example.test/events/" auth_method="NONE"
//ptah:schema:externaltable name="events" schema="ext" data_source="ext/events_bucket" location="2026/" columns="id Int64 NOT NULL, kind Utf8" options="FORMAT=csv_with_names;CSV_DELIMITER=\;"
type Externals struct{}

A data source takes name, schema, source_type, location, auth_method and options; a table takes name, schema, data_source, location, columns and options. A Go options attribute is NAME=value pairs separated by ;, where \; writes a semicolon and \\ a backslash, and a Go columns attribute is name Type [NOT NULL] entries separated by commas. Option names are read in upper case; an option that has an attribute of its own, REFERENCES, which the server keeps, a column with DEFAULT, PRIMARY KEY or FAMILY, and an object declared twice are refused with the host's words. Each source describes the external objects of the directories it reads completely, so an object it leaves out is dropped; a ptah:not-described external_data_source or external_table header entry of a YQL document keeps one. HCL has no block for either kind.

The comparison reads both sides as the host does, against the database the current side was read from: source type, location and auth method exactly, options by upper-case name, and a *_SECRET_PATH option relative to the database, which the server keeps absolute; a table's data source relative to the database, its location, its columns in order with their types and nullability, and its options, with PARTITIONED_BY read as the list it holds. REFERENCES and the access metadata are not compared. A declaration on a target without external_data_sources is refused, and so is a plan that drops a data source a declared table reads. A data source dropped and created again, or replaced with a new source type, takes each declared table over it along, as a change whose operands may describe the same table.

YDB alters neither object (Alter operation for EXTERNAL_DATA_SOURCE objects is not implemented), so the planner replaces a changed one: with CREATE OR REPLACE where the session reports external_object_replace, and otherwise by a DROP and a CREATE. A data source dropped and created again is one operation; a table is a drop before the statement on its data source and a restore after it. Tables are dropped before the data sources they read and created after, and every step asks for the host's dependent phase, the one window creations and drops share. YDB refuses to drop or replace a data source a column table's TTL names, so the TTL of a column table the plan keeps is reset before a replacement of such a source and set again after it, as the host's embedded engine carries it (a target without tiered_ttl refuses the replacement), a column table the plan drops is dropped before the sources its TTL names, and a plan that leaves a table with a TTL naming a data source it drops is refused (its TTL reads external data source ..., which the plan drops). The planner refuses, with the host's words, every statement on a target without external_data_sources, an option ending in _SECRET_PATH before 25.4, where the session reports external_data_source_secret_paths false, a secret path outside the database, two objects at one path, and a table over a data source the batch writes that is not ObjectStorage. No statement holds data YDB stores, so a drop or a replacement is behavioral and never destructive.

Execution reads the object before the first statement and refuses one that changed since the plan read it, REFERENCES aside, or a path that holds something else. It runs each statement once, gives an object dropped and created again the owner and the direct access entries the old one held, and reads the object back, requiring the declaration, the owner and the access entries, or the object's absence after a drop. An unconfirmed result is UNKNOWN and is never retried. A dev realm session refuses both kinds, which reach past the realm.

TestExternalLifecycleSubprocess creates a data source and a table over it, moves the data source with access entries on both objects, refuses a change planned before another writer changed the table, and drops both, reading the server back after each step. TestExternalReplacementSubprocess turns EnableReplaceIfExistsForExternalEntities on through the server's dynamic configuration and replaces the data source, which keeps the table, and then the table in place, each keeping its owner and access entries. TestHostAppliesExternalObjects_HappyPath drives ptah schema apply from a YAML, a Go and a YQL source: it creates both objects, finds nothing to change, moves the data source, adds a column to the table, finds nothing to change after each, and drops both.

Common column semantics

Inspection requires the negotiated schema.observed-columns feature. Row and column tables report complete common column facts, including decimal precision and scale, literal default presence, and serial start and increment settings. The last sequence restart remains observed evidence; it is not a request to restart the sequence during source conversion. Native column facets retain the exact settings needed to check execution before-state. Unknown or contradictory native metadata refuses the complete inspection reply.

Column default changes use captured native before-state and typed values. The provider plans one ALTER COLUMN SET DEFAULT or DROP DEFAULT, outside a transaction, and verifies the column before and after execution. An empty string is a present literal. A stale before-state refuses execution; an unconfirmed write has an unknown outcome and is not retried. The session reports alter_column_default from the release line and EnableSetDropDefaultValue.

A default change, a column made nullable, a column removal and a sequence change each hold the native column they change. The side the host builds from a schema file for a diff of two files states its column facets completely and carries one only for a column whose declaration states a native setting, so a column without one is held as the common column says, its common default read back as the native literal. A diff of two files plans these changes offline, as Ptah's built-in YDB engine did. A side that does not state its column facets refuses them.

The parent planner derives column additions and removals from the complete captured table. Nullable additions without defaults use native AlterTable. Additions with typed literal defaults use one pinned YQL ADD COLUMN; existing rows receive the default, including an empty string for a NOT NULL column. They require add_column_with_default, established from the release line and EnableAddColumsWithDefaults. A NOT NULL addition without a default is refused. A removal checks its captured column definition and refuses key, index, TTL, or partitioning dependencies before dispatch. Each operation verifies the remaining columns and table settings after its single write. The graph orders additions before the default and sequence changes, and removals after them. The table's family, TTL, partitioning, index and changefeed steps are outside that sequence. Each carries its own edges to the column steps it depends on, and otherwise runs where the host places it: the host drops an index before it adds a column.

The negotiated schema.columns service compares the exact columns the host pairs inside their full captured tables. It reports type and default equivalence, effective nullability, and sequence start and increment differences. Primary and serial columns have effective NOT NULL; the original declaration is retained. Readback distinguishes absent defaults, empty literal values and expressions. Creation projection reports every effective column in declaration order, including its typed literal default and comment.

Table compilation maps supported SQL aliases using the selected target's capabilities. Row-table literal defaults compile into typed native values; expressions are refused. A default has one source declaration, either a common column field or native column settings. Wide date/time types and parameterized Decimal follow the cluster flags; small integer and document defaults follow the measured release line. A column table accepts no defaults or serial columns. Serial columns are created with their sequences and the sequence of an existing Serial changes in place; see Serial columns.

Offline table preparation

The experimental schema.prepare-tables service derives column key membership from a complete captured row-table declaration. It uses the same native compiler as CREATE, including ordered composite keys and native properties. Source values, observations, and explicit existence knowledge remain unchanged. The service opens no session and refuses the whole batch when any declaration is unsupported. It does not predict server defaults or enable the host's common SQL table planner.

Offline CREATE prediction

The experimental schema.project-table-creations service predicts column key membership and effective desired table properties from complete declarations. It uses the native CREATE compiler, including composite key order and authored settings. Computed values remain on the captured table owner. Explicitly excluded table properties stay excluded, and source declarations remain unchanged. Offline representation conversion preserves an absent database binding. Live capture still requires the database that supplied the observation.

Unspecified server defaults remain unspecified. The service returns no observed overrides and makes no claim that a table exists or a migration ran. Both process boundaries validate complete batches, identities, exclusions, and key membership. Live tests feed the prediction into native planning, execute CREATE, and inspect the resulting table on both qualified server lines.

Offline schema validation

The experimental schema.validate service checks the captured desired schema before the host plans. The host asks for it and shows the reply to the user; without the service ptah schema plan stops with target "ydb" has no schema validation service. The service reads only the request. It opens no session and loads no source file.

Each table declaration is compiled with the CREATE rules, so validation refuses what planning would refuse: a missing or unknown key column, a type with no YDB counterpart, a table without columns, populated declaration fields and facets the provider does not carry (including common indexes), and invalid names. Access requirements and temporary tables are valid declarations that only CREATE refuses, so validation accepts them. A setting the target takes only with a capability is refused where the request's capabilities lack it: the settings of a Serial's sequence, and a family's cache mode, which the host's bundled validation refuses for --server-version 25.1 and 25.3 and accepts from 25.4. Table paths that coincide or nest are refused across the schema, because a YDB table holds no children. Directory names are checked as path elements. The native models in the whole-schema capture must normalize.

A refusal of the declaration is a diagnostic in a completed reply. Cancellation, an identifier context the provider cannot interpret and a capture whose receipt differs from the provider's own normalization are errors and carry no diagnostics. Diagnostics are sorted, so the same schema gives the same reply in any capture order. A key, type or unsupported-field refusal is UNSUPPORTED_FEATURE, as in the host's bundled YDB validation; other refusals are INVALID_SCHEMA. A standalone enum, and a directory comment, character set or collation, are reported as unsupported and OMITTED_DECLARATION only when the request asks that nothing be skipped, which is also the host's behavior. So is what an offline render of a table's creation goes without (see Offline schema rendering), in the host's bundled validation's words: ydb: column "app.accounts.email": type modifier=VARCHAR(255): length 255 would be skipped and identity start=100 would be skipped.

internal/schemavalidation/testdata/corpus holds declarations and what the host's bundled validation answered, recorded by testdata/oracle (the program header says how to run it against a host checkout). Rows named refused- agree with the host on code, deferred- rows are ones the host accepts and this provider refuses, and the others agree on acceptance and on what would be skipped. The provider imports no host code. A subprocess test runs the service through the SDK client, and a host contract test (PTAH_HOST_TEST_BIN) checks that the host reaches the service and shows its refusal.

Native row-table creation

The SDK planner accepts captured CREATE lifecycles for row tables. It retains ordered composite primary keys, native column types and nullability, TTL, storage settings, column families, attributes, partitioning settings, starting layouts, bloom filters, read replicas, and tiering. Column families and typed literal defaults come from native column facets. Missing creation settings request server defaults. The common and native primary key declarations must agree on order. A declaration that orders no key, such as a YAML table with two columns marked primary: true, is keyed by the columns it marks primary in the order it declares them, as the host's embedded engine keys it; one that marks none is refused.

A family's cache mode travels as field 6 of the request's family, its upstream wire number, which the pinned Go bindings do not declare and keep as an unknown field; the operation payload writes it as the family's cacheMode, its upstream JSON name. The table service of 25.1.4.7 does not know the field and creates the table without it, so a creation that states one is refused where the session reports no column_family_cache_mode, and the creation is confirmed by the cache mode the table reads back with.

A starting layout becomes the partitions of the CreateTable request, checked against the key columns: uniform_partitions needs a Uint32 or Uint64 first key column; a split point holds at most one value per key column, each value a whole number in the range of its integer column or a string for a Utf8 or String column, and no other key type takes one. Each value travels as an optional element of a tuple over the leading key columns. Split points out of order are left to YDB, which refuses them. The creation is confirmed by the minimum partition count YDB sets from the layout, unless the declaration states a minimum of its own. The operation payload is checked again against the same rules, so a forged request cannot state a layout the rules refuse.

The planner can replace a complete common CREATE TABLE step in the shared graph. Its replacement carries table and physical-path effects and forbids SQL transactions. A refusal returns no successful planning prefix. Rendering emits a native operation envelope and validates its safety metadata again.

Execution establishes absence through the complete listing of the parent directory, or through the proof that the directory does not exist, which CreateTable then creates with the table. An existing path is refused, including an existing table with the same schema. The provider binds the session and absolute path, checks its guarding leases, sends CreateTable once, and reads the result back. It confirms columns, key order, requested settings, and the absence of unexpected indexes or changefeeds. A failed reply or unconfirmed result remains UNKNOWN. The native API provides no atomic compare-and-set; these checks do not establish exclusive ownership against another schema writer.

A table is created with the global indexes it declares, synchronous or asynchronous, unique or not, with their key and covered columns, in the one CreateTable request, as the host's embedded engine writes them inside its CREATE TABLE. That is the only way to give a new table a unique index on a release that refuses one on an existing table. What a YDB index has no clause for is refused in the embedded engine's words: an expression, an order, a prefix, a NULLS order or an operator class on a key column, an index on the table's key, a covered key column. A vector, full-text or local index is refused as a common index, since the pinned table protocol builds none of them with a table. An index names its table as the host names it: by its name, by its directory and name, or, where a part holds a dot, by the quoted reference "app.records" for the root table app.records and app."v.1" for the table v.1 of the directory app. Execution confirms each index by reading it back. In the TEXT form each index is an INDEX clause of the CREATE TABLE.

YDB has no UNIQUE constraint: it refuses one as a parse error and holds the same rows with a global unique index, a second row with the same value refused and rows whose value is NULL not. A session reports unique_constraints as false, and a host that reads it so turns a declared UNIQUE constraint, and a column's UNIQUE, into that index before it compares and plans, under the name its YDB renderer gives it: the constraint's name, or <table>_<columns>_key. A UNIQUE over the key folds into the key. The provider then creates and reads an index, never a constraint. Schema validation and the creation projection read the source the operator wrote past its UNIQUE, since the index changes no column of the table; a UNIQUE the host does not turn into an index, such as a deferrable one, is refused when it is planned.

The vector and full-text indexes of the table's native index set are built by the creation itself, with the rules of Vector and full-text indexes, as the host's embedded engine built them: the native CreateTable call carries each one as the upstream arm of its index, and the TEXT form, an inspection and a migration file write every index of the table inside its CREATE TABLE, in name order, the vector or full-text one as INDEX name GLOBAL USING vector_kmeans_tree ON (...) WITH (...). YDB then builds no index over rows the table holds, so the operation service lists no index build for the table. A target that cannot build one refuses the whole creation before the table is created. A global index the native set states is refused at creation: a new table takes its global indexes from the common declaration.

Access requirements, temporary tables, changefeeds, column compression or encoding, and richer common constraints are refused. Removal and rebuild are described below, and alteration covers column families, the addition and removal of columns, the TTL, the automatic partitioning settings, the read replicas, the key bloom filter, and the partitioning and read replicas of an index (see below). An unknown populated field cannot disappear during compilation. A mixed SQL/native batch is validated before its first unit executes.

Subprocess tests cover creation, inspection, stale plans, retained table data, and refusal of a malformed later unit on YDB 25.1.4.7 and 26.2.1.14. Unit tests exercise lost replies, lost confirmation, lease failure, and session cleanup. With the validation service in place, ptah schema plan through this provider gets past validation and stops at the host's common-step stage ("A common step is outside the supported table creation lifecycle"), because the host's common table comparison does not yet preserve complete native declarations. These SDK checks do not establish a complete native CLI table migration workflow.

Serial columns

A column is a Serial when its type is Serial, BigSerial or SmallSerial (or one of the spellings YDB shares with PostgreSQL: SERIAL, SERIAL4, BIGSERIAL, SERIAL8, SMALLSERIAL, SERIAL2), or when an Int16, Int32 or Int64 column is declared auto_increment or with an identity, as the host's embedded engine reads it. The host sends the start and the increment of its sequence with the common column, as identity_start and identity_increment: Go and YAML write them as attributes of the field, and a YQL schema as the START WITH and INCREMENT BY of an ALTER SEQUENCE on the sequence's absolute path. No native facet holds them. A Serial is NOT NULL whatever the declaration says, and its default is its sequence.

//ptah:schema:field name="id" type="BIGSERIAL" primary="true" identity_start="100" identity_increment="5"
//ptah:schema:field name="id" type="BIGINT" primary="true" auto_increment="true" identity_increment="3"

The native CreateTable call creates the column with its sequence, named _serial_column_<column>, starting at the declared start, stepping by the declared increment and ending at the maximum of the column's type. Measured on 25.1.4.7 and 26.2.1.14, a sequence created this way issues its start first and records no restart. The call gives a sequence the Int64 maximum unless the request names one, whatever the column's width, so the request names the maximum YQL's Serial and SmallSerial have. Execution confirms the sequence by reading it back, and the operation payload is held to the same sequence, so a forged request cannot name another one, count down or cycle.

The TEXT form writes the column as its Serial type and, for a start or an increment other than 1, the ALTER SEQUENCE the host's embedded engine writes after the CREATE TABLE, with RESTART WITH the start where it is not 1: START alone leaves the next value at 1. The statement names the sequence by its absolute path, so a creation that names no database, such as an offline render's, is written without it, and the render reports the start and the increment as omissions, as the host's embedded engine did (see Offline schema rendering). A plan against a database carries the path of the database it read in the creation, since the host names the table it plans by no database, so a migration file holds the statement, and YDB records the restart, which it replays on every later ALTER SEQUENCE. A table created from such a file therefore keeps the settings it was created with (see below), where a table the native call creates can change them.

The planner refuses, in the embedded engine's words, an auto-increment column of a type YDB has no Serial for, GENERATED ALWAYS, identity options other than the start and the increment, a default on a Serial, a start or an increment below 1 or not a whole number, and a start or an increment a creation or a change would give a 16-bit or 32-bit Serial, a change back to the defaults included. Every line reports serial_sequence_keeps_range false: any ALTER SEQUENCE raises such a sequence's maximum to the Int64 maximum, and the column then stores the value after its own maximum as a negative number without an error. The native call could create such a sequence with its range kept, but no later change and no migration file could, so it is refused at creation too. A table whose 16-bit or 32-bit Serial already holds the declared start and increment needs no statement and plans nothing, and validation, which cannot tell, takes the declaration, as the host's embedded engine does. A column table takes no Serial, and YDB adds no Serial column to a table that exists (Column addition with serial data type is unsupported).

When the declared start or increment of an existing Serial differs from what the read reports, the planner writes one step, serial-sequence/N: one YQL ALTER SEQUENCE ... START WITH ... INCREMENT BY ... on the sequence's absolute path through the query service, outside a transaction and without a restart, since a restart onto a value a row holds fails the next insert. The table service has no call that changes a sequence: an AlterTable that names it answers SCHEME_ERROR. Execution reads the column first and refuses one that changed since the plan (NOT_STARTED), sends the statement once, and reads the sequence back. The next row still takes the value the sequence computed before the change: with an increment of 3 moved to 7 after the rows 1 and 4, the next rows are 7 and 14. The planner refuses to change a sequence some ALTER SEQUENCE restarted, since the next insert would take the restart's value again (Conflict with existing key), and the sequence of a 16-bit or 32-bit Serial. A column that becomes or stops being a Serial changes type, which only a rebuild makes, and a rebuild of a table with a Serial is refused.

The session reports serial_columns and serial_sequence_options on every line, and serial_sequence_keeps_range on none. Subprocess tests create a Serial by the native call and by its TEXT form and change a sequence both ways; host tests apply Go and YQL schemas with Serial keys, read the sequences back, insert rows, change an increment, and show each refusal against the server state that makes it necessary, on 25.1.4.7 and 26.2.1.14.

Native table removal

A table the declaration no longer holds, which the host plans to drop, is dropped by this provider: the planner replaces the host's steps for the table with one removal step, and its receipts name it for every table facet. A row table goes with its indexes and changefeeds, and a column table the same way. The removal is written as YQL's DROP TABLE, as the host's embedded engine writes it, in either form: execution runs it through the query service once, outside a transaction, and confirms that the path is gone. A table whose read records its changefeeds as uninspected or unrepresentable, and carries none, is not dropped (dropping table "app/t": the changefeed namespace is not fully described: ...): DROP TABLE takes every changefeed with it, and the plan cannot tell whether a replication reads one.

Before the statement is sent, execution reads the path: it must still hold a table of the store the plan read, with the columns the plan read, in order, and the same key columns. A table that changed or went since the plan is refused with nothing dropped (NOT_STARTED). This is not an atomic compare-and-set: the native API has none. The live tests drop a row table natively and by text, a column table, and refuse a table a column was added to after planning.

Native table rebuild

YDB changes neither a table's key nor a column's type in place, makes no column NOT NULL, and takes a starting layout only when it creates a table. With the host's planning policy asking for a rebuild (contract 25.3, planning.policy; --allow-table-rebuild on the native CLI), the planner makes such a change the way the host's embedded engine does, in five steps and one more for each index whose held settings differ from a new index's (see below): it creates the table anew from the declaration under a scratch name (__ptah_rebuild_<table>), copies the rows with one INSERT ... SELECT, renames the old table out of the way (__ptah_replaced_<table>), renames the new one into its place, and drops the old one. Where a table the declaration holds or the plan drops already has a scratch name, the rebuild takes the next free one (__ptah_rebuild_<table>_1 and on). The host's steps for the table are rewritten to the rename that puts the new table in place. A REBUILD the host proposes is planned the same way, and refused like any other rebuild without the policy.

A key change reaches the planner from a host that lists schema.observed-primary-keys and planning.primary-key-changes (contract 25.26): the read reports each table's key in key order, under the name <table>_pkey the host's embedded engine gave it, so a declaration that orders the same columns otherwise is a key change. The host hands it over as an alteration of the table, and the planner rebuilds the table with the declared key, or without the policy refuses it as the embedded engine did: adding constraint <table>_pkey, the primary key, which requires target capability primary_key_alterable, unavailable on this ydb target. The reverse plan rebuilds the table with the key it held.

The copy reads each column as the new table declares it. A changed type is converted with CAST and unwrapped, so a value that does not convert fails the copy with a message naming the table and the column instead of becoming NULL; a column made NOT NULL is unwrapped too, which fails the copy on a NULL row. The copy is one data query, so YDB's refusal shows that nothing of it applied: schema apply stops PARTIAL after the new table's creation, and the reason carries the message (see Error reasons). The new table keeps the old table's attributes, and with them the comments Ptah keeps there, and each storage setting the declaration does not state, as the old table holds it: every column family, the default one and one no column sits in included, with the settings the declaration leaves out (its cache mode among them, which a target without column_family_cache_mode refuses to write), the family each column sits in where the declaration does not say (see Column families), the automatic partitioning, the read replicas, the key bloom filter (KEY_BLOOM_FILTER = DISABLED for a table that states none, as the host's embedded engine names it), the TTL where the declaration states no table properties, and the run interval of the TTL it holds where the declaration states a TTL, which names none since YQL has no setting for one. A cluster's table profile may have given the old table any of them, and YQL drops neither a family nor a setting, so a rebuild changes none that nobody declared. Each index of the new table takes the automatic partitioning and the read replicas its own facet declares (see Index settings from a source), and for every setting the facet leaves out what the old index under its name holds, or the old index the host renames it from, as the host's embedded engine resolves a rebuilt index's declaration. An index the rebuild adds takes its declared settings over the ones YDB gives a new index. The call that creates a table takes no index settings, so a table-rebuild/N/index-settings/M step after the creation and before the copy gives the index every setting with ALTER TABLE ... ALTER INDEX ... SET, over the settings YDB gives a new index. The read describes each index the common model describes beside the native index set for that reason. A facet for an index the new table does not create refuses the rebuild. Without the policy, the change is refused in the embedded engine's words, naming the request the host passes, such as --allow-table-rebuild.

The host's own steps for the table, such as the addition or the removal of an index or a column, are rewritten to the rename that puts the new table in place, and that rename names every object they change, as the host's plan graph requires. An index the plan drops and adds again under its name, such as one whose columns change, cannot be named twice by one step, so the removal, and every removal the host orders before it, is rewritten to the rename that moves the old table out of the way. A host step that changes something before such a removal refuses the rebuild.

Every step writes as a YQL statement in the TEXT form, so a migration file holds the rebuild, unless the old table holds user attributes or a TTL run interval other than YDB's own: YQL's CREATE TABLE sets neither, so such a rebuild has no TEXT form (NO_TEXT_FORM) and runs natively. A native execution checks each step's precondition first: the copy needs the new table empty and the source holding every copied column, a rename needs the table holding the captured columns and key, and a free target. YDB renames no table that carries a changefeed, so a rebuild of a table that holds changefeeds drops them in a step before the old table moves and gives the new table its changefeeds in a step after it is in place: the ones the declaration states, or the ones the old table held where the declaration does not state them, with their consumers. Each stream restarts. The rebuild's first step carries the note, after the rebuild's own notes, in the host's embedded engine's words: Changefeed feed of table app/cf is dropped before the swap and added again after it, because YDB moves no table that carries a changefeed, and a changefeed the new table does not get is noted as dropped with its topic. A changefeed an asynchronous replication controls refuses the rebuild, as does a column table, whose rebuild needs an explicit data migration. So does a read that does not say which changefeeds the table holds, since the rename would fail after the new table is created and filled: a read that captured none while the declaration states some, a read whose coverage records them as uninspected or unrepresentable and carries none, as the host records them for two HCL documents (the changefeed namespace is not fully described: ...), and a changefeed the read cannot represent. A native execution gives the new table the access control entries the old table holds before it copies the rows, as the host's embedded engine gave a rebuilt table back its grants: the new table is created with its directory's permissions alone, and the old one is dropped with its own. A grant or revoke the plan makes on the table runs after the rebuild, on the new table. A migration file runs later and reads no server, so under planning.table-grants (contract 25.33, ADR 0111 in Ptah), where the host sends the grants the read found on each held table, the TEXT form grants them on the new table after the old one is dropped, where the host's embedded engine wrote them, one GRANT to a line: every grant except a privilege the table holds by default and one the plan revokes. They follow the drop so the rebuild's statements stay the consecutive steps the migration lint reads as one rebuild. A read that did not describe the table's grants leaves the file without them, and a note above the rebuild says so; a host that does not send the grants leaves the new table with its directory's permissions. The live tests rebuild a table natively and by text on both release lines, read the copied rows back, and fail a copy whose value does not convert.

A table whose declaration holds a Serial column is not rebuilt: the new table's sequence would start at 1 while the copied rows keep their values, so the next insert would collide (Conflict with existing key), and ALTER SEQUENCE ... RESTART WITH takes only a literal, so no statement can move it past the copied rows.

Secrets

The ydb-secrets comparison and planning services manage scheme-secret presence. Inspection records the secret's path without reading its value. A declaration names an approved environment input:

secrets:
  pg_password:
    schema: app
    value_env: PTAH_SECRET_PG_PASSWORD

The equivalent Go annotation is //ptah:schema:secret name="pg_password" schema="app" value_env="PTAH_SECRET_PG_PASSWORD". A YQL schema declares CREATE SECRET \app/pg_password` WITH (value = $PTAH_SECRET_PG_PASSWORD);. Schema sources refuse inline values. Input names must start with PTAH_SECRET_` and contain only ASCII letters, digits, and underscores. A refused declaration gives the reason a host prints, in the words of Ptah's built-in YDB engine: the secret, by its YAML key, the attribute and what it takes. It never repeats the value a declaration wrote or the variable it names:

provider decode source failed (InvalidArgument): secret "pw": invalid value_env: the variable it names does not start with PTAH_SECRET_ and a name after it; Ptah reads a secret's value only from a variable under that prefix, so a migration file cannot copy any other variable of the machine that applies it

An absent secret is created. An existing secret is unchanged unless the operator requests rotation with --rotate-secret app/pg_password. Rotation of an absent declared secret creates it. A rotation request must name a secret retained by the selected desired schema. An omitted secret is removed only when desired and current coverage establish that removal. A declaration that omits a secret a declared object still reads by its path is refused: YDB drops a secret an external data source, an async replication or a transfer names and keeps the object, which then names a secret that is gone.

The units.sensitive-inputs service reports input names before approval. Previews and saved operations contain references only. Immediately before execution, the host checks those references again and resolves the approved inputs. The provider receives ephemeral bindings and never reads its process environment for secret values. A missing binding refuses the entire batch before its first write; an explicitly empty value is accepted. SQL migrations can use these references as the complete value of CREATE SECRET or ALTER SECRET. Expressions, assignments, and references in unrelated statements are refused. Query errors redact expanded values.

Native execution checks presence before dispatch and again afterward. Rotation also confirms that the owner and permissions remain unchanged. Values cannot be read back, so these checks cannot detect a concurrent value rotation. Secret operations run outside SQL transactions and are never retried automatically. Targets without the secrets capability refuse declarations and mutations. Any secret names still returned by their directory listing remain unknown observations; they cannot become planned removals.

External data source planning records its old and new secret-path dependencies. Creation follows its secret's creation, and a reference is removed before its old secret is dropped. Absolute references must resolve inside the selected database or dev realm. The creation or change of an async replication or a transfer reads the secret its connection names by path, since YDB looks the secret up when it creates the object or changes its connection, so it follows the secret's creation; a path outside the database is refused.

Native table alteration

The same planning service (ydb-row-table-create) accepts captured ALTER lifecycles for row tables. It owns five facets of an altered table: its column families, the addition and removal of its columns, its TTL, its settings (the automatic partitioning, the read replicas and the key bloom filter), and its global secondary indexes with their partitioning and read replicas. Each becomes native AlterTable calls, except the settings of an index, which the native call cannot state and YQL's ALTER INDEX changes. Everything else about an altered table stays with the host: other table settings, and any column step the native call cannot state. The service also plans the changefeeds of a new or an altered table (see Changefeeds).

Column families

The service reads the declared and the observed table and column properties, and plans one call that adds families, changes the settings the declaration states, and moves columns between families. The rules follow the host's family rules:

• A setting the declaration leaves out is kept as the table holds it. A cluster profile gives new tables settings of their own, so the table is not reset. A declared COMPRESSION_NONE matches a family that reads back unspecified.
• A cache mode the declaration states is set where the family holds another one or none. A family that never had one reads back with none, so a declared regular is set once and then holds, as the host's embedded engine sets it. The request carries it as field 6 of the family, as a creation does (see Native row-table creation). A declared cache mode, regular included, is refused on a target without column_family_cache_mode, with the flag's name, whatever the table holds: the table service of 25.1.4.7 would alter the table without it.
• A family only the table holds stays. No family is dropped, because YDB has no way to drop one.
• A column sits where the declaration places it. A column property record names its family, and an empty family in a record means the default family. A source that describes the column properties of every column it declares (a Go, YAML or YQL source) places a column it names in no family in the default family, so a column the table holds elsewhere moves back, as the host's embedded engine moves it. A source that could not describe them, such as HCL, says nothing of where a column sits, and the column keeps its family. A rebuild places the columns of the new table by the same rule.
• A key column stays in the default family. A column that names a family neither the declaration nor the table has is refused. So is a keep_in_memory that only one side sets, and a declaration that states families when the table's current families were not captured.
• A column the plan removes is never named, so it is never moved.

A table whose families already match yields no family step and receipts that say so.

Column addition and removal

The host plans one common step for each column it adds or drops. The service replaces such a step with a native step of its own, so the column steps and the family step form one graph with one owner.

• A step is replaced when its footprint is a read of the table and the creation or removal of one column of that table. A step with any other footprint stays with the host.
• An added column must be nullable, with no default, key membership, unique, check, reference, generated, identity, collation or comment, and no column property but its family. YDB refuses a NOT NULL column on an existing table (measured on 25.1.4.7 and 26.2.1.14), and accepts a default on 26.2 only. The host keeps the step for any column outside this subset.
• An added column starts in the default family. A column property record that names another family moves it in a separate family step, which runs after the addition.
• A dropped column must not be one the declaration states. The service refuses a request in which the host drops a declared column, adds an undeclared one, or changes one column twice.
• Each replaced step becomes one native call, named row-table-add-column/N or row-table-drop-column/N. The family step is row-table-alter/N. Ordering among the host's steps is kept, and the family step runs after the addition of every column it moves and before every removal. Both ends of those edges are steps of this provider. Where a step stays with the host, the edge names the host step.

TTL

The service sets, changes and removes the TTL of an existing row table with one native AlterTable step, row-table-ttl/N. A step carries the whole TTL the table holds afterward (set_ttl_settings) or removes it (drop_ttl_settings), and also the TTL the table held when the plan was made.

• The TTL is managed when the declaration states table properties for this target, or when the desired coverage establishes the table properties of the table: complete knowledge of the kind, or an absence or a request for defaults recorded for the table. A table the declaration leaves without a ttlSettings then requests none, and a TTL the table holds is removed. Unknown or unrepresentable desired knowledge with no table properties leaves the TTL alone. A declaration that states table properties while the table's current TTL was not captured is refused; a table managed by its coverage alone, whose current properties were not captured, is left alone.
• The declared and the held TTL are the same when the mode, the column, the unit and the seconds are the same. A runIntervalSeconds the declaration leaves out is not compared. A step that replaces a TTL carries the run interval the table held unless the declaration states one, because the call resets an interval it does not carry (measured on 25.1.4.7 and 26.2.1.14). The server refuses a run interval below its configured limit, 900 seconds on the test servers, after the call; the plan does not check it.
• The column must be declared, with a type the mode reads: a Date, Datetime, Timestamp, Date32, Datetime64 or Timestamp64 column for a date TTL, and a Uint32, Uint64 or DyNumber column with a unit for an integer TTL. The server refuses each other combination only after the call, so the plan refuses it. A key column is allowed.
• The TTL step runs after the addition of the column it reads, and before the removal of the column the table's TTL reads now, whoever carries those steps out. The server refuses a TTL on a column the table does not hold, and refuses to drop the column a TTL reads. The family step and the TTL step do not depend on each other.
• Impact is behavioral for a set and for a removal: a TTL decides which rows YDB deletes, and restoring a prior TTL does not recover deleted rows.
• The host capabilities RowDeletionPolicy and RowDeletionPolicyEpochColumn are not session capabilities of this provider. The host keeps refusing a TTL its target cannot write.

Partitioning settings, read replicas and key bloom filter

The service changes the automatic partitioning settings, the read replicas and the key bloom filter of an existing row table with one native AlterTable step, row-table-partitioning/N, as the host's embedded engine changes them with one ALTER TABLE ... SET. The step carries only the settings that change, and also the value the table held for each group it changes: the whole partitioning, the filter, or the replicas. The settings are AUTO_PARTITIONING_BY_SIZE, AUTO_PARTITIONING_BY_LOAD, AUTO_PARTITIONING_PARTITION_SIZE_MB, AUTO_PARTITIONING_MIN_PARTITIONS_COUNT and AUTO_PARTITIONING_MAX_PARTITIONS_COUNT, which the declaration spells partitioningBySize, partitioningByLoad, partitionSizeMb, minPartitionsCount and maxPartitionsCount.

• The settings are managed when the declaration states table properties for this target, and only the settings it states are compared. A setting the declaration leaves out stays as the table holds it, so the defaults a table reports for the settings the declaration omits never count as a difference. The native call has no reset, so removing a setting from the declaration does not return it to its default.
• Partition columns are set when the table is created. A declaration that names other columns than the table holds is refused; the same columns are no change.
• A starting layout is set when the table is created, and YDB keeps no record of it except the minimum partition count it sets from it: the count of the layout, unless the declaration states a minimum of its own. That minimum is the only witness an existing table has, so a declaration whose layout sets the minimum and whose minimum differs from the table's is a change only a rebuild makes. The new table is created with the layout and the held value of every other setting, the minimum aside, which the layout sets. Without a rebuild asked for, the planner refuses it with UNIFORM_PARTITIONS alter is not supported (or the PARTITION_AT_KEYS spelling), the two minimums, and the request that asks for a rebuild, such as --allow-table-rebuild. A layout beside a declared minimum leaves no witness and is not compared. The comparison reports the layout as a difference, so that the plan reaches the planner.
• A minimum count above a nonzero maximum is refused, checked against the value the table holds for a bound the declaration leaves out.
• A partition size declared without partitioningBySize turns partitioning by size on, as the server does when a size is set and as the host's embedded engine reads the declaration (measured on 25.1.4.7 and 26.2.1.14 for a table and an index that did not split by size). A size beside a declared partitioningBySize of DISABLED is refused.
• When the step switches partitioning by size or by load on, it names every setting the table keeps: the ones the declaration states and the held value of the others. Switching either on resets the minimum count the call leaves out to 1, and switching partitioning by size on resets the size too (measured on 26.2.1.14 through the table service for partitioning by load), so a minimum nobody declared would otherwise drop to 1. The size the table takes is 2048 MB unless the declaration states one, and the TEXT form names it. An index's settings follow the same rule.
• keyBloomFilter takes ENABLED or DISABLED. A table that reports the filter as unspecified keeps none, so a declared DISABLED matches it (measured on 25.1.4.7 and 26.2.1.14 by the host's embedded engine).
• readReplicasSettings takes perAzReadReplicasCount or anyAzReadReplicasCount. A count of zero in either mode is no replicas, and a table that holds some loses them through PER_AZ:0, since YDB has no reset for them. Replicas the declaration leaves out stay.
• Setting one of the three groups resets nothing of the others, so the step names a filter or replicas only where they change. Execution confirms that the groups the step does not name read back as they were.
• Impact is behavioral: the settings decide when YDB splits and merges partitions and where it serves reads. The step reads the table and alters its settings only, and does not depend on the family, column or TTL steps.

Index alteration

The service adds and drops global secondary indexes of an existing row table. Each index is one native AlterTable step: row-table-add-index/N reads the table and creates one index object, row-table-drop-index/N reads the table and drops one. The index object is a host subject of kind index, named by the index and parented by the table, so two index steps of one table never share an effect. An addition is ADD INDEX name GLOBAL [SYNC|ASYNC] [UNIQUE] ON (keys) [COVER (columns)]; a removal is DROP INDEX name. Impact is additive for an addition and destructive for a removal.

A host that lists planning.index-renames pairs an index the table holds with one the declaration states under another name and one definition, and sends the pair as one rename step. The service claims it with row-table-rename-index/N, a native AlterTable call with rename_indexes, whose text form is RENAME INDEX old TO new. The step reads the table, drops the old name and creates the new one. The index keeps its kind, its columns and its partitioning settings, and is not built again (measured on 25.1.4.7 and 26.2.1.14). Impact is behavioral: a query that names the old name stops reading through the index. The host moves the index's comment to the new name with its own steps. Execution refuses before it writes when the table no longer holds the index as planned or holds an index under the new name. After the call it requires the old name to be gone and the new one to hold the index as it was, settings included. YDB renames an index only with EnableMoveIndex on, so a rename step on a session without index_rename is refused when it is planned. A session reports the capability only to a host that lists the feature; any other host plans a rename as a removal and an addition.

The common indexes a source declares, such as a ptah:schema:index directive, are compared by the host. Inspection reports a row table's global indexes as its common indexes: a synchronous one, unique or not, with no method, and an asynchronous one with the method async. For a host that lists schema.index-methods, table preparation states the kind of each declared index in the same spelling, reading its type with the host's embedded engine's spellings: none, BTREE, GLOBAL, SYNC and GLOBAL SYNC are synchronous, and ASYNC and GLOBAL ASYNC are asynchronous, in any case and with _ for a space. The host compares the two spellings, so a declaration that changes only an index's kind is a removal and an addition of the index under its name, which this service carries out as the two steps above, the removal first. A host without the feature compares no kind, and such a change plans nothing.

Index settings from a source

A Go ptah:schema:index directive, the index entry of a YAML table and the index block of an HCL document state a global index's partitioning and read replicas with the attributes the host's embedded engine reads there: auto_partitioning_by_size, auto_partitioning_partition_size_mb, auto_partitioning_by_load, auto_partitioning_min_partitions_count, auto_partitioning_max_partitions_count and read_replicas_settings, with the spellings a table directive takes. They decode into the index's own facet (index-properties), owned by the index, and a value an index cannot take, such as auto_partitioning_by_load="true" or a count of zero, is refused where it was written. For a host that lists schema.observed-index-facets (contract 25.21, ADR 0068 in Ptah), inspection reports the same facet on each common index, with every setting the server describes on it, defaults included, so the host compares one owner's facet on both sides. A host that does not list the feature reads no index's settings, and the provider refuses a declared setting from it, since the comparison could not see what the index holds.

//ptah:schema:index name="by_body" fields="body" auto_partitioning_by_load="ENABLED" auto_partitioning_min_partitions_count="2"

• A setting the source leaves out keeps what the index holds, as the host's embedded engine compares it. A differing setting is a github.com/stokaro/ptah-ydb-plugin/row-table-properties-change on the index, planned as the row-table-index-settings/N step above.
• A table the plan creates is created with its indexes, and each index whose facet declares settings then takes them in a row-table-index-settings/N step after the creation, since no call that creates an index takes them. An index added to a table that exists takes them after its addition.
• An index the plan renames in place keeps its settings: the host compares a renamed index's facet under its new name, and a changed setting is a settings step after the rename, as the host's embedded engine writes RENAME INDEX and then ALTER INDEX ... SET. An index the plan drops is left alone.
• Settings declared on an index that is not a global sync, async or unique index, or on one the table neither holds nor gains, are refused.
• A table rebuild gives each index of the new table its declared settings, and what the old index holds for every other (see Native table rebuild).
• migrations generate writes the reverse of a change of an index's settings into the down file: every setting the index held, restored in place (see Reverse migrations).
• An export writes an index's facet as the settings that differ from the ones YDB gives a new index, as the host's embedded engine writes them, so ptah introspect writes auto_partitioning_by_load="ENABLED" auto_partitioning_min_partitions_count="2" on the directive, and the export applies back with nothing planned.

The declaration is the index set model (table-indexes, desired side) as a facet of the table, beside the table properties. It is managed when the declaration states it: an index the table holds and the set leaves out is dropped, and a declaration with no index set leaves the indexes alone. The comparison reads the specification of each declared index: the name, the kind, the ordered key columns and the ordered covered columns. The declared implementation requirements on the server-owned index table are part of the model and have to be valid, but they are not compared and not sent: YDB builds that table itself.

• YDB cannot alter an index in place. An index the table holds under a declared name with another kind, other key columns, other covered columns or another key order is a change the planner refuses with a diagnostic that names both definitions. It does not plan a drop followed by an addition, because that rebuilds the index. Declare the removal and the addition in separate plans if a rebuild is wanted.
• The rules above are those of global sync, async and unique indexes. A vector or full-text index follows its own (see Vector and full-text indexes). An index to add that states its partitioning or read replicas is added without them, since no call that adds an index takes them, and a row-table-index-settings/N step after the addition gives it the declared ones over the settings YDB gives a new index (automatic partitioning by size at 2048 MB, none by load, at least one partition, no read replicas; measured on 25.1.4.7 and 26.2.1.14), as the host's embedded engine writes ALTER INDEX ... SET after ADD INDEX. Execution of that step requires the new index to hold exactly those settings.
• A unique index added to a table that exists needs unique_index_on_existing_table. YDB 25.1 and 25.2 never add one, and later lines keep it behind EnableAddUniqueIndex, off by default; without it the server refuses the call (Adding a unique index to an existing table is disabled) with an outcome the provider cannot confirm. So a session without the capability refuses such an addition when it is planned, in the host's embedded engine's words: adding unique index "<name>" to table "<table>", which exists already, which requires target capability unique_index_on_existing_table, unavailable on this ydb target. A table the plan creates takes its unique indexes with it and needs nothing.
• An index reads columns the declaration states. A declared index that reads a column the declaration does not state is refused, and so is a kept index whose column the declaration drops.
• Ordering against column steps of the same table, whoever carries them out: an index is added after the addition of every column it reads, and dropped before the removal of every column it reads or covers. A column removal step that does not name its column is assumed to remove any column. Execution reads the indexes of the table before dropping a column as well, so a column an index still reads is refused before the call.
• The partitioning and the read replicas of an index the table holds, stated in the settings of its global kind, change in place with one step, row-table-index-settings/N, that reads the table and alters the index. It runs ALTER TABLE ... ALTER INDEX ... SET (...) through the query service, as the host's embedded engine writes it, because the native AlterTable call has no field for an index's settings. The rules are the table's: a setting the declaration leaves out keeps what the index holds, the statement names every partitioning setting the index ends with when the partitioning changes, and replicas declared as none are written as PER_AZ:0. A starting layout and partition columns are refused by name. The server describes an index's settings on the index itself (measured on 25.1.4.7 and 26.2.1.14), and the comparison reads them there. Impact is behavioral.
• YDB builds a new index in the background. The provider waits for the index to read back as ready, bounded by ten minutes, before it confirms the step. Inspection refuses an index that is not ready, so a table with an index in the build is never compared.

Vector and full-text indexes

A vector index (vector_kmeans_tree) and a full-text index (fulltext_plain, fulltext_relevance) are entries of the native index set, with their settings, and the comparison, the planner and execution follow the rules of the host's embedded engine for them:

• An index the set states and a table that exists does not hold is added with one step, row-table-add-index/N, of its own operation kind, row-table-native-index-operation. The native AlterTable call of the pinned bindings carries neither kind, so execution runs YQL's ALTER TABLE ... ADD INDEX through the query service, as the host's embedded engine writes it: INDEX name GLOBAL USING vector_kmeans_tree ON (prefix, vector) [COVER (columns)] WITH (distance=cosine, vector_type=float, vector_dimension=3, levels=1, clusters=2), the metric spelled similarity= where the index names a similarity, and INDEX name GLOBAL USING fulltext_relevance ON (column) WITH (...) with the analyzers in name order and the language quoted. The TEXT form is the same statement.
• An index the table holds and the set leaves out is dropped by the native AlterTable call, row-table-drop-index/N.
• YDB changes nothing of a vector or full-text index in place (ALTER INDEX ... SET (levels=2) answers Unknown table setting: levels). An index whose columns, kind or settings differ is dropped and added again under its name, the removal first, and a renamed index is dropped under its old name and added under its new one. The table is not rebuilt. A global index under the name of a vector or full-text one, or the other way round, is refused.
• The comparison reads an index's name, its ordered key and covered columns, its kind and its vector or full-text settings, exactly; a full-text filter switched off is read as one the index does not name, because YDB applies no filter it is not told to. The settings of the tables that implement the index are not compared: the server reports its defaults for them on every index, and YDB changes none of them on a vector index that exists. An addition that states other settings for them than YDB gives a new index is refused.
• A vector index names one metric, a vector type, a dimension from 1 to 16384, levels from 1 to 16 and clusters from 2 to 2048, with clusters to the power of levels at most 2^30, the limits 25.3 and later hold every index to; 25.1 and 25.2 check none of them, and the planner holds every line to them, as the host's embedded engine does. Its last key column is the vector, a String; the others are its prefix. Overlap settings are refused by name.
• A full-text index reads one String or Utf8 column of a table whose key is one Uint64 column (26.2 answers Fulltext index requires primary key column ... to be of type 'Uint64'), analyzed by the one column entry of its settings, with a tokenizer. An n-gram filter names both its lengths, a language goes with the snowball filter, and the snowball filter takes no n-gram filter, as 26.2 answers.
• A target without vector_indexes refuses a vector index before a statement is sent, one without vector_bit_type an index over bit vectors, and one without full_text_indexes a full-text index (see Feature flags). A server answers each with Vector index support is disabled, bit vector type is not supported (25.1, over rows), FULLTEXT_RELEVANCE index subtype is not supported (25.1) or Fulltext index support is disabled (26.2 with EnableFulltextIndex off).
• Execution reads the table, requires the index to be absent and its columns to exist, runs the statement once, reads the table back until the index is ready, and requires it to hold the requested definition and everything else to be unchanged.
• A table rebuild of a table that declares or holds a vector or full-text index is refused: the new table is created with its global indexes alone.

Index declarations of the provider's types

The host's frontends build a common index from every index declaration. For a host that lists source.index-kinds (contract 25.24), the provider claims the index types it models outside the common model, and the host sends a declaration of one of them whole, as one fragment, and builds no common index for it: vector_kmeans_tree, fulltext_plain and fulltext_relevance, which declare the table's native index set, and bloom_filter, bloom_ngram_filter and min_max, which declare a column table's local indexes in its storage. A type may be written with a space for each underscore and with GLOBAL USING or LOCAL USING in front, in any case. The fragment carries the index's columns, the columns it covers and a pgvector operator class (ops, which names a vector index's metric as the host's embedded engine reads it: vector_cosine_ops is distance=cosine, vector_ip_ops similarity=inner_product), and the settings the index's own directive, entry or block writes:

//ptah:schema:index name="docs_emb" fields="gen,emb" include="body" type="vector_kmeans_tree" distance="cosine" vector_type="float" vector_dimension="3" levels="1" clusters="2"
//ptah:schema:index name="ft" fields="body" type="fulltext_relevance" tokenizer="standard" use_filter_lowercase="true"
//ptah:schema:index name="bf" fields="body" type="bloom_filter" false_positive_probability="0.01"

A YAML index entry takes the same keys, and an HCL index block the same attributes, a count as a number. Every vector or full-text declaration of a table makes one native index set, and every local one joins the column storage its table directive or entry states. Refused by name: a unique index, an index comment, a setting of another kind of index, an operator class on an index other than a vector one, a local index of a row table, a vector or full-text index of a column table, and an index declared twice. A setting the index cannot take is refused with the host's embedded engine's words, at the source for its grammar and at planning for its limits.

A Go or YAML source of such a host states the native index set of every table it declares, so a table that writes none of these indexes is declared to have none, and one the table holds is dropped; a column table's local indexes are managed likewise. Without the feature a source cannot declare them, and a table keeps the ones it holds. An export writes each index back as a declaration of its type, spelled as the host's embedded engine wrote it in HCL: type = "GLOBAL USING vector_kmeans_tree" with the columns, the covered columns and the settings, and a local index's method alone, type = "bloom_filter". The embedded engine wrote no full-text or local setting in HCL, and refused the full-text document it wrote; the provider writes and reads the full-text settings there. HCL loses a column table's local indexes with its storage.

Execution

Execution reads the table first. A family operation requires every touched family and moved column to match the plan, and every family to add to be absent. An addition requires the column to be absent. A removal requires the column to exist and not to be a key column, an index column or stored column, the TTL column or a partitioning column; the server refuses each of those only after the call, which would leave the outcome uncertain, so the provider reads them first. A TTL operation requires the table to hold exactly the TTL the plan captured, or none, and a TTL to set must read a column the table holds with a type its mode reads. A settings operation requires the table to hold exactly the partitioning settings, the key bloom filter and the read replicas the plan captured for each group it changes. An index addition requires the index to be absent and to read columns the table holds, and an index removal requires the table to hold the index as the plan captured it. A change of an index's settings requires the table to hold the index as captured, settings included. A table that changed since planning is refused before the call.

After the call the provider reads the table back and checks that the requested change holds and that nothing else moved: other families and columns, column names and types, their order, the key, and the TTL. The TTL must read back as the one the request sets, run interval included, or as none after a removal, and as before for every other operation. After an index operation the added index must be ready and hold the requested kind and columns, a dropped index must be gone, and every other index must read back unchanged. After a settings operation the table must hold every partitioning setting the request names and the partition columns it held; the settings the server derives from them, such as the partition size it no longer reports once partitioning by size is off, are not compared. The key bloom filter and the read replicas must read back as the ones set, or as they were, and a request that names no partitioning setting leaves the partitioning as it was. After a change of an index's settings the index must keep its definition and be ready, and hold the settings by the same rules, while every other index, every column and the key read back unchanged. An added column must read back with its type, nullable and in the default family. A failed reply or unconfirmed read remains UNKNOWN and is never retried. As for creation, these checks do not establish exclusive ownership against another schema writer.

Live tests alter a table on YDB 25.1.4.7 and 26.2.1.14: families and column placements; adding columns, placing one in a family and dropping others in a single plan; changing each partitioning setting on its own and together, including switching partitioning by size off and on again; setting, changing, moving and removing a TTL, including the move of a TTL to an added column in the same plan that drops the column it read; and the refusal of an indexed column, of a TTL column the server would refuse, and of a stale alteration before the call; and adding and dropping sync, async, covering and unique indexes, with the refusal of a changed index definition. They set, change and remove the read replicas and the key bloom filter of a table, alone and with its partitioning, and the partitioning and read replicas of sync, async and unique indexes, including switching partitioning by size off and on again with a minimum the declaration leaves out, and refuse a stale settings plan before the call. The TEXT form of each change is run on a twin table and reads back the same. They plan again from the read back to show the declaration no longer changes anything, and check that rows survive.

Table property comparison

The offline models.compare service ydb-row-table-properties compares the table and column properties of a row table that exists on both sides. It reports one github.com/stokaro/ptah-ydb-plugin/row-table-properties-change for a table whose TTL, partitioning settings, read replicas, key bloom filter or column families differ from the declaration, one for each column that sits in another family, and one for a table whose index set differs from the declared one: an index added, removed or changed, or an index whose partitioning or read replicas differ from the ones it declares. An index change carries both complete index sets and names them with a facet key, so a table can have a property change and an index change at once. A changefeed set that differs is reported the same way, with the changefeed set as its facet; a source that describes the changefeeds of every table it declares states a table with none as one without changefeeds (see Changefeeds). The comparison applies the rules of the alteration planner, so a difference it reports is one the planner acts on, and a table that differs only in its TTL, its settings or its families still reaches the planner. A change carries both complete facets, replaces no owner, removes no state and leaves access unchanged. A facet present on one side only states nothing to compare; the planner decides what the missing read means. The planner accepts exactly these changes, accounts for each one with the steps of its table alteration, and refuses every other change kind.

The host names a table and its columns without a database, because they live in the session's, while the provider's inspection names that database in the owner of every facet it reads. The comparison and the planner read a facet whose subject names no database without its owner's, so both sides of one table name one owner. A facet whose owner is another table than its subject names is refused.

SQL schema reading

The provider serves schema.read-sql for the ydb engine in document mode. ptah schema plan and ptah schema apply hand it the documents of a --schema-file or of a schema directory of .sql files, and it reads them as one load without a database session. The reader is a port of Ptah's own YQL schema reader at commit a4e286dbf5ac42a609bd6ccd2b60dcc3335c210d, so a load reads to the model Ptah's bundled YDB owner reads, and a load Ptah refuses is refused with Ptah's words and at the position Ptah names.

The reply carries the tables with their keys, columns, defaults, serial columns, comments and indexes; views; users, groups and memberships; grants and revocations; coordination nodes as this provider's coordination model; and topics with the consumers CREATE TOPIC and ALTER TOPIC ... ADD CONSUMER declare, as this provider's topic model; and the data sources and external tables CREATE [OR REPLACE] EXTERNAL DATA SOURCE and CREATE [OR REPLACE] EXTERNAL TABLE declare, as this provider's external object models; and streaming query definitions as the streaming query model; and secrets whose values are approved input references; and the async replications and transfers CREATE ASYNC REPLICATION and CREATE TRANSFER declare, with what a later ALTER ASYNC REPLICATION ... SET or ALTER TRANSFER ... SET of the load changes folded in, as this provider's replication and transfer models. For a host that lists schema.sql-facets (contract 25.21), it carries the changefeeds ALTER TABLE ... ADD CHANGEFEED declares, with the consumers ALTER TOPIC <table>/<changefeed> ADD CONSUMER adds to their topics, as one facet of each table in this provider's changefeed model, the value a YAML changefeeds: key of the table declares.

Under the same feature it carries the settings of a CREATE TABLE as the facets a YAML or Go source declares for the same table (see Offline row-table declarations), so a table means the same in all three formats:

• a row table's partitioning (AUTO_PARTITIONING_*), read replicas, key bloom filter, starting layout (UNIFORM_PARTITIONS or PARTITION_AT_KEYS), column families and TTL, as one row-table facet of the table;
• for each column that FAMILY <name> places in a family other than the default one, a column facet that names the family;
• a column table's hash key (PARTITION BY HASH), shard count (AUTO_PARTITIONING_MIN_PARTITIONS_COUNT) and TTL, as one column-storage facet. A TTL that only deletes is the storage's one deleting tier, as the row_deletion_* properties beside a YAML or Go column table are;
• for a host that also lists schema.observed-index-facets, the partitioning (AUTO_PARTITIONING_*) and read replicas a global sync, async or unique index declares in its WITH clause, as one facet of the index, the value a Go index directive or a YAML index entry declares.

A setting the statement leaves out keeps what the table or the index holds, and a row table declared without a TTL has its TTL removed, as for a YAML or Go source.

The reply says what each document declares, traces every object to its statement, and claims complete coverage of the row-table, column, column-storage, coordination, topic, external data source, external table, streaming query, secret, async replication and transfer models, of the changefeed model under that feature, so a table a load declares without a changefeed is one with none, of the index settings model where the host also lists schema.observed-index-facets, and of the native index model where it also lists source.index-kinds, so a table a load declares without a vector or full-text index is one with none. A vector or full-text index of a CREATE TABLE is no common index then: it reaches the host in the native index set of its table, and a local index in the storage of its column table, the values a YAML or Go source of the same index declares, so a YQL schema renders, plans and applies them as the host's embedded engine did. A ptah:not-described header entry of kind coordination_node, topic, external_data_source, external_table, streaming_query, secret, replication or transfer limits that claim, for one object or, without a name, for the kind. An object the claim leaves out keeps the entries of its access list too: the host plans no REVOKE on an object its source does not describe.

A replication or a transfer is read as the host's reader reads it: the items of FOR ... AS ..., ENDPOINT and DATABASE as the connection string they spell, the lambda of USING byte for byte, and in an ALTER only the settings YDB changes in place. A malformed statement is refused without naming a token, since its connection clause may hold a credential written by mistake.

A limit names an object by its path from the database root, a dot being part of a name, so app.events names a topic at the root and app/events one in the directory app. A dotted limit of a topic, a secret or an external object that names nothing the database holds, while the plan would drop the object the same path written with a slash names, refuses the comparison with the spelling that keeps that object, as the host's embedded engine refuses it. A dotted limit that names an object the database holds is read as written.

A refused token is quoted with at most its first 64 bytes, where Ptah quotes it whole: a schema file can hold a secret in a string literal.

A statement that changes an object no document of the load declares is refused in Ptah's words: an ALTER TABLE ... ADD CHANGEFEED of a table, a COMMENT ON an object, an ALTER TOPIC ... ADD CONSUMER of a topic, an ALTER USER of a user, or an ALTER SEQUENCE of a Serial the load does not declare. Under schema.read-sql-undeclared (contract 25.38, ADR 0120 in Ptah) the refusal is an UNDECLARED error, which the host reads as its own reader's refusal, so a YQL file beside an HCL file that declares the table is told that the YQL files change only what the YQL files declare, as the host's embedded engine said. A host that does not list the feature gets an INVALID error with the same words.

A load that declares something no reply field or model of this provider can carry is refused by name, with no partial reading:

• the settings of a table, for a host that does not list schema.sql-facets, which has no home for a facet of a table;
• a column family on a column table, and a row table that declares both UNIFORM_PARTITIONS and PARTITION_AT_KEYS, with the words a YAML or Go source of the same table is refused with;
• the partitioning and read replicas of an index, for a host that does not list both schema.sql-facets and schema.observed-index-facets: without the second the comparison could not see what an index holds, as for a YAML or Go source;
• a vector, full-text or local index, for a host that does not list both schema.sql-facets and source.index-kinds;
• a local index of a row table, a vector or full-text index of a column table, and a comment on one of them, with the words a YAML or Go source of the same index is refused with, and a vector index's partitioning, which the native index set does not hold;
• changefeeds and the consumers of their topics, for a host that does not list schema.sql-facets, which has no home for a facet of a table. The load then claims nothing about changefeeds and leaves those of its tables as they are;
• a user's password, for a host that does not list source.sql-role-passwords: the reply's role carries none, so the host would create the user without it;
• a grant on a resource pool whose name holds a dot, a quote or a bracket: Ptah names a pool it reads from a provider as it names a table, so the grant would come back on a renamed pool.

Under source.sql-role-passwords (contract 25.33, ADR 0112 in Ptah) the reply carries the password a document gives a user as the document writes it: a literal after PASSWORD, a YDB password hash after HASH, or the $PTAH_SECRET_... reference a migration file Ptah generated names it by. The host binds it as it binds a password a YAML or Go source declares, so the plan names the user's execution input and no value (see Common objects), and execution writes HASH for a hash, as the host's embedded engine did. Measured on 25.1.4.7 and 26.2.1.14: a user created from PASSWORD 'x' logs in with x, one created from HASH logs in with the password the hash stands for, and the next run plans nothing.

Some of what a reading carries is not planned yet. Table creation refuses column defaults, and table comments on a host that does not list schema.common-objects, which carries them. Ptah refuses views and roles from a provider's reading. A coordination node is planned as the coordination services describe.

internal/yqlschema/testdata holds the differential corpus: every load of YQL schema documents Ptah's own tests read, the YDB fixtures Ptah renders, and edge cases, each with the answer Ptah's reader recorded. The unit suite compares the provider's reading with each answer, refusal for refusal. The corpus README says how the answers are recorded again.

Offline SQL analysis

The SQL analysis service reads default-mode YQL without a database session. It preserves source byte ranges and line numbers, original tokens, and quoted case. Semicolons inside quoted or multiline literals, comments, action definitions, inline actions, and lambda bodies remain inside their enclosing statement. Leading and trailing comments stay in the returned source ranges.

The scanner and value decoder were extracted from Ptah's YQL implementation, with its lexical regression corpus. The provider keeps no host-package import. Live tests compare reconstructed statements with results from both supported YDB test lines, including escaped quotes, multiline strings, compound bodies, and numeric suffixes.

Native safety judgments cover topic and consumer positions, transfer consumers, replica tables, coordination state, secrets, external data access, and streaming checkpoints. Common table creation and a single ADD COLUMN clause retain the host's common DDL assessment. Unmodeled effects, including arbitrary compound bodies and changes to automatic expiration, report UNKNOWN and require review. The analysis does not replace the server's full SQL grammar validation.

Malformed literals, incomplete compound bodies, invalid UTF-8, NUL, and unsupported translation settings refuse the whole batch. Only the default lexer and --!syntax_v1 are accepted; ANSI mode is refused even when its directive appears later in a source, because splitting could move it to a new query's head. Whitespace-only sources are accepted. A source containing only comments or terminators is refused because the SDK requires tokens for every non-whitespace source range. No prefix of a refused batch is returned.

A request contains at most 256 sources and 3 MiB. Each source contains at most 65,536 significant tokens, and the complete response is bounded by 3 MiB. Cancellation returns no analysis. The native CLI still requires integration of YDB's table comparison rules; the provider does not claim the common table identity contract to bypass that requirement.

Offline capabilities

The provider lists engine.default-capabilities (contract 25.27) and states the facts an offline command plans ydb with: those a session states for a 26.2 server whose address names no monitoring endpoint, with the facts of the common objects. Each release line states its own the same way, and a --server-version selects the line it equals or the longest line it starts with followed by a dot, so 26.2.1.14 selects 26.2. ptah schema render, sql lint and the other commands that plan without a database read them. A command that connects plans with the session's facts.

The facts also carry the keys the host reads and no session states, with the values of the built-in engine's presets, on every line. The host reads them through its capability registry when it plans a connected command too: index_covering_columns, for one, makes an unchanged list of covered columns compare as unchanged, so a second apply of the same schema plans nothing for the index. A host of an earlier contract receives no facts, plans offline against unknown facts, and the provider refuses what needs one, such as a table creation, which needs nontransactional DDL established.

Text scanning

The provider lists sql.scan (contract 25.26). ScanSQL returns the tokens of one YQL text with the same scanner, so the host reads YQL wherever it reads SQL text without parsing it: the lineage of view bodies, the indentation of the sql template function, and the scans of migration bodies. Both quote styles and @@...@@ delimit strings, a type suffix belongs to its string, and a backticked name carries the path it stands for. A --! comment is a setting the server reads only at the head of a text; the host clears head for a range inside one, where the comment is a comment. A string whose escape YDB refuses carries no value. A semicolon is a SEMICOLON token only where it ends a statement of the text: one that ends a statement inside a DEFINE ACTION or DEFINE SUBQUERY body, a DO BEGIN block or a lambda is an OPERATOR, so the host's statement splitter keeps such a body whole, as Ptah's built-in YDB engine split it. A host of an earlier contract never calls the RPC and reads YQL by its shared rules.

Migration lint

The provider serves the SDK's lint.migrations service for the ydb engine without a database session. It publishes its rules as data, with the codes, titles and severities of the host's built-in YDB owner, whether a rule also reads the down half of a migration, and the shared rules a finding replaces on the same statement.

A split reads each migration file as YQL, with the lexer the SQL analysis uses. It returns every statement's byte range, line, words and the comments directly above it, which the host reads suppression directives from. It also reports the facts the host's shared rules read: a drop that loses state (a topic, a transfer, a coordination node, a streaming query, a replication dropped with CASCADE), YQL's unique index, the steps of a table rebuild, and a safety judgment for the statements only YDB's owner recognizes. The splitter reads what a rebuilt table holds from the directory itself: the up migrations of lower versions in version order, then the statements of the file before the rebuild. An annotation judges the statements a host split without a dialect, and declines a statement YQL would split differently.

A check judges every file of a run under the rules the request enables. A rule reads the statement text and the capability map the request carries, never the server version. The rules that need to know what a table looks like before a statement read it from the directory itself, as the splitter does: whether an index or the TTL uses a column, a table's partitioning settings, the views that read it, its changefeeds and column families, the Serial columns and sequence restarts, failed-over replications, and the comments Ptah's COMMENT ON statements left. A table created outside the directory is unknown and reports nothing.

internal/migrationlint/testdata holds the host's answers for a corpus of migration directories and requests, recorded at one host commit, and the tests require the same statements, facts, annotations and findings, message for message. Its README says how to record them again.

Statement facts for ptah sql lint

For a host that lists lint.statement-facts, which the engine lists beside lint.migrations, a split also says what each statement is, which table it creates without a primary key, and which capabilities it needs. ptah sql lint reads a YDB file through these facts when this provider serves the engine, in place of the host's built-in YQL reading, and reports what that reading reports:

• a CREATE TABLE or an ALTER TABLE is examined; any other CREATE, ALTER or DROP is a schema statement no rule examines, named by its first two words;
• any other statement changes no schema object and is named by its first word, or as a named expression;
• a CREATE TABLE without a PRIMARY KEY clause names its table, and a table written as a named expression keeps the name as written;
• the capabilities are those the migration lint rules read: a unique index or a column with a default added to an existing table, a vector index, and a vector index over bit vectors. Each names the index or the column and the clause that needs it. The host compares the key with the capabilities of the server version it lints for.

A statement that starts with no word, such as a string or a parenthesized query, is an expression the linter does not lint, and the first statement of a file that opens with a UTF-8 byte order mark is read like any other. The built-in reading skips both and reports nothing, which lets such a file read as clean; the provider is stricter on purpose.

internal/migrationlint/testdata/statementfacts holds YQL files with the built-in reading's findings on three release lines, recorded at one host commit. The unit tests and an end-to-end run of the host's ptah sql lint through this provider require the same rules at the same positions about the same tables and capabilities. Its README says how to record them again.

A host that does not list the feature is offered none of these facts, and an annotation request that asks for them is refused.

The host owns output formats, severity overrides, suppression and the rules every dialect shares. The host's built-in owner reports nothing from these rules on a run that names no dialect; the provider is only asked about its own engine, so a host does not send it the codes of such a run.

Rows, row writes and seeds

The provider serves the row services of the contract (data.type-rows, data.read-rows, data.render-writes) and the seed service (data.seeds), with the answers of the host's embedded YDB engine: its row typing (internal/ydbdata), its row diff statements (migration/datadiff) and its seeds (migration/seeder and the YDB seed service). The host keeps which rows are declared, the row diff, the ordering and the seed policy.

The engine declares the host's YDB query dialect for the host's query builder, and a session reports the capabilities the builder reads, at the host's line presets: returning_clause from 25.3, since 25.1 and 25.2 fail UPDATE ... RETURNING on a table with a unique index (wrong returning expr type), and common_table_expressions, correlated_subqueries, non_equi_joins and offset_without_limit on no line. The builder refuses a clause whose key a session leaves out.

YQL types every literal and converts few of them, so every value is compared and written in its column's YDB type: the live table's type where the column exists, and otherwise the type the declared type lands on for the request's capabilities (VARCHAR(255) is Utf8, TIMESTAMP is Timestamp64 with wide_date_time_types and Timestamp without it, DECIMAL(10,2) needs parameterized_decimal). TypeRows returns the declared and the live rows in one canonical form, so a declared 12.50 pairs with the 1.5 a Decimal reads back, a moment in another zone with the one in UTC a read returns, PT26H with the interval of the same length and an upper-case UUID with the lower-case one YDB stores. A Decimal is a decimal at its column's scale, a Date a date, the other date and time types a timestamp, an Interval a count of nanoseconds, String and Yson bytes, Json and JsonDocument JSON text (JsonDocument with sorted keys and no spaces), and Utf8, Uuid and DyNumber text. A value its column cannot hold refuses the request with the host's words, and a column no side types keeps its values.

RenderRowWrites writes the host's statements: INSERT INTO with the columns sorted by name, UPDATE ... SET ... WHERE with the columns outside the key sorted and the key predicate in sorted order, DELETE FROM ... WHERE, and a key column holding null as IS NULL. An UPSERT is YQL's UPSERT INTO in the shape of the INSERT; the host's row diff writes none. A value a column cannot hold, or a column the write names no type for, refuses the batch with UNSUPPORTED_VALUE and a message that names the write and the column but never the value.

ReadRows reads one page of a row or column table in the order of the requested key columns and then of the primary key columns the request leaves out, so the order is total, and resumes after the last row of a page through its token. YDB orders a null first, and the cursor steps past nulls and equal keys. A page holds at most the requested rows and 3 MiB. A missing table is a complete page that says so. A Decimal value with more digits than its precision, which is how YDB stores infinity and NaN, fails the read rather than being returned as digits the column never held.

A seed file runs as the queries YQL reads it into, in one serializable transaction with its ledger row, so the file and its row commit together or not at all. A transaction YDB aborts for a conflicting one runs again, at most five times. A file holding a scheme statement, a BATCH UPDATE or BATCH DELETE, a translation setting other than --!syntax_v1 or a delimiter directive is refused before anything runs, with the position of the statement, and so is a statement that leaves the dev realm of a realm session. A file that hits an existing key is KEY_CONFLICT, and nothing of it stayed. A failed commit that YDB did not answer with a key conflict or an abort is UNKNOWN.

The ledger is the host's table schema_seeds at the root of the session's database, created with the host's statement, so a ledger the embedded engine wrote is read and extended here, and the other way round. A table of that name with another layout is refused and never adopted. A missing ledger reads as missing and is not created by a read.

The engine also declares the query-builder spelling the host's YDB descriptor states (query.dialect): named parameters, YQL quoting with paths, unsigned row bounds, UPSERT INTO, and the capabilities the builder assumes for YDB when its caller names no target.

internal/rowdata/testdata holds row cases and the host's answers for them, and internal/yqlquery/testdata YQL texts and the queries the host splits each into; the README there says how to record them again. The one difference is the wording of a write refusal, which names the value in the host's message and never in the provider's. A live test writes rows of every value kind through the services, reads them back in pages and types both sides into one form; the seed tests run in a dev realm and read a ledger written with the host's statements. Against a Ptah whose schema apply and ptah seed serve declared rows and seed files through a provider, the end-to-end tests run both through this one in a dev realm and read the table back.

Embedding store

The provider serves the SDK's embedding store (embeddings.store, ADR 0032) for the ydb engine, for a host that lists the feature too; the engine declares the immutable consistency mode. YDB has no triggers, so the outbox mode, which records each source change in the transaction that makes it, is not offered. The host keeps the lifecycle of ptah inference; the provider stores what the host decided and refuses nothing for a lifecycle reason.

The layout is the host's embedded YDB engine's (internal/embedydb), so a database either one wrote runs on through the other:

• the run state is the four tables ptah_embedding_generation, ptah_embedding_run, ptah_embedding_event and ptah_embedding_pointer at the root of the session's database, with the host's columns, keys and nullability, a timestamp as Timestamp64 on 25.2 and newer and as Timestamp on 25.1; an existing table of another shape is refused with NOT_OWNED and never altered;
• a generation beside its source adds the vector column, a nullable String, and four Utf8 columns named by the suffixes _generation, _input_hash, _source_version and _state; a chunked generation keeps a table of its own keyed by the source key and <column>_chunk_ordinal, marked as the generation's by its comment;
• a vector is stored as the bytes Knn::ToBinaryStringFloat produces, and the index is a vector_kmeans_tree with the metric's distance or similarity and the specification's levels and clusters.

A compare-and-write (MutateEmbeddingState, CommitEmbeddingBatch) is one serializable YDB transaction that reads what it compares, compares the token, writes and commits. A transaction YDB aborts, or a server that is briefly overloaded or unavailable, runs again, at most twelve times; a commit that fails any other way is UNKNOWN. A token is a digest of the records read, so a record that changed or appeared gives another one. Written records come back as a later read returns them: optional text empty, times in UTC to the microsecond.

A batch makes each written key's rows equal to its set: on the source's columns an UPDATE by the key, so a key the source no longer has creates no row, and in a table of its own an UPSERT that selects the key out of the source, with the rows past the set's end removed and a tombstone removing them all. A write into a generation whose index exists is refused as UNSUPPORTED on 25.2 and older, whose vector index answers a search as the table stood at its build.

A source is paged by its primary key, which the specification's key fields have to be. The verification walk reads both sides in one query per page, one table read once or a full join of two, and never splits a key's rows across pages. A search asks twice, through the index with VIEW and by exhaustive scan. Building an index needs vector_indexes, which 25.1 has with EnableVectorIndex on; the provider reads the flag where the address names the monitoring endpoint. A drop checks ownership before anything goes: a table by its mark, columns by their comment and by holding no other generation's rows.

internal/embedstore/testdata holds the host's answers for the run-state tables on 25.1 and 26.2, a corpus of specifications and a set of vectors, and the tests require the same statements, objects or refusals, and bytes. Its README says how to record them again. The live tests run the store on both lines in dev realms. Against a Ptah build they read a database the host's embedded engine prepared, backfilled and indexed, search it through the engine's index, and append an event the engine then reads; and against a Ptah whose ptah inference takes --provider-config they run the host's YDB inference flows through this provider: two generations from plan to cutover, a rollback and a retirement, a chunked generation in a table of its own, the refusals, and the answers an agent reads, which carry no source content.

Offline schema rendering

The provider uses schema.declarations to create authored coordination nodes, topics, external sources and tables, resource pools, classifiers, secrets and streaming queries. The host validates the whole schema and combines these creations with table and common object steps before rendering SQL. Declaration operations retain their desired definitions and contain no observed-state claims.

A rendered schema opens no session. Capability limits, conflicting object paths, duplicate classifier ranks, and explicit native access requirements refuse the whole batch. Use live schema planning for access requirements that a CREATE statement cannot express. External tables follow their declared data sources; classifiers follow their declared resource pools. External sources follow their declared secrets. A streaming query follows each topic and table its text reads or writes, since YDB compiles a query when it creates it. A path the text names by its database is read below the database the query's value names, the one a read took it from, as schema inspect --format sql renders it, and the text stays as written. A value that names no database, such as a query a source declares for an offline render, cannot place such a path and is refused. Secret statements retain approved input references; rendering opens no session and reads no input value.

Under render.omissions (contract 25.33, ADR 0110 in Ptah) a render reports what its statements go on without, as the host's embedded engine reported it, and the host carries the omissions into the result of a schema render and of a node render, where a caller that must not lose anything refuses the rendering. They are the part of a created column's declared type YDB does not keep, such as VARCHAR(255): length 255 for a Utf8 column or TIMESTAMP(3): precision 3 (YDB keeps microseconds), in either form, and in the TEXT form of a creation that names no database, such as an offline render's, the start and the increment of a Serial's sequence: only an ALTER SEQUENCE naming the sequence by its absolute path sets them, so the table is written without them, where a plan against a database writes the statement (see Serial columns). Each names its column <directory>.<table>.<column>, as the host did. The creation carries the declared type a column keeps only in part, since the operation holds the YDB type alone.

Operator statements and object scopes

The sessions.statements service runs operator checks, migration test assertions and script steps on a pinned connection. It returns typed rows in bounded pages and leaves the affected-row count absent because YDB does not report it. A result ends at its last page, explicit close, transaction end or session close. Cleanup failures remain failures on a repeated close.

A read-only request proves that its single SELECT reads local tables or views, then runs in a snapshot read-only transaction. External tables, views that reach them, UDFs, code evaluation and file or secret functions are refused before execution. A transport failure has an unknown outcome and is never retried. Confirmed atomic query failures preserve the server's issue text for expected-error assertions, without the driver's address and transport wrapper. Bound arguments retain JSON, decimal, date, timestamp and interval types; precision the engine cannot represent is refused before execution.

The provider offers sessions.typed-arguments. A host that lists it sends each argument with the type its caller bound it at, and the provider binds the parameter at that YDB type: INT8 to INT64 as Int8 to Int64, UINT8 to UINT64 as Uint8 to Uint64, FLOAT32 as Float, FLOAT64 as Double, STRING as Utf8, BYTES as String, TIMESTAMP as Timestamp and INTERVAL as Interval. An optional type binds as Optional<T> and its null as a null of T, a list as List<T> (an empty one keeps its element type), and an untyped null as Void, as a plain null binds. So a Go int32 writes an Int32 column, which an Int64 parameter cannot, and a nil *string writes a null into an Optional<Utf8> one. A typed timestamp outside the Timestamp range is refused rather than bound as a Timestamp64.

The negotiated scope.selections declaration makes secrets, streaming queries, coordination nodes, topics, external data sources, external tables, async replications and transfers selectable by their directory and name. Resource pools and classifiers use database-wide names, including literal dots. A kept classifier or streaming query keeps the pool it requires when the input contains that pool, the pool default for a query that names none; an explicit exclusion of the pool still wins. A table's changefeeds go with the table. An external table keeps no data source and a data source no secret: an --include of a table plans that table alone, whether it is created or dropped, as the host's embedded engine did. The host keeps a requirement on each side of a comparison alone, so a kept source would be dropped with a table the declaration leaves out, and created again beside a held one. Grants follow the object their target names: each kind except the classifier declares the grant object type the host's embedded engine names it with, such as ASYNC REPLICATION or TRANSFER. A grant on a type no selection declares is read as a grant on a table of the same name, so a scoped plan would leave it out.

Extraction still required

The source reference is Ptah commit 7b4b19f49fadf03ce81def2f6168ea60b3261a71. Remaining work includes more source formats, comparison, the alteration of table settings other than column families, planning, rendering and operation models for other objects, and the capabilities of the feature flags the host plans with. The provider advertises implemented inspection, normalization, coordination YAML, Go and HCL source decoding and encoding, index entry key claims, comparison, planning, rendering, text units, table rebuilds the planning policy asks for, views and comments on the common wire, SQL, offline schema validation, native coordination, topic, external object, resource pool, streaming query, async replication, transfer and secret lifecycles, approved execution inputs, row-table creation and row-table alteration execution, transaction, history, metadata, lock, offline SQL analysis, SQL schema reading, migration lint, migration splits, row and seed services, dev database cleanup, replay guard, embedding store and object count services. The remaining capabilities stay unadvertised.

The host's embedded implementation still needs removal when these operations are wired through this executable. Provider release workflows, default Ptah bundling, and independent installation acceptance remain pending. No v1 stability or full YDB support is claimed by this development state.