absdb

package module
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 29, 2026 License: MIT Imports: 23 Imported by: 0

README

go-absolute-database

CI Go Reference Go Report Card

A Go library for reading and writing ComponentAce Absolute Database (.abs) files, with no Delphi, no Windows and no vendor runtime involved.

Absolute Database is a single-file embedded database engine for Delphi that replaced the Borland Database Engine, and it is still used by commercial Windows applications — notably SoundPlan, the dominant DACH noise-calculation tool, which stores result tables, train-type catalogues, address lists and attribute tables as .abs files. If you have such files and no Delphi, this library reads them.

There is no public specification of the format. Everything here was reverse-engineered from real files; see Provenance and legal basis.

Independent project. Not affiliated with, authorised by, endorsed by or sponsored by ComponentAce. "Absolute Database" and "ComponentAce" are their trademarks, used here only to name the format this library reads. See NOTICE.

Install

go get github.com/cwbudde/go-absolute-database

Go 1.25 or newer. The library depends only on the standard library plus golang.org/x/text (Windows-1252 decoding) and golang.org/x/crypto (Blowfish).

Reading

db, err := absdb.Open("project.abs")
if err != nil {
    return err
}
defer db.Close()

table, err := db.Table("Addresses")
if err != nil {
    return err
}

r, err := table.Open()
if err != nil {
    return err
}

for r.Next() {
    rec := r.Record()
    fmt.Println(rec.Int(0), rec.String(1), rec.Time(2))
}

return r.Err()

Encrypted databases open with absdb.OpenWithPassword. All eight of the engine's algorithms are supported: Rijndael-128/256, Blowfish, Twofish-128/256, Square, DES and DES-Triple.

Writing

Writing is opt-in. Open returns a read-only handle and every write path checks that flag, so a program that only reads cannot corrupt a file by accident.

db, err := absdb.OpenForWrite("project.abs")

That gives you record insert, update and delete with index maintenance, plus CREATE TABLE, DROP TABLE, CREATE INDEX, DROP INDEX, ALTER TABLE ADD/DROP COLUMN, database creation and compaction. Writes are held to a deliberately harsh standard: for each supported operation, a test requires this package to reproduce byte for byte the file the vendor's own DB Manager produced for the same SQL statement. Reading a write back correctly is not accepted as evidence.

Where this package cannot meet that standard it refuses the write rather than guessing — a table declaring any constraint, or an index whose ordering is not reproduced, is rejected with a named error. docs/writing.md lists every refusal and why it exists.

Command line

go install github.com/cwbudde/go-absolute-database/cmd/absdb@latest

absdb info    <file>              database header, geometry, encryption
absdb tables  <file>              the table catalog
absdb schema  <file>              column definitions
absdb dump    <file>              records as text
absdb pages   <file>              page map by type
absdb blob    <file> <row> <col>  extract one BLOB
absdb hexpage <file> <page>       hex dump one page

Status

Reading is complete: header, schema, records, BLOBs, indexes, multi-table catalogs and all eight encryption algorithms. Writing covers records and schema operations, as listed above. Creating an encrypted database is not supported, and there is no database/sql driver yet. PLAN.md tracks what is built and what comes next; docs/open-questions.md tracks what is still undecoded.

Documentation

Start at docs/README.md. The format reference lives under docs/format/ — the file header, pages, internal files, schema, records, indexes, BLOBs and encryption, each documented against real files rather than against a specification that does not exist.

Testing

just test        # or: go test ./...
just ci          # everything CI runs, plus the parts CI cannot

A green CI badge is narrower than it looks. testdata/ is gitignored: most fixtures are real private project files and are never committed, and every test that opens one skips when the file is absent. What CI covers is the synthetic and unit tests plus the forty committed fixtures, which are ours — they hold invented tables and invented values. Those forty do exercise the full encrypted read path for all eight algorithms and the byte-for-byte write tests, but validation against the private corpus happens locally only. docs/testing.md is explicit about the boundary.

This is interoperability work: reading data that belongs to the people who own the files, using an independently written program.

  • The format is not protected by copyright. In the EU that follows from Case C-406/10, SAS Institute v World Programming, in which the CJEU held that neither a program's functionality nor the format of its data files is a form of expression protected by copyright.
  • Study and observation of a lawfully obtained program, and decompilation for interoperability, are permitted under Directive 2009/24/EC Arts. 5(3) and 6 — in German law §69d(3) and §69e UrhG — and §69g(2) UrhG voids contract terms that purport to forbid them.
  • No ComponentAce code, header, binary or documentation is included here, and none ever has been. Redistribution is a separate question from reverse engineering, and this repository stays on the right side of it by containing nothing of theirs.

docs/provenance.md records the full reasoning, including the questions that are still open, and NOTICE carries the third-party attributions.

Licence

MIT — see LICENSE. Third-party notices for the zlib level-1 port and for DEC are in NOTICE.

Documentation

Overview

Package absdb reads ComponentAce Absolute Database (.abs) files.

The binary format is reverse-engineered from real .abs files and the C++ header files shipped with the Absolute Database SDK.

Index

Constants

View Source 
const (
	PageTypeSystemDir = 2  // System directory
	PageTypeFileHdr   = 3  // File header (page 0)
	PageTypeTableList = 6  // Table catalog (uncompressed internal file)
	PageTypeSystem    = 7  // A table's "system" internal file; role unidentified, see TableInfo.systemPageNo
	PageTypeSchema    = 8  // Schema metadata (zlib-compressed column defs)
	PageTypeTableInfo = 9  // Table info (record counts)
	PageTypeData      = 10 // Data page (row storage)
	PageTypeIndex     = 12 // B-tree index page
)

Page type constants from the TABSDiskPageHeader.PageType field.

View Source 
const (
	// PageTypeBlob is the page type for BLOB data storage.
	PageTypeBlob = 11
)

Variables

View Source 
var (
	ErrNotABS         = errors.New("absdb: not an Absolute Database file")
	ErrTruncated      = errors.New("absdb: file is truncated")
	ErrPageOutOfRange = errors.New("absdb: page number out of range")
)
View Source 
var (
	ErrBlobNotFound = errors.New("absdb: BLOB data not found")
	ErrBlobTrunc    = errors.New("absdb: BLOB data truncated")

	// ErrBlobSize reports a BLOB header whose declared sizes are negative or
	// larger than the file could possibly hold.
	ErrBlobSize = errors.New("absdb: BLOB size out of range")

	// ErrBlobChain reports a malformed BLOB page chain: a cycle, a page
	// without a disk page header, or a chain too long to be plausible.
	ErrBlobChain = errors.New("absdb: malformed BLOB page chain")
)
View Source 
var (
	// ErrNoCatalog is returned when the file holds no table catalog page.
	ErrNoCatalog = errors.New("absdb: no table catalog page found")

	// ErrBadCatalog is returned when the catalog page cannot be parsed.
	ErrBadCatalog = errors.New("absdb: malformed table catalog")

	// ErrNoSuchTable is returned when a name matches no catalog entry.
	ErrNoSuchTable = errors.New("absdb: no such table")

	// ErrAmbiguousTable is returned when a table is selected by the empty name
	// but the database holds more than one, so there is no "the" table.
	ErrAmbiguousTable = errors.New("absdb: database holds more than one table")
)
View Source 
var (
	// ErrPageUnattributed reports that the file holds an allocated page this
	// package cannot assign to a table or to the database itself. A schema
	// operation refuses rather than proceed, because a page it cannot name is a
	// page it might leave allocated with nothing referring to it.
	ErrPageUnattributed = errors.New("absdb: file holds a page that belongs to no table")

	// ErrTableHasBlobPages reports a drop of a table that owns BLOB pages. They
	// are reachable only through the table's BLOB page index, which names the
	// pages a BLOB starts on and not the ones it continues on, so freeing what
	// it lists would leak the rest.
	ErrTableHasBlobPages = errors.New("absdb: table owns BLOB pages this package cannot free")

	// ErrLastTable reports a drop of the database's only table. The engine
	// shortens the file when the catalog empties, and no fixture pins the rule
	// it shortens by, so this package refuses instead of writing a file that
	// would differ from the engine's.
	ErrLastTable = errors.New("absdb: cannot drop the database's only table")

	// ErrCatalogNotWritable reports a table catalog that cannot be rewritten in
	// place: one that is compressed, or one that spans more than a single page.
	// Neither occurs in any fixture.
	ErrCatalogNotWritable = errors.New("absdb: table catalog cannot be rewritten in place")
)
View Source 
var (
	// ErrColumnExists reports an AddColumn whose column name is already used
	// by the table, case-insensitively -- the same matching Schema/DropColumn
	// use for every other name lookup in this package.
	ErrColumnExists = errors.New("absdb: column already exists")

	// ErrLastColumn reports a DropColumn that would leave the table with no
	// columns at all. No fixture or analysis says what the engine does with a
	// zero-column table, so this package refuses rather than create one.
	ErrLastColumn = errors.New("absdb: cannot drop a table's only column")

	// ErrColumnIndexed reports a DropColumn naming a column an index covers.
	// The index format does not let this package repair or rebuild the
	// index, so the column staying in place is the only safe outcome.
	ErrColumnIndexed = errors.New("absdb: column is covered by an index this package cannot repair")

	// ErrColumnConstrained reports a DropColumn naming a column a NOT NULL,
	// PRIMARY KEY, UNIQUE or MINVALUE/MAXVALUE constraint record covers.
	//
	// The record format is decoded now (ddl_constraint.go), so this is found
	// by parsing the constraint array and asking each record which columns it
	// covers, not by the text scan that used to stand in for it. What has not
	// changed is the outcome: dropping the column would leave a constraint
	// naming a column the file no longer has, and nothing in the corpus says
	// what the engine does with one. Reading the record is not the same as
	// knowing how to rewrite the array around a removal, and this package will
	// not guess at the second from the first. What would settle it: an
	// engine-produced ALTER TABLE ... DROP COLUMN fixture run against a
	// constrained table.
	ErrColumnConstrained = errors.New("absdb: column is named by a constraint this package cannot repair")

	// ErrRecordWontFit reports an existing record that would no longer fit
	// its page once re-encoded under the new column layout. It is
	// deliberately distinct from ErrTableFull: that sentinel means no free
	// slot exists, this one means an occupied slot's own record has grown
	// past what the page can hold. Neither implies page splitting, which
	// this package does not do.
	ErrRecordWontFit = errors.New("absdb: an existing record would not fit its page after the column change")
)

ALTER TABLE ADD COLUMN and ALTER TABLE DROP COLUMN.

Read this file's honesty warning before trusting it more than the corpus does. These two operations are the only writes in this package that do not reproduce the engine's bytes, and that is a decision rather than a gap.

The fixtures exist: MultiTable-alteradd.abs and MultiTable-alterdrop.abs are what DBManager wrote for the two statements below, run against MultiTable.abs. They show the engine does not edit a table in place at all. It runs CREATE TABLE <temp> / copy the rows / rename <temp> to the original name / DROP TABLE the original -- four transactions, three catalog writes, a new object id for the table and every one of its columns, the old pages tombstoned and six new ones allocated. ddl_alter_test.go's own section comment lays out the counter-by-counter evidence, and TestEngineAlterTableRebuildsTheTable pins it.

Reproducing that sequence was considered and rejected, and the reason has since expired. It needed six free pages, and at the time nothing here could grow a database: MultiTable.abs is the only file in the corpus that has six, Writes.abs has three, every Employees-*.abs has two, and the private fixtures have between none and five, so an engine-faithful ALTER TABLE would have been byte-perfect on one fixture and refused on every other file this package exists to read. The splice below works on all of them.

ddl_grow.go has since removed that constraint -- the file extends by whole extents on demand -- so the rebuild is no longer blocked, merely not done. It would still have to reproduce all four transactions, the three catalog writes and a fresh set of object ids, which is a good deal more than allocating six pages. Recorded as an open question so the reasoning does not outlive the constraint that produced it.

So the guarantee here is weaker than DropTable's or CreateTable's, and differently shaped: not byte identity, but semantic identity against the engine's own output for the same statement -- same tables, same columns, same rows, checked by TestAlterTableMatchesEngineSemantically -- plus round-trip correctness and the B-tree leaf oracle. Column ids are the one thing that legitimately differs, because the engine's rebuild reallocates them and this splice does not.

Two edits, and the second is the one with teeth:

  1. The column-definition stream (page type 8, zlib-compressed) is edited surgically: schemaColumnSpans locates each column definition's exact byte range using parseColumnDef -- the same function Schema() uses to read the file -- so the splice point is derived, not guessed. Only the bytes between the columnCount field and the first byte past the last column definition are touched. Everything from there on (indexCount, index records, constraint records, the reserved field, and the two trailing page numbers -- see /home/christian/.claude/jobs/61b3bdd6/tmp/index-definition-format.md) is copied through unexamined and unmodified. That tail's own internal layout does not matter to this splice: it only ever moves as a whole block, never parsed except where DropColumn needs to know which columns an index covers (columnCoveredByIndex, below).

  2. Every existing record is decoded under the OLD schema and re-encoded under the NEW one (rewriteDataPages), because nullFlagBytes -- and therefore the byte offset of every field in every record -- depends on the column count. A column count crossing a multiple of 8 changes the null-flag prefix width and shifts every field in every record on every page, not just the field being added or removed. This is exactly the shape of the null-flag sizing bug docs/format/records.md guards against, so TestAlterTableColumnCountBoundary constructs tables that cross a multiple of 8 deliberately rather than relying on the corpus to happen to contain one.

What this file refuses rather than guesses at, and why:

  • a column type serializeColumnDef (ddl_create.go) has no corpus evidence for: reusing that function rather than writing a second serializer is what keeps ADD COLUMN's on-disk column definition byte-identical in shape to CREATE TABLE's, for the two type combinations either has evidence for.
  • a table with BLOB pages: nothing here can rewrite a record carrying a live BLOB reference safely, matching DropTable's own ErrTableHasBlobPages refusal.
  • dropping a column an index covers, or one a constraint names: both records embed their covered columns' names (columnCoveredByIndex, columnNamedByConstraint), which is precise enough to check rather than refusing on any indexed or constrained table outright, the way writer.go's ErrIndexNotMaintained does for insert/delete.
  • a record that would not fit its page after the column change: no page-splitting exists in this package (see writer.go's ErrTableFull), so a page that cannot hold its own existing rows under the new record width is a hard refusal, checked before any byte is written.

Neither operation touches the table catalog, a table's counters file, or any index tree: RecordID identity -- the (page, slot) pair a B-tree leaf entry references -- is preserved exactly, because rewriteDataPages never moves a record to a different slot. That is what lets an existing index keep pointing at the right rows across the schema change without this package having to touch the index pages at all.

View Source 
var (
	// ErrTableExists reports a CREATE TABLE naming a table already present in
	// the catalog.
	ErrTableExists = errors.New("absdb: table already exists")

	// ErrUnsupportedColumnType reports a column type the serializer has no
	// corpus evidence for. Only the two combinations CREATE TABLE Delta (X, Y)
	// exercises -- Int32/Integer and Varchar/String -- are known; guessing at
	// the padding or terminator byte for any other type risks writing a file
	// the engine cannot read back. See docs/... FINDING 2: the stream this
	// serializer writes also carries an index-definition array this package
	// does not build (a fresh table's is empty), so it must be edited
	// surgically rather than through a general re-serializer.
	ErrUnsupportedColumnType = errors.New("absdb: column type has no corpus evidence for CREATE TABLE")

	// ErrColumnDefault reports a column whose definition carries a DEFAULT
	// clause. serializeColumnDef always writes the no-default marker, because
	// Column has no field for a default and CREATE TABLE has no syntax here to
	// declare one. Re-serializing a parsed column that has one would therefore
	// drop it silently -- the table would read back fine and would no longer
	// fill the column in on an insert that omits it -- so the column is
	// refused instead. testdata/Constraints.abs's CDefault is the fixture.
	ErrColumnDefault = errors.New("absdb: column carries a DEFAULT clause this package cannot write")

	// ErrColumnAutoIncOptions reports a column whose AUTOINC parameters are
	// not the engine's defaults. serializeColumnDef writes those defaults
	// unconditionally, so re-serializing such a column would silently reset a
	// real INCREMENT, INITIALVALUE, MINVALUE, MAXVALUE or CYCLED clause.
	// Types.abs's TAutoInc is the only table anywhere that carries any.
	ErrColumnAutoIncOptions = errors.New("absdb: column carries AUTOINC options this package cannot write")
)

CREATE TABLE -- the second schema operation Stage 3 of Phase 8 unblocked. DROP TABLE (ddl.go) never touches a compressed stream; CREATE TABLE writes one (the column definitions), which is why it waited for internal/zlib1.Compress.

What CREATE TABLE Delta (X, Y) writes, measured against MultiTable.abs (see docs/writing.md and the analysis this file was built from):

  • allocates five pages and no data page -- a data page arrives with the first insert: two chained type-7 pages for a 6000-byte all-zero "system" internal file (systemPageNo in the catalog; its role is still unidentified), one type-8 page for the column definitions, one type-9 page for the counters, one type-12 page for the empty record-page index root. Five is the count at a 4096-byte page size only: the system file spans however many pages its bytes need, which is three (and so six pages in all) at 2048 -- see systemFilePageCount;
  • appends a 272-byte entry to the table catalog;
  • writes the column-definition internal file: columnCount, one definition per column, an empty index-definition array, and the 8-byte trailer systemIndexRoots reads;
  • writes the 28-byte counters file tableInfoOffsets already knows the layout of;
  • moves LastUsedPageNo, LastObjectID and the State counters the pages and the header carry.

Object ids are handed out one per table and one per column: docs/format/pages.md records Delta's X and Y taking 13 and 14 after Delta itself took 12, which is what moves LastObjectID from 11 to 14 -- one more than the column count, because the table itself takes one too.

Pages are allocated lowest free page number first: docs/format/pages.md records a table created after a drop taking the freed pages before any higher one.

A newly allocated page's ABSP State is seeded by the engine with an unreproducible random value (see newPageState in ddl.go), so CREATE TABLE cannot be byte-identical to the engine's own output the way DROP TABLE is. TestCreateTableMatchesEngineByteForByte asserts every byte outside the five new pages' State words instead, and says why in its own comment.

View Source 
var (
	// ErrEncryptionUnsupported reports a request to create an encrypted
	// database, or to compact one. The 260 bytes of key material an encrypted
	// file carries at header offsets 80..339 are located but undecoded; see
	// this file's comment.
	ErrEncryptionUnsupported = errors.New("absdb: creating an encrypted database is not supported")

	// ErrBadGeometry reports a CreateDatabaseOptions this package will not
	// build a file from: a page size the format cannot express or that the
	// system pages do not fit in, an extent of no pages, or a connection count
	// out of range.
	ErrBadGeometry = errors.New("absdb: invalid database geometry")
)
View Source 
var (
	// ErrIndexExists reports a CREATE INDEX naming an index that already exists
	// on the table.
	ErrIndexExists = errors.New("absdb: index already exists")

	// ErrNoSuchIndex reports a DROP INDEX naming an index the table does not
	// have.
	ErrNoSuchIndex = errors.New("absdb: no such index")

	// ErrNoSuchColumn reports a CREATE INDEX or a DROP COLUMN (ddl_alter.go)
	// naming a column the table does not have.
	ErrNoSuchColumn = errors.New("absdb: no such column")

	// ErrMultiColumnIndex reports a write to a table carrying an index over
	// more than one column. The records are read now -- Constraints.abs's
	// CIdxMulti pins their layout -- but a multi-column key concatenates its
	// columns into one leaf entry, and this package builds and compares only
	// the single-column, int32-keyed leaf its engine measurement covers. It
	// accompanies ErrIndexNotMaintained rather than replacing it, so a caller
	// can tell this refusal apart from the other index shapes
	// maintainableIndexColumn declines.
	ErrMultiColumnIndex = errors.New("absdb: multi-column indexes are not supported")

	// ErrIndexBacksConstraint reports a DROP INDEX naming the index a PRIMARY
	// KEY or UNIQUE constraint record is built on. Dropping it would leave
	// that record naming an index the file no longer has, and nothing in the
	// corpus says what the engine does about the constraint when its index
	// goes away -- DBManager drops the constraint, not the index. Refusing is
	// the only outcome this package can show to be safe.
	ErrIndexBacksConstraint = errors.New("absdb: index implements a PRIMARY KEY or UNIQUE constraint")

	// ErrUnsupportedIndexColumn reports a CREATE INDEX over a column whose type
	// the leaf-entry format has no corpus evidence for. Every measured index in
	// the corpus covers an Int32/Integer column, and the engine's own leaf
	// entry layout (docs/format/indexes.md) is "[null flag byte] + int32 LE key", so that is
	// the only column type this package builds an index over.
	ErrUnsupportedIndexColumn = errors.New("absdb: index column type has no corpus evidence for CREATE INDEX")

	// ErrSchemaTailNotUnderstood reports a column-definition stream whose tail
	// -- the index array, the constraint array and the two trailing page
	// numbers -- does not parse as the layout ddl_constraint.go documents. It
	// used to cover every table carrying a constraint record at all, which was
	// most real tables; testdata/Constraints.abs retired that, and what is left
	// is the genuinely unknown: a constraint kind the corpus does not show, a
	// reserved field that is not zero, a size field that disagrees with the
	// string it introduces, or bytes left over once both arrays are read.
	ErrSchemaTailNotUnderstood = errors.New("absdb: schema stream tail is not understood")

	// ErrIndexTooManyRows reports a CREATE INDEX over a table with more rows
	// than fit on a single B-tree leaf page. Multi-page index trees are not
	// built by this package.
	ErrIndexTooManyRows = errors.New("absdb: table has too many rows for a single-page index")
)
View Source 
var (
	// ErrRecordSize is returned when the record buffer handed to encodeInto is
	// not exactly Reader.recordSize bytes long.
	ErrRecordSize = errors.New("absdb: record buffer has the wrong size")

	// ErrValueCount is returned when encodeRecord is given a number of values
	// that does not match the table's column count.
	ErrValueCount = errors.New("absdb: wrong number of values for the table's columns")

	// ErrColumnRange is returned for a column index outside the schema.
	ErrColumnRange = errors.New("absdb: column index out of range")

	// ErrValueType is returned when the Go type of a value cannot be stored in
	// the column at all — a string into an INTEGER column, say.
	ErrValueType = errors.New("absdb: value type does not fit the column")

	// ErrValueRange is returned when the value's type is right but the value
	// itself does not fit: outside the column's numeric range, or a string
	// longer than the column holds.
	ErrValueRange = errors.New("absdb: value does not fit the column")

	// ErrStringEncoding is returned for a string the column's character set
	// cannot represent, and for one containing a NUL, which would read back
	// truncated at the terminator.
	ErrStringEncoding = errors.New("absdb: string cannot be stored in the column's character set")

	// ErrBlobWrite is returned for a BLOB, CLOB or WideCLOB column. Writing one
	// needs BLOB page allocation, which is not implemented; this is a scope
	// boundary, not an oversight.
	ErrBlobWrite = errors.New("absdb: BLOB and CLOB columns cannot be written yet")

	// ErrColumnNotWritable is returned for a column whose storage this package
	// can read but not write: Extended, whose 64-bit significand does not
	// survive the round trip through the float64 Record.Float returns, so
	// rewriting a row would quietly truncate a column nobody touched. A GUID
	// column is writable: it stores Char, and its text goes through the
	// string path like any other fixed string. So is a TimeStamp, which the
	// engine keeps only to the hour -- a value carrying minutes or seconds is
	// refused with ErrValueRange rather than silently rounded.
	ErrColumnNotWritable = errors.New("absdb: column type cannot be written yet")
)

Encoding is the inverse of the decode path in reader.go: it turns Go values into the exact bytes a record slot holds. Every convention here was read off real .abs files rather than inferred:

  • Null flags are the same bitmap Record.IsNull reads, and the spare high bits of the last flag byte are set — Employees-*.abs (4 columns, one flag byte, 0xf0), RCFQ0011.abs (7 columns, 0x80) and RCON0011.abs (36 columns, five flag bytes ending 0xf0) all show it, and Reader.validateLayout already rejects a file that lacks it.
  • A NULL column's field bytes are zero in every fixture, so writing a NULL zeroes the field as well as setting the bit.
  • Char/Varchar fields hold Size+1 bytes: the Windows-1252 text, a NUL terminator, then padding. Fresh records written by the Absolute Database Manager zero-pad (Employees-*.abs), but records the engine has updated in place keep whatever the record buffer held before (RCFQ0011.abs stores "Nacht\x00\x46\x40…", the tail of a Double that shared the buffer). Nothing can reconstruct that stale padding from a Go value, so the encoder always zero-pads; the reader stops at the terminator either way.
  • Booleans are a WordBool: 0x0001 for true, 0x0000 for false. No fixture holds any other bit pattern.

BLOB, CLOB and WideCLOB values are out of scope: storing one means allocating BLOB pages and maintaining their chain, which this package does not do yet. Such a column accepts nil — a NULL reference is a complete record — and rejects everything else rather than being half-written.

View Source 
var (
	// ErrNoIndex is returned when the requested index does not exist.
	ErrNoIndex = errors.New("absdb: no index found")

	// ErrKeyNotFound is returned when a lookup finds no matching entry.
	ErrKeyNotFound = errors.New("absdb: key not found")

	// ErrMalformedIndex reports a structurally invalid index: a cyclic page
	// chain, a descent deeper than maxTreeDepth, an empty internal node, a
	// non-leaf page in the leaf chain or a zero-length key.
	ErrMalformedIndex = errors.New("absdb: malformed index")
)
View Source 
var (
	ErrNoData     = errors.New("absdb: no data pages found")
	ErrNoMoreRows = errors.New("absdb: no more rows")
	ErrBadLayout  = errors.New("absdb: record layout does not match the data")
)
View Source 
var (
	ErrNoSchema    = errors.New("absdb: no schema page found")
	ErrBadSchema   = errors.New("absdb: malformed schema data")
	ErrCompression = errors.New("absdb: decompression failed")
)
View Source 
var (
	// ErrReadOnly is returned by every write operation on a database that was
	// opened with Open rather than OpenForWrite.
	ErrReadOnly = errors.New("absdb: database is open read-only")

	// ErrWriterClosed is returned by a TableWriter that has already been
	// committed, rolled back or closed.
	ErrWriterClosed = errors.New("absdb: writer is closed")

	// ErrNoRecord reports a RecordID that does not address an occupied slot of
	// a data page of this table.
	ErrNoRecord = errors.New("absdb: no such record")

	// ErrSlotOccupied reports an insert into a slot that already holds a record.
	ErrSlotOccupied = errors.New("absdb: record slot is occupied")

	// ErrTableFull reports that no record slot could be found or made. It is a
	// page-splitting ceiling, not a space one: running out of free pages no
	// longer stops an insert, because growTable's allocation extends the file
	// by whole extents when it has to (ddl_grow.go). What is left is the
	// table's single-page record-page index root, which holds a fixed number of
	// entries and would have to split to hold more (appendRecordPageEntry), and
	// a row too wide for any record to fit a data page at all. Neither is
	// something growth can lift.
	ErrTableFull = errors.New("absdb: no free record slot in any data page")

	// ErrIndexNotMaintained reports a write against an index this package will
	// not edit, rather than one it silently leaves stale. Single-page indexes
	// over an Int32 column are maintained (see writer_index.go); a tree deep
	// enough to have split, a key of another shape, and a schema whose index
	// definitions cannot be read are all refused here.
	ErrIndexNotMaintained = errors.New("absdb: table has an index this package cannot maintain")

	// ErrConstraintsNotEnforced reports a write against a table whose schema
	// declares constraints, none of which this package checks: nothing here
	// rejects a NULL in a NOT NULL column, a value outside a MINVALUE/MAXVALUE
	// pair, or a duplicate under a PRIMARY KEY or UNIQUE clause. Letting the
	// write through would leave the file holding a row the engine would have
	// refused, which reads back fine here and is not what the engine wrote --
	// the same reason maintainableIndexColumn refuses an index it would order
	// differently. Refusing is the only outcome that cannot corrupt the
	// table's own rules.
	ErrConstraintsNotEnforced = errors.New("absdb: table carries constraints this package does not check")

	// ErrBlobReferenceLost reports an update that would overwrite a column
	// still holding a BLOB reference. The BLOB pages it points at would stay
	// allocated with nothing referring to them, and this package cannot free
	// them yet.
	ErrBlobReferenceLost = errors.New("absdb: update would drop a BLOB reference")

	// ErrBookkeepingMismatch reports that the engine's own record counters do
	// not agree with the records actually stored, so a write cannot bring them
	// forward without guessing.
	ErrBookkeepingMismatch = errors.New("absdb: stored record counts do not match the records on the page")
)
View Source 
var ErrConstraintsNotRebuilt = errors.New("absdb: table carries constraint records a rebuild would lose")

ErrConstraintsNotRebuilt reports a compaction of a table carrying constraint records. A table this package re-creates is built by CreateTable, whose schema stream carries an empty constraint array, so compacting such a table would quietly return a database that no longer enforces its NOT NULL, PRIMARY KEY, UNIQUE or MINVALUE/MAXVALUE rules. Refusing is the only outcome that cannot lose them.

View Source 
var ErrDatabaseTooLarge = errors.New("absdb: database cannot grow past its first allocation map page")

ErrDatabaseTooLarge reports growth that would take the file past what its first allocation map page can describe: more pages than the Page Free Space map on page 0 has bits for, or more extents than the Extent Allocation Map on page 1 has bit pairs for. The engine spills both maps onto further pages at that size; this package does not, because no fixture shows it doing so (the whole corpus tops out at 78 pages), and writing a second map page by guesswork would corrupt the first.

View Source 
var ErrNoPassword = errors.New("absdb: database is encrypted but no password was supplied")

ErrNoPassword indicates that an encrypted database was accessed without a password.

View Source 
var ErrOutOfSpace = errors.New("absdb: not enough free pages")

ErrOutOfSpace reports an allocation that found too few free pages even after the file grew to make room for it. Running out of free pages is no longer a refusal on its own -- allocatePages extends the file by whole extents (see ddl_grow.go) -- so reaching this means the allocation maps and the file disagree about what is free, not that the database is full. Growth that would take the file past what those maps can describe is refused earlier, with ErrDatabaseTooLarge.

View Source 
var ErrRijndaelKeySize = errors.New("absdb: rijndael key must be 16, 24 or 32 bytes")

ErrRijndaelKeySize indicates a key length this cipher does not accept.

View Source 
var ErrSquareKeySize = errors.New("absdb: square key must be 16 bytes")

ErrSquareKeySize indicates a key length this cipher does not accept.

View Source 
var ErrTripleDESKeySize = errors.New("absdb: 3TDES key must be 16 or 24 bytes")

ErrTripleDESKeySize indicates a key length this cipher does not accept.

View Source 
var ErrTwofishKeySize = errors.New("absdb: twofish key must be 16 or 32 bytes")

ErrTwofishKeySize indicates a key length this package does not implement.

View Source 
var ErrUnsupportedCipher = errors.New("absdb: unsupported encryption algorithm")

ErrUnsupportedCipher indicates an unsupported encryption algorithm.

View Source 
var ErrUnsupportedCipherMode = errors.New("absdb: unsupported cipher mode")

ErrUnsupportedCipherMode indicates an unsupported TCipherMode.

View Source 
var ErrWrongPassword = errors.New("absdb: incorrect password")

ErrWrongPassword indicates the provided password does not match.

View Source 
var Magic = [16]byte{
	'A', 'B', 'S', '0', 'L', 'U', 'T', 'E',
	'D', 'A', 'T', 'A', 'B', 'A', 'S', 'E',
}

Magic is the 16-byte file signature: "ABS0LUTEDATABASE" (note: zero, not letter O).

Functions

func CompactDatabase added in v0.1.1 

func CompactDatabase(srcPath, dstPath string) error

CompactDatabase writes a compacted copy of the database at srcPath to dstPath, which must not already exist.

It is the engine's own Database -> Compact Database: a rebuild into a new file rather than a defragment in place. The result holds the same tables, in the same order, with the same columns, rows and indexes, on the smallest run of pages they fit on -- no free page at all -- with a reset transaction counter and freshly allocated object ids. See this file's comment for the evidence, and for what the rebuild reproduces of the engine's own output.

The source is opened read-only and is never modified. Everything the rebuild cannot reproduce is refused before dstPath is created, so a refusal leaves no file behind; see "What stays refused" in this file's comment for the list.

Types

type BTreeEntry 

type BTreeEntry struct {
	Key    []byte // key bytes (KeyPrefixSize bytes)
	PageNo int32  // referenced page number
	ItemNo uint16 // referenced item number within the page (leaf entries only)
}

BTreeEntry is a single entry in an index page.

func (BTreeEntry) RecordID 

func (e BTreeEntry) RecordID() (int32, uint16)

RecordID returns the entry's reference as a (PageNo, ItemNo) pair.

type BTreePageHeader 

type BTreePageHeader struct {
	IsRoot         bool
	IsLeaf         bool
	LeftPageNo     int32 // left sibling page (-1 = none)
	RightPageNo    int32 // right sibling page (-1 = none)
	HasKeys        bool
	HasSuffixes    bool
	KeyPrefixSize  uint16 // key size per entry (short key)
	EntryCount     uint16 // number of entries on this page
	PagePrefixSize uint16 // page-level prefix size
}

BTreePageHeader is the on-disk header at the start of every index page body.

type BaseFieldType 

type BaseFieldType byte

BaseFieldType represents the low-level storage type (TABSBaseFieldType). 

const (
	BftUnknown     BaseFieldType = 0
	BftChar        BaseFieldType = 1
	BftWideChar    BaseFieldType = 2
	BftVarchar     BaseFieldType = 3
	BftWideVarchar BaseFieldType = 4
	BftInt8        BaseFieldType = 5
	BftInt16       BaseFieldType = 6
	BftInt32       BaseFieldType = 7
	BftInt64       BaseFieldType = 8
	BftUint8       BaseFieldType = 9
	BftUint16      BaseFieldType = 10
	BftUint32      BaseFieldType = 11
	BftSingle      BaseFieldType = 12
	BftDouble      BaseFieldType = 13
	BftExtended    BaseFieldType = 14
	BftDate        BaseFieldType = 15
	BftTime        BaseFieldType = 16
	BftDateTime    BaseFieldType = 17
	BftBlob        BaseFieldType = 18
	BftClob        BaseFieldType = 19
	BftWideClob    BaseFieldType = 20
	BftLogical     BaseFieldType = 21
	BftCurrency    BaseFieldType = 22
	BftBytes       BaseFieldType = 23
	BftVarBytes    BaseFieldType = 24
)

type BlobRef 

type BlobRef struct {
	PageNo int32  // page number where BLOB data starts
	ItemNo uint16 // item number within the page (usually 0)
}

BlobRef is a reference to BLOB data stored in a record field.

func (BlobRef) IsNull 

func (ref BlobRef) IsNull() bool

IsNull returns true if the BLOB reference points to no data.

type Column 

type Column struct {
	Name      string        // column name
	ID        uint32        // internal column ID
	BaseType  BaseFieldType // low-level storage type
	FieldType FieldType     // high-level field type
	Size      uint32        // max size for variable-length types (string length, etc.)
	Position  int           // 0-based position in the column list
	// contains filtered or unexported fields
}

Column describes a single column in a table.

func (Column) HasDefault added in v0.1.1 

func (c Column) HasDefault() bool

HasDefault reports whether this column's definition carries a DEFAULT value. The value itself is not decoded; what this answers is whether re-serializing the column would drop a clause the engine wrote, which is why CreateTable and compaction refuse a column that has one.

func (Column) IsBLOB 

func (c Column) IsBLOB() bool

IsBLOB returns true if this column stores BLOB data (Memo, Graphic, etc.).

func (Column) NotNull added in v0.1.1 

func (c Column) NotNull() (notNull, known bool)

NotNull reports whether a NOT NULL constraint record names this column.

known is false when the table's constraint array was not read -- because the column came from parseSchema directly, or because the schema tail did not parse -- and it is the difference between "this column is nullable" and "this was never established". A caller that treats an unknown as nullable is making the guess this package otherwise refuses to make for it.

type CreateDatabaseOptions added in v0.1.1 

type CreateDatabaseOptions struct {
	// PageSize is the size of one page in bytes. 4096 (Empty.abs) and 2048
	// (Empty-p2048-e4.abs) are the two values the fixtures pin; any other value
	// large enough to hold the system pages is accepted but unmeasured.
	PageSize int

	// PageCountInExtent is how many pages an extent groups, which is the unit
	// the file grows by (ddl_grow.go). 8 and 4 are the two the fixtures pin.
	PageCountInExtent int

	// MaxConnections sizes the connection/lock table on page 3. 500 and 100 are
	// the two the fixtures pin.
	MaxConnections int

	// Encrypted requests an encrypted database, which is refused with
	// ErrEncryptionUnsupported. The field exists so that asking is an explicit
	// refusal rather than a silently unencrypted file.
	Encrypted bool
}

CreateDatabaseOptions is the geometry of a database CreateDatabase makes. The zero value selects the DBManager's own defaults -- 4096-byte pages, 8 pages to an extent, 500 connections -- which is what Empty.abs carries.

type CryptoAlgorithm 

type CryptoAlgorithm byte

CryptoAlgorithm identifies the encryption algorithm (TABSCryptoAlgorithm). 

const (
	CryptoRijndael128 CryptoAlgorithm = 0
	CryptoRijndael256 CryptoAlgorithm = 1
	CryptoDESSingle   CryptoAlgorithm = 2
	CryptoDESTriple   CryptoAlgorithm = 3
	CryptoBlowfish    CryptoAlgorithm = 4
	CryptoTwofish128  CryptoAlgorithm = 5
	CryptoTwofish256  CryptoAlgorithm = 6
	CryptoSquare      CryptoAlgorithm = 7
)

func (CryptoAlgorithm) String added in v0.1.1 

func (a CryptoAlgorithm) String() string

String returns the algorithm name as used by the Absolute Database UI.

type CryptoHeader 

type CryptoHeader struct {
	HeaderSize   int16
	Algorithm    CryptoAlgorithm
	Mode         byte
	ControlBlock [controlBlockSize]byte
	ControlCRC   uint32
}

CryptoHeader holds the parsed TABSCryptoHeader from page 0.

type DiskPageHeader 

type DiskPageHeader struct {
	State      int32  // page state
	PageType   uint16 // page type (see PageType* constants)
	NextPageNo int32  // next page in chain (-1 = none)
	CRC32      uint32 // CRC32 checksum
	CRCType    byte
	HashType   byte
	CipherType byte
	MACType    byte
	ObjectID   int32  // table/object this page belongs to (-1 = system)
	RecPageNo  int32  // record ID: page number
	RecItemNo  uint16 // record ID: item number within page
}

DiskPageHeader is the 40-byte TABSDiskPageHeader found at offset 0x17C in every page.

type FieldType 

type FieldType byte

FieldType represents the high-level field type (TABSAdvancedFieldType). 

const (
	FieldUnknown       FieldType = 0
	FieldChar          FieldType = 1
	FieldString        FieldType = 2
	FieldWideChar      FieldType = 3
	FieldWideString    FieldType = 4
	FieldShortInt      FieldType = 5
	FieldSmallInt      FieldType = 6
	FieldInteger       FieldType = 7
	FieldLargeInt      FieldType = 8
	FieldByte          FieldType = 9
	FieldWord          FieldType = 10
	FieldCardinal      FieldType = 11
	FieldAutoInc       FieldType = 12
	FieldAutoIncInt8   FieldType = 13
	FieldAutoIncInt16  FieldType = 14
	FieldAutoIncInt32  FieldType = 15
	FieldAutoIncInt64  FieldType = 16
	FieldAutoIncUint8  FieldType = 17
	FieldAutoIncUint16 FieldType = 18
	FieldAutoIncUint32 FieldType = 19
	FieldSingle        FieldType = 20
	FieldDouble        FieldType = 21
	FieldExtended      FieldType = 22
	FieldBoolean       FieldType = 23
	FieldCurrency      FieldType = 24
	FieldDate          FieldType = 25
	FieldTime          FieldType = 26
	FieldDateTime      FieldType = 27
	FieldTimeStamp     FieldType = 28
	FieldBytes         FieldType = 29
	FieldVarBytes      FieldType = 30
	FieldBLOB          FieldType = 31
	FieldGraphic       FieldType = 32
	FieldMemo          FieldType = 33
	FieldFmtMemo       FieldType = 34
	FieldWideMemo      FieldType = 35
	FieldGUID          FieldType = 36
)

func (FieldType) String 

func (ft FieldType) String() string

String returns the field type name.

type File 

type File struct {
	// contains filtered or unexported fields
}

File represents an opened Absolute Database file.

func CreateDatabase added in v0.1.1 

func CreateDatabase(path string, opts CreateDatabaseOptions) (*File, error)

CreateDatabase writes a new, empty Absolute Database file at path and returns it open for writing. The caller closes it.

The file it writes is the one the DBManager's File -> Create Database produces for the same settings, byte for byte apart from the three randomly seeded page State words no writer can reproduce; see this file's comment for the layout and the fixtures behind every field of it.

It refuses rather than guess when the geometry is one the layout does not fit (ErrBadGeometry) and when encryption is asked for (ErrEncryptionUnsupported). path must not already exist: the file is created exclusively, so CreateDatabase can never overwrite a database.

func Open 

func Open(path string) (*File, error)

Open opens an Absolute Database file for reading.

func OpenForWrite added in v0.1.1 

func OpenForWrite(path string) (*File, error)

OpenForWrite opens an Absolute Database file for reading and writing.

Nothing is written until a TableWriter is committed. Opening for write on its own does not modify the file.

func OpenForWriteWithPassword added in v0.1.1 

func OpenForWriteWithPassword(path, password string) (*File, error)

OpenForWriteWithPassword opens an encrypted file for reading and writing. It reports the same errors as OpenWithPassword.

func OpenWithPassword 

func OpenWithPassword(path, password string) (*File, error)

OpenWithPassword opens an encrypted Absolute Database file. If the file is not encrypted, the password is ignored.

It returns ErrWrongPassword when the password does not match and ErrUnsupportedCipher when the file uses a cipher this package cannot decrypt; the two are distinct because a caller can do nothing about the latter.

func (*File) AddColumn added in v0.1.1 

func (db *File) AddColumn(table string, column Column) error

AddColumn appends a new column to the end of a table's schema. Every existing row reads back with the new column NULL.

It fails with ErrReadOnly unless the file was opened with OpenForWrite, and refuses rather than guess when the table already has a column of that name, when the column's type has no corpus evidence backing its on-disk encoding (ErrUnsupportedColumnType), when the table owns BLOB pages (ErrTableHasBlobPages), or when adding the column would make an existing record too wide for its page (ErrRecordWontFit).

func (*File) Close 

func (db *File) Close() error

Close closes the database file.

func (*File) CreateIndex added in v0.1.1 

func (db *File) CreateIndex(table, index, column string) error

CreateIndex adds a single-column index to a table.

It fails with ErrReadOnly unless the file was opened with OpenForWrite, and refuses rather than guess when: the table does not exist (ErrNoSuchTable), the index name is already used (ErrIndexExists), the column does not exist (ErrNoSuchColumn), the column is not an Int32/Integer column (ErrUnsupportedIndexColumn), the table's schema stream tail does not parse (ErrSchemaTailNotUnderstood), or the table has more rows than fit on one index leaf page (ErrIndexTooManyRows).

A table carrying NOT NULL, PRIMARY KEY, UNIQUE or MINVALUE/MAXVALUE constraints is no longer refused: its constraint records are parsed, and the new index record is spliced in ahead of them so they come back byte for byte. Note that the new index is a plain one -- CreateIndex neither creates nor enforces a constraint.

func (*File) CreateTable added in v0.1.1 

func (db *File) CreateTable(name string, columns []Column) error

CreateTable adds a new, empty table to the database: no rows, no index and no BLOB column. Columns are given in name/type order; any ID or Position the caller sets on them is ignored, because the engine assigns both itself (see the file comment).

It fails with ErrReadOnly unless the file was opened with OpenForWrite, and refuses rather than guess when the table already exists, when the catalog cannot be grown in place, or when a column's type has no corpus evidence backing its on-disk encoding (ErrUnsupportedColumnType). A file without enough free pages is no longer among those refusals: it grows by whole extents to make room, the way the engine does (ddl_grow.go).

func (*File) CryptoHeader 

func (db *File) CryptoHeader() *CryptoHeader

CryptoHeader returns the parsed crypto header, or nil if the file is not encrypted.

func (*File) DropColumn added in v0.1.1 

func (db *File) DropColumn(table, column string) error

DropColumn removes a column from a table. Every remaining column keeps its original value in every row.

It fails with ErrReadOnly unless the file was opened with OpenForWrite, and refuses rather than guess when the table has no such column, when it is the table's only column (ErrLastColumn), when an index covers the column (ErrColumnIndexed), when a constraint record names the column (ErrColumnConstrained), when the table owns BLOB pages (ErrTableHasBlobPages), or when dropping the column would still leave a record wider than its page (ErrRecordWontFit) -- a defensive check, since dropping a column only ever shrinks a record, but one this package makes rather than assumes.

AddColumn carries no equivalent constraint check: a constraint record can only name a column that already exists, and AddColumn's new column has no name any existing constraint could already be referring to. Nothing in this package rewrites the constraint region either way -- it is always copied through byte for byte -- so the only way it can go stale is losing a column it still names, which is DropColumn's risk alone.

func (*File) DropIndex added in v0.1.1 

func (db *File) DropIndex(table, index string) error

DropIndex removes an index from a table, freeing its B-tree pages.

It fails with ErrReadOnly unless the file was opened with OpenForWrite, and refuses rather than guess when the table does not exist (ErrNoSuchTable), the index does not exist (ErrNoSuchIndex), the table's schema stream tail does not parse (ErrSchemaTailNotUnderstood), or the index implements a PRIMARY KEY or UNIQUE constraint (ErrIndexBacksConstraint).

func (*File) DropTable added in v0.1.1 

func (db *File) DropTable(name string) error

DropTable removes a table and every page it owns from the database.

It reproduces what the engine's own DROP TABLE writes, byte for byte; see the file comment for what that is. It fails with ErrReadOnly unless the file was opened with OpenForWrite, and refuses rather than guess when the table owns BLOB pages, when it is the database's only table, or when the file holds a page that cannot be attributed to a table.

Like every write here, the change is not crash-atomic: a crash midway leaves some pages written.

func (*File) Encrypted 

func (db *File) Encrypted() bool

Encrypted returns true if the database is encrypted.

func (*File) OpenIndex 

func (db *File) OpenIndex() (*IndexReader, error)

OpenIndex creates an IndexReader by scanning all index pages.

func (*File) OpenTable 

func (db *File) OpenTable() (*Reader, error)

OpenTable creates a Reader over the database's only table. It reports ErrAmbiguousTable when the file holds more than one; use Table to name it.

func (*File) OpenTableWriter added in v0.1.1 

func (db *File) OpenTableWriter() (*TableWriter, error)

OpenTableWriter opens the database's only table for modification. It fails with ErrReadOnly unless the file was opened with OpenForWrite, and reports ErrAmbiguousTable when the file holds more than one table.

func (*File) PageCount 

func (db *File) PageCount() int

PageCount returns the total number of pages in the file.

func (*File) PageSize 

func (db *File) PageSize() int

PageSize returns the page size in bytes.

func (*File) ReadBlob 

func (db *File) ReadBlob(ref BlobRef) ([]byte, error)

ReadBlob reads the BLOB data for the given reference from the database. Returns the raw (decompressed) bytes. For Memo fields, convert to string.

func (*File) ReadPage 

func (db *File) ReadPage(n int) (Page, error)

ReadPage reads a single page by its zero-based page number.

A page spans pageSize+diskPageHeaderOffset bytes on disk: the whole block the page starts in plus the leading diskPageHeaderOffset bytes of the next block, which still belong to this page's payload. Both are fetched with one ReadAt.

func (*File) ScanPages 

func (db *File) ScanPages() ([]PageSummary, error)

ScanPages reads all pages and returns their disk page headers.

func (*File) Schema 

func (db *File) Schema() (*TableSchema, error)

Schema reads and parses the schema of the database's only table. It reports ErrAmbiguousTable when the file holds more than one; use Table to name it.

func (*File) Table added in v0.1.1 

func (db *File) Table(name string) (*Table, error)

Table returns a handle to the named table. Names are matched without regard to case, as the engine's own SQL does.

An empty name selects the database's only table. It reports ErrAmbiguousTable when there is more than one, which is the case this package used to get silently wrong: it read every data page in the file through the first table's schema, so a second table's rows came back as the first table's garbage.

func (*File) Tables added in v0.1.1 

func (db *File) Tables() ([]TableInfo, error)

Tables returns the database's tables in catalog order.

func (*File) Unlock added in v0.1.1 

func (db *File) Unlock(password string) error

Unlock verifies the password and installs the derived key, so that subsequent page reads are decrypted. It is a no-op for unencrypted files.

func (*File) VerifyPassword 

func (db *File) VerifyPassword(password string) bool

VerifyPassword reports whether the given password is correct for this encrypted database. It returns false for unencrypted files and for files whose cipher this package cannot use; Unlock distinguishes those cases.

func (*File) Version 

func (db *File) Version() float64

Version returns the database engine version (e.g. 5.13, 7.10, 7.61).

func (*File) Writable added in v0.1.1 

func (db *File) Writable() bool

Writable reports whether the file was opened for writing.

type GUID added in v0.1.1 

type GUID [16]byte

GUID is a 128-bit globally unique identifier, held in the order it is printed: g[0] is the first hex pair of the first group.

The engine does not store the Win32 GUID struct, whatever TABSGuid's typedef suggests. A GUID column is a fixed 38-character Char column and holds the value as text -- Types.abs's TGuid stores the bytes "{3F2504E0-4F89-11D3-9A0C-0305E82C3301}" followed by a NUL -- so there is no endianness to reverse and no struct to unpack.

func ParseGUID added in v0.1.2 

func ParseGUID(s string) (GUID, bool)

ParseGUID reads the engine's stored text. Both forms the engine accepts are taken -- braced as DBManager writes it, and bare as it stores a bare literal -- and anything else reports false rather than a partly filled value.

func (GUID) String added in v0.1.1 

func (g GUID) String() string

String formats the GUID canonically, as 8-4-4-4-12 lowercase hex digits with no braces. The zero GUID formats as all zeros.

type IndexInfo 

type IndexInfo struct {
	RootPageNo int  // root page of the B-tree
	KeySize    int  // key size in bytes
	EntryCount int  // entries on the root page (whole tree only for root-only trees)
	IsInternal bool // true for system indexes (RecordPage, BlobPage)
}

IndexInfo describes a discovered index.

type IndexReader 

type IndexReader struct {
	// contains filtered or unexported fields
}

IndexReader provides index-based lookups on a table.

func (*IndexReader) FindByPrimaryKey 

func (ir *IndexReader) FindByPrimaryKey(key int32) (dataPageNo int32, itemNo uint16, err error)

FindByPrimaryKey looks up a record by its primary key (AutoInc/RecNo value). Returns the data page number and item number, or ErrKeyNotFound.

func (*IndexReader) FindByStringKey 

func (ir *IndexReader) FindByStringKey(value string) (dataPageNo int32, itemNo uint16, err error)

FindByStringKey searches a secondary string index for the given value.

Known limitation: index definitions are not parsed yet, so an index cannot be mapped to the column it covers. The lookup therefore always uses the first secondary index of the table and silently returns ErrKeyNotFound for values of any other indexed column.

func (*IndexReader) Indexes 

func (ir *IndexReader) Indexes() []IndexInfo

Indexes returns information about all discovered indexes, system indexes included. Use UserIndexes to get only the indexes over table rows.

func (*IndexReader) PrimaryKeyIndex 

func (ir *IndexReader) PrimaryKeyIndex() (IndexInfo, error)

PrimaryKeyIndex returns the user index over the primary key: the one with primaryKeySize keys (1 null flag byte + int32). Tables whose primary key is composite have no such index and yield ErrNoIndex.

func (*IndexReader) ScanIndex 

func (ir *IndexReader) ScanIndex(rootPageNo int) ([]BTreeEntry, error)

ScanIndex reads all entries from the specified index root page, following the B-tree leaf chain for multi-page indexes.

func (*IndexReader) SecondaryIndexes 

func (ir *IndexReader) SecondaryIndexes() []IndexInfo

SecondaryIndexes returns the user indexes other than the primary key index, that is UserIndexes minus the primaryKeySize entry.

func (*IndexReader) UserIndexes added in v0.1.1 

func (ir *IndexReader) UserIndexes() []IndexInfo

UserIndexes returns every index defined over table rows, that is all discovered indexes except the engine's internal page indexes (systemKeySize keys). Only user index entries reference real rows: the PageNo of a system index entry is an engine-internal value, not a data page number.

type Page 

type Page struct {
	Number  int
	Data    []byte
	Payload []byte
	Header  *DiskPageHeader // nil if no ABSP marker found
	// contains filtered or unexported fields
}

Page represents a single page read from the database file.

Data is the raw pageSize-byte block the page starts in, kept so that the ABSP header at diskPageHeaderOffset can be located at its documented offset. Payload is the page's usable data area: pageSize-diskPageHeaderSize bytes running from pageDataOffset in this block into the first diskPageHeaderOffset bytes of the next block. Data and Payload overlap and share one backing array.

func (Page) Freed added in v0.1.1 

func (p Page) Freed() bool

Freed reports whether the engine has released this page. A freed page keeps its type, its owner and its old contents, so nothing but this distinguishes a dropped table's data page from a live one.

func (Page) IsEmpty 

func (p Page) IsEmpty() bool

IsEmpty returns true if the page's block (Data, the raw pageSize bytes this page starts in, header included) contains only zero bytes. It deliberately covers the block rather than the payload: an all-zero block carries no ABSP header and is therefore never encrypted.

func (Page) PageData 

func (p Page) PageData() []byte

PageData returns the usable data portion of the page: everything after this page's disk page header, continuing into the next block up to that block's header. For a 4096-byte page this is 4056 bytes.

type PageSummary 

type PageSummary struct {
	Number int
	Empty  bool
	Header *DiskPageHeader
}

PageSummary is a lightweight summary of a scanned page.

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

Reader iterates over data records in a table.

The record layout is derived from the schema, not searched for: 

nullFlagBytes  = ceil(numColumns / 8)          // spare high bits are set
fieldDataSize  = sum(fieldStoreSize(col))
recordSize     = nullFlagBytes + fieldDataSize // no trailer, no padding
recordsPerPage = max n with ceil(n/8) + n*recordSize <= payload
bitmapBytes    = ceil(recordsPerPage / 8)

A data page therefore starts with a bitmapBytes-long occupancy bitmap (bit set = slot occupied), followed by recordsPerPage fixed-size record slots.

func (*Reader) Err 

func (r *Reader) Err() error

Err returns any error encountered during iteration.

func (*Reader) Next 

func (r *Reader) Next() bool

Next advances to the next occupied record slot. Returns false when no more records are available. Record may be called any number of times per Next.

func (*Reader) Record 

func (r *Reader) Record() Record

Record returns the record the Reader is currently positioned on. It has no side effects: calling it twice for the same Next returns the same record. Before the first Next, or after Next reported false, it returns a record whose columns all read as NULL.

func (*Reader) RecordID added in v0.1.1 

func (r *Reader) RecordID() (RecordID, bool)

RecordID returns the identity of the record the Reader is currently positioned on, for handing to a TableWriter. The second result is false before the first Next and after Next has reported false.

It lives here rather than in reader.go because addressing a record only matters when something is going to write to it.

func (*Reader) Schema 

func (r *Reader) Schema() *TableSchema

Schema returns the table schema.

type Record 

type Record struct {
	// contains filtered or unexported fields
}

Record represents a single data record.

func (Record) Blob 

func (rec Record) Blob(col int) ([]byte, error)

Blob reads the BLOB data for the given column from the database. Returns the raw bytes. Returns nil for NULL BLOBs.

func (Record) BlobRef 

func (rec Record) BlobRef(col int) BlobRef

BlobRef returns the BLOB reference for a BLOB/Memo/Graphic column. It returns a null reference when col is out of range or the record's field data is too short to hold the reference.

func (Record) Bool 

func (rec Record) Bool(col int) bool

Bool returns the boolean value of the column. WordBool is 2 bytes, non-zero meaning true.

func (Record) Bytes 

func (rec Record) Bytes(col int) []byte

Bytes returns a copy of the raw stored bytes for the column, or nil if the column index is out of range or its field is truncated.

func (Record) Float 

func (rec Record) Float(col int) float64

Float returns the floating-point value of the column: Single is read as a 4-byte IEEE-754 value, Double and Currency as 8-byte ones, and Extended as an x87 80-bit one rounded to float64. All other columns return 0.

Extended is the one lossy conversion here. It carries a 64-bit significand against float64's 53, so a value the engine wrote can round on the way out and cannot be written back unchanged -- which is why an Extended column stays refused by the write path with ErrColumnNotWritable.

func (Record) GUID added in v0.1.1 

func (rec Record) GUID(col int) GUID

GUID returns the value of a GUID column. Any other column, a NULL, and text that is not a GUID all yield the zero GUID, which is what every other typed accessor here does for a column it cannot read; use IsNull to tell a NULL apart from a zero value.

This is the one accessor that dispatches on FieldType rather than BaseType, because it has to: a GUID column stores Char, so BaseType alone cannot tell it from any other fixed string. String reads the same column as its raw text, braces and all.

func (Record) Int 

func (rec Record) Int(col int) int32

Int returns the value of an integer column widened to int32. Narrower columns (Int8, Uint8, Int16, Uint16) are widened keeping their own sign. Columns that do not store a plain integer return 0, and so do values that do not fit an int32 — an Int64 column, or a Uint32 column above MaxInt32. Use Int64 to read those without loss.

func (Record) Int16 added in v0.1.1 

func (rec Record) Int16(col int) int16

Int16 returns the value of a SmallInt column. Columns that do not store a plain integer return 0, and so do values outside the int16 range — reading a wider column through Int16 yields 0 rather than its truncated low bytes.

func (Record) Int64 

func (rec Record) Int64(col int) int64

Int64 returns the value of an integer column widened to int64. A Currency column is not an integer column -- it stores a double -- and reads 0 here; use Float.

func (Record) IsNull 

func (rec Record) IsNull(col int) bool

IsNull returns true if the column at the given index is null.

func (Record) Memo 

func (rec Record) Memo(col int) (string, error)

Memo reads a Memo (text BLOB) column and returns it as a string. Returns empty string for NULL values.

func (Record) String 

func (rec Record) String(col int) string

String returns the string value of the column. Char and Varchar columns are decoded from Windows-1252, WideChar and WideVarchar from UTF-16LE; both stop at the first null terminator. Other columns return the empty string — BLOB and CLOB columns hold only a reference, use Memo to read their text.

func (Record) Time 

func (rec Record) Time(col int) time.Time

Time returns the time.Time value of a Date, Time, DateTime or TimeStamp column. Any other column, and any truncated field, yields the zero time.

A TimeStamp is only accurate to the hour, because that is all the engine keeps: see timeStampToTime.

func (Record) Uint16 added in v0.1.1 

func (rec Record) Uint16(col int) uint16

Uint16 returns the value of a Word column. Columns that do not store a plain integer return 0, and so do values outside the uint16 range — including the negative value of a signed column, which is not reinterpreted as a large unsigned one.

func (Record) Uint32 

func (rec Record) Uint32(col int) uint32

Uint32 returns the value of an unsigned integer column. Narrower columns are zero-extended. Columns that do not store a plain integer return 0, and so do values outside the uint32 range — including the negative value of a signed column, which is not reinterpreted as a large unsigned one.

type RecordID added in v0.1.1 

type RecordID struct {
	PageNo int
	Slot   int
}

RecordID addresses a single record slot: the data page it lives on and its slot index within that page. It is stable as long as the record is not deleted, because no write operation moves a record between slots.

type Table added in v0.1.1 

type Table struct {
	// contains filtered or unexported fields
}

Table is a handle to one table of a database. It is the scope every read and write is performed in: a Reader built from it sees only its own data pages, and a TableWriter built from it advances only its own counters.

func (*Table) Info added in v0.1.1 

func (t *Table) Info() TableInfo

Info returns the catalog entry this handle was built from.

func (*Table) Name added in v0.1.1 

func (t *Table) Name() string

Name returns the table's name, which is empty for a file with no catalog.

func (*Table) Open added in v0.1.1 

func (t *Table) Open() (*Reader, error)

Open creates a Reader for this table's data records. A table without data pages yields a valid Reader whose Next reports no rows.

func (*Table) OpenIndex added in v0.1.1 

func (t *Table) OpenIndex() (*IndexReader, error)

OpenIndex creates an IndexReader over the indexes this table owns.

Attribution is by evidence, not by ObjectID: an index page's ABSP header records no owner, so the only thing that ties an index to a table is where its entries point. A user index is this table's when its leaf entries name this table's data pages; the engine's internal record-page index is this table's when its keys are those pages.

Two cases are therefore not returned for a multi-table database: an index whose leftmost leaf is empty, which offers no evidence either way, and the engine's BLOB page index, whose keys are BLOB pages and whose owner nothing in the file records. Neither arises for a single-table file, where every index in the file is returned because there is no other table it could belong to.

func (*Table) OpenWriter added in v0.1.1 

func (t *Table) OpenWriter() (*TableWriter, error)

OpenWriter opens this table for modification. It fails with ErrReadOnly unless the file was opened with OpenForWrite.

func (*Table) Schema added in v0.1.1 

func (t *Table) Schema() (*TableSchema, error)

Schema reads and parses this table's column definitions.

type TableInfo added in v0.1.1 

type TableInfo struct {
	// Name is the table's name, Windows-1252 decoded.
	//
	// The SoundPlan fixtures store the file's own name here, extension and all
	// ("RCON0011.abs"), because each of their tables lives in its own database.
	// That is what the engine was given, not something this package adds.
	Name string

	// ID is the table's identifier, and is what ABSP.ObjectID holds on each of
	// its data pages. It is the only field that partitions pages by table:
	// schema, table-info, index and BLOB pages all carry ObjectID 0xFFFFFFFF.
	ID int

	// SchemaPageNo is the page holding this table's compressed column
	// definitions (page type 8).
	SchemaPageNo int

	// InfoPageNo is the page holding this table's record and change counters
	// (page type 9).
	InfoPageNo int
	// contains filtered or unexported fields
}

TableInfo is one entry of the database's table catalog.

The catalog lives in the type-6 system internal file, a plain array of 272-byte TABSTableListItem records with no count field of its own: the internal file header's decompressed length divides by the entry size to give the number of tables.

type TableSchema 

type TableSchema struct {
	Columns []Column
}

TableSchema holds the parsed schema for one table.

type TableWriter added in v0.1.1 

type TableWriter struct {
	// contains filtered or unexported fields
}

TableWriter modifies the records of a table. Changes are buffered until Commit; see the file comment for what Commit does and does not guarantee.

A TableWriter holds its own view of the pages it touches, so a Reader opened on the same File keeps seeing the committed state until the writer commits.

func (*TableWriter) Close added in v0.1.1 

func (w *TableWriter) Close() error

Close rolls back any uncommitted changes. It is safe to call after Commit, which makes `defer w.Close()` the correct idiom next to an explicit Commit.

func (*TableWriter) Commit added in v0.1.1 

func (w *TableWriter) Commit() error

Commit writes every modified page back and flushes the file. The writer is closed afterwards, whether or not the write succeeded: a failed commit may have written some pages already, so continuing to use the buffers would build on a state that is no longer known.

func (*TableWriter) Delete added in v0.1.1 

func (w *TableWriter) Delete(id RecordID) error

Delete removes the record at id by clearing its occupancy bit. The record's bytes are left in place, which is what the engine itself does: the slot is free, and the next insert overwrites as much of it as the new record needs.

func (*TableWriter) Insert added in v0.1.1 

func (w *TableWriter) Insert(values []any) (RecordID, error)

Insert stores a new record in the first free slot of the first data page that has one, and returns the slot it used. It reports ErrTableFull when every slot of every data page is occupied.

func (*TableWriter) Record added in v0.1.1 

func (w *TableWriter) Record(id RecordID) (Record, error)

Record returns the current bytes of a record as this writer sees them, including changes buffered but not yet committed.

func (*TableWriter) Rollback added in v0.1.1 

func (w *TableWriter) Rollback()

Rollback discards every buffered change. Because nothing is written before Commit, the file is left exactly as it was.

func (*TableWriter) Schema added in v0.1.1 

func (w *TableWriter) Schema() *TableSchema

Schema returns the schema of the table being written.

func (*TableWriter) Update added in v0.1.1 

func (w *TableWriter) Update(id RecordID, values []any) error

Update overwrites the record at id with values, one per column. A nil value writes a NULL. The record must exist; Update never creates one.

An update that moves an indexed column's value moves that index entry with it, as a removal followed by a sorted insertion — what Writes-idx-upd.abs shows the engine doing. An update that leaves every indexed column alone writes no index page at all, so it does not advance a State counter for a page whose contents did not change.

This is what the index caveat here used to warn about: before the covered column could be read out of the schema stream, Update went through and left the index describing a key the row no longer had.

func (*TableWriter) UpdateColumn added in v0.1.1 

func (w *TableWriter) UpdateColumn(id RecordID, col int, value any) error

UpdateColumn overwrites a single column of an existing record, leaving every other column byte-for-byte as it was. It is the narrowest write this package offers, and the only one that cannot disturb a column it was not asked about.

It maintains indexes exactly as Update does, so updating an indexed column through it moves that index entry too.

Source Files

View all Source files

Directories

Path Synopsis
cmd
absdb command
Command absdb inspects and dumps ComponentAce Absolute Database (.abs) files.
 Command absdb inspects and dumps ComponentAce Absolute Database (.abs) files.
internal
zlib1
Package zlib1 is a port of the C zlib library's compression level 1, exact enough to reproduce its output byte for byte.
 Package zlib1 is a port of the C zlib library's compression level 1, exact enough to reproduce its output byte for byte.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.