harness

Documentation

Overview

Package harness wires the ana CLI against a real TextQL endpoint for live smoke tests. It duplicates the cmd/ana/main.go verb-building logic so tests can call CLI verbs and raw RPCs through the same transport.

Index

• type APIKeyHandle
• type ConnSpec
• type H
  • func Begin(t *testing.T) *H
  • func (h *H) Client() *transport.Client
  • func (h *H) CreateAPIKey(suffix string) APIKeyHandle
  • func (h *H) CreateChat(suffix string, connectorIDs []int) string
  • func (h *H) CreateConnector(suffix string, spec ConnSpec) int
  • func (h *H) CreateServiceAccount(suffix string) string
  • func (h *H) DryRun() bool
  • func (h *H) End()
  • func (h *H) Endpoint() string
  • func (h *H) ExpectOrgID() string
  • func (h *H) MustNotTouch(resourceType string, id any)
  • func (h *H) RecordManualRevert(what, reason string)
  • func (h *H) Register(fn func())
  • func (h *H) RegisterAPIKeyCleanupByName(name string)
  • func (h *H) RegisterConnectorCleanup(id int)
  • func (h *H) RegisterConnectorCleanupByName(name string)
  • func (h *H) RegisterServiceAccountCleanupByName(name string)
  • func (h *H) ResourceName(suffix string) string
  • func (h *H) RotateAPIKey(id string) APIKeyHandle
  • func (h *H) Run(args ...string) (string, string, error)
  • func (h *H) RunJSON(args ...string) (map[string]any, error)
  • func (h *H) RunStdin(stdin string, args ...string) (string, string, error)
  • func (h *H) SnapshotChatName(id string)
  • func (h *H) SnapshotConnector(id int)
• type ManualRevertLog
  • func (l *ManualRevertLog) Count() int
  • func (l *ManualRevertLog) Flush(dir string) (string, error)
  • func (l *ManualRevertLog) HasEntries() bool
  • func (l *ManualRevertLog) Record(what, reason string, when time.Time)

Types

type APIKeyHandle

type APIKeyHandle struct {
	ID    string
	Token string
}

APIKeyHandle is returned from CreateAPIKey: the key id plus the one-shot plaintext token the server emits. The token is not persisted by the harness — callers that want to use it must capture it before End fires.

type ConnSpec

type ConnSpec struct {
	Host     string
	Port     int
	User     string
	Password string
	Database string
	SSL      bool
}

ConnSpec is the minimum postgres-dialect spec the e2e suite exercises. Keep fields narrow so the test data dir does not turn into a config editor.

type H

type H struct {
	Prefix string
	// contains filtered or unexported fields
}

H is the per-test live-smoke harness. Construct via Begin; the returned H holds a transport.Client plus a root *cli.Group (the verb tree) bound to a temp config. Every mutation registered with the helpers gets reverted in LIFO order by End.

func Begin

func Begin(t *testing.T) *H

Begin sets up a harness for t. Skips the test unless both ANA_E2E_ENDPOINT and ANA_E2E_TOKEN are set. Validates the token against ANA_E2E_EXPECT_ORG_ID (aborts with t.Fatalf on mismatch) and sweeps any anacli-e2e-* leftovers from prior crashed runs before returning.

func (*H) Client

func (h *H) Client() *transport.Client

Client exposes the raw transport.Client for out-of-band setup/teardown (sweep lists, parent-cascade deletes). Prefer Run for test body actions.

func (*H) CreateAPIKey

func (h *H) CreateAPIKey(suffix string) APIKeyHandle

CreateAPIKey posts CreateApiKey and defers RevokeApiKey on the latest id (the id may be rotated mid-test; Rotate swaps the deferred id too).

func (*H) CreateChat

func (h *H) CreateChat(suffix string, connectorIDs []int) string

CreateChat posts CreateChat bound to the given connector ids, returning the chat id. Cleanup cascades to any child messages, shares, etc.

func (*H) CreateConnector

func (h *H) CreateConnector(suffix string, spec ConnSpec) int

CreateConnector posts CreateConnector and defers DeleteConnector.

func (*H) CreateServiceAccount

func (h *H) CreateServiceAccount(suffix string) string

CreateServiceAccount posts CreateServiceAccount and defers DeleteServiceAccount. Server mandates either assumedRoles or inheritAllRoles=true AND a real ownerMemberId (Error 207), so we pre-fetch the caller's memberId via GetMember and inherit the caller's roles rather than plumbing role UUIDs through the suite.

func (*H) DryRun

func (h *H) DryRun() bool

DryRun reports whether ANA_E2E_DRYRUN=1 is set.

func (*H) End

func (h *H) End()

End runs every registered cleanup in LIFO order, then flushes the ledger and fails the test if any manual-revert entries were recorded.

func (*H) Endpoint

func (h *H) Endpoint() string

Endpoint returns the ANA_E2E_ENDPOINT value the harness validated at Begin. Tests that need to assert endpoint-referencing stdout (e.g. OAuth callback URLs) should read it through here rather than re-fetching the env var.

func (*H) ExpectOrgID

func (h *H) ExpectOrgID() string

ExpectOrgID returns the ANA_E2E_EXPECT_ORG_ID value the harness validated.

func (*H) MustNotTouch

func (h *H) MustNotTouch(resourceType string, id any)

MustNotTouch marks id as off-limits. Any helper that mutates this id must call h.forbiddenCheck first; the default pathway through the typed helpers does this automatically.

func (*H) RecordManualRevert

func (h *H) RecordManualRevert(what, reason string)

RecordManualRevert appends a ledger entry; End will flush and fail.

func (*H) Register

func (h *H) Register(fn func())

defer registers fn to run in LIFO order when End fires. Exported indirectly via Register so callers can deregister on successful delete.

func (*H) RegisterAPIKeyCleanupByName

func (h *H) RegisterAPIKeyCleanupByName(name string)

RegisterAPIKeyCleanupByName registers a best-effort cleanup that, at End time, lists api keys and revokes any whose name equals `name`. Tests that drive `auth keys create` through the CLI call this BEFORE the create so an orphan key can't survive if the post-create lookup or assertion errors out. No-op under ANA_E2E_DRYRUN and for empty names.

func (*H) RegisterConnectorCleanup

func (h *H) RegisterConnectorCleanup(id int)

RegisterConnectorCleanup registers a LIFO cleanup that DeleteConnectors id. Tests that create a connector via `h.Run("connector","create",…)` (rather than the raw-RPC CreateConnector helper) call this after parsing the id out of stdout so the ledger still owns the teardown. No-op under ANA_E2E_DRYRUN and for invalid ids (<= 0) so the helper can't schedule a real RPC when no mutation actually happened.

func (*H) RegisterConnectorCleanupByName

func (h *H) RegisterConnectorCleanupByName(name string)

RegisterConnectorCleanupByName registers a best-effort cleanup that, at End time, lists connectors and deletes any whose name equals `name`. Tests that drive `connector create` through the CLI register this BEFORE invoking the command so an orphan can't escape if id-extraction from stdout fails after the server has already created the row. The id-based cleanup registered after a successful parse runs first (LIFO); this helper then no-ops on the follow-up list because the row is already gone. No-op under ANA_E2E_DRYRUN and for empty names so the helper can't schedule a real RPC when no mutation actually happened.

func (*H) RegisterServiceAccountCleanupByName

func (h *H) RegisterServiceAccountCleanupByName(name string)

RegisterServiceAccountCleanupByName registers a best-effort cleanup that, at End time, lists service accounts and deletes any whose displayName equals `name`. CLI-driven create tests call this before the create so a successful create followed by a failed post-create lookup cannot orphan the SA. No-op under ANA_E2E_DRYRUN and for empty names so the helper can't schedule a real RPC when no mutation actually happened.

func (*H) ResourceName

func (h *H) ResourceName(suffix string) string

ResourceName returns a prefixed, test-owned name safe for server-side mutation. Passing the caller-specific suffix helps the sweep recognise leftovers without ambiguity.

func (*H) RotateAPIKey

func (h *H) RotateAPIKey(id string) APIKeyHandle

RotateAPIKey rotates the key most recently created by CreateAPIKey and swaps the deferred revoke to point at the successor.

func (*H) Run

func (h *H) Run(args ...string) (string, string, error)

Run invokes the ana CLI via cli.Dispatch with a verb and args. stdout and stderr are captured and returned. Global flags (--json, --profile, etc.) may appear in args; Dispatch handles them. If DryRun is set, Run records the intent and returns an empty result without calling the RPC.

func (*H) RunJSON

func (h *H) RunJSON(args ...string) (map[string]any, error)

RunJSON runs the CLI with --json prepended (after any pre-verb globals), then decodes stdout as a map. Convenience over Run for tests that assert on structured output.

func (*H) RunStdin

func (h *H) RunStdin(stdin string, args ...string) (string, string, error)

RunStdin is Run with a stdin payload, used by leaves that accept secrets via --*-stdin flags (e.g. connector create snowflake password --password-stdin). An empty stdin is equivalent to Run.

func (*H) SnapshotChatName

func (h *H) SnapshotChatName(id string)

SnapshotChatName captures the chat's current summary and defers a rename back to that value. Safe to use on chats the test did NOT create, as long as only the `summary` field is mutated.

func (*H) SnapshotConnector

func (h *H) SnapshotConnector(id int)

SnapshotConnector captures the current connector record keyed by id and registers a cleanup that restores it via UpdateConnector. Only the scalar fields the server lets you re-send (name, type, dialect block) are restored. Password fields can't be read back from the API, so mutations that change passwords MUST be scoped to a test-created connector (use CreateConnector instead) — SnapshotConnector is only safe for fields that round-trip through Get/Update unchanged.

type ManualRevertLog

type ManualRevertLog struct {
	// contains filtered or unexported fields
}

ManualRevertLog collects entries for mutations the harness could not auto-revert. End flushes them to a markdown file and fails the test.

func (*ManualRevertLog) Count

func (l *ManualRevertLog) Count() int

Count returns the number of entries.

func (*ManualRevertLog) Flush

func (l *ManualRevertLog) Flush(dir string) (string, error)

Flush writes a `e2e-manual-revert-<ts>.md` file in dir and returns its path. The repo .gitignore should exclude that pattern.

func (*ManualRevertLog) HasEntries

func (l *ManualRevertLog) HasEntries() bool

HasEntries reports whether any entries were recorded.

func (*ManualRevertLog) Record

func (l *ManualRevertLog) Record(what, reason string, when time.Time)

Record appends one entry. Safe for concurrent use.