README
¶
Tech AI Knowledge Engine
A generic, layered service substrate for building multi-language (Go + Python)
backend systems — the reusable "how we build services" library. It carries no
product-domain logic: just the
foundation, clients, repositories, services, pipelines, workflows, transport, and
batch-execution machinery that a service stack is assembled from, behind
dependency-free core interfaces.
MIT-licensed and semantically versioned (the git tag is the source of truth). The design rules — layers, swappable components, decorators, error codes — are in ARCHITECTURE.md; contribution rules are in CONTRIBUTING.md.
Layout
go.mod ONE Go module: github.com/gt-tech-ai/knowledge-engine-toolkit (packages under go/)
go/
core/ Layer 0 — pure interfaces + types (no external deps)
foundation/ Layer 1 — config, logger, metrics, tracer, resilience, cache, stream (bounded backpressured results)
clients/ Layer 2 — external-system integration (db, cache, messaging, rpc, storage, lock)
repos/ Layer 3 — data-access base (repository, cursor, decorators)
services/ Layer 4 — domain-service base
pipelines/ Layer 5 — stateless transforms
workflows/ Layer 6 — multi-step orchestration
transport/ Layer 7 — HTTP/RPC/JSON:API
execution/ batch fan-out substrate (discover → map → fan-out → aggregate)
helpers/ small shared utilities
tests/ black-box tests (unit + integration via testcontainers), fixtures, generated mocks
python/
techai_webutils/ the Python mirror of the Go substrate (published to PyPI)
Layers import downward only (core → foundation → clients → … → transport); .golangci.yml
carries depguard rules for the boundaries. Every swappable component is a Kind + Config +
factory (NewFromConfig in Go, new_<type>_from_config in Python), selected by configuration
and injected at a composition root. See ARCHITECTURE.md.
Consuming
Go:
go get github.com/gt-tech-ai/knowledge-engine-toolkit@latest
import "github.com/gt-tech-ai/knowledge-engine-toolkit/go/foundation/logger"
Python:
pip install techai-webutils # or: uv add techai-webutils
Developing
Mocks are generated by mockgen into go/tests/mocks/; regenerate them (as CI does) after changing
a mocked interface:
go install go.uber.org/mock/mockgen@v0.6.0
go generate ./go/tests/mocks/...
Then run the checks:
# Go
go vet ./go/...
golangci-lint run ./go/... # layer-boundary rules + the linter set
go test ./go/... # unit
go test -tags=integration ./go/... # integration (real deps via testcontainers)
# Python
cd python/techai_webutils
uv run ruff check . && uv run basedpyright src && uv run pytest
Integration tests use real dependencies in throwaway Docker containers (testcontainers) — never fakes.
License
MIT — see LICENSE.
Directories
¶
| Path | Synopsis |
|---|---|
|
go
|
|
|
clients/analytics
Package analytics builds the AnalyticsStore: the zero-infrastructure stub (the default) or the CQL store (Apache Cassandra locally, Amazon Keyspaces in staging, chosen by the nested CQL config).
|
Package analytics builds the AnalyticsStore: the zero-infrastructure stub (the default) or the CQL store (Apache Cassandra locally, Amazon Keyspaces in staging, chosen by the nested CQL config). |
|
clients/analytics/cassandra
Package cassandra is the Cassandra-protocol AnalyticsStore (Apache Cassandra or Amazon Keyspaces behind clients/interfaces.CQLSession).
|
Package cassandra is the Cassandra-protocol AnalyticsStore (Apache Cassandra or Amazon Keyspaces behind clients/interfaces.CQLSession). |
|
clients/analytics/stub
Package stub is the default analytics store: it accepts every write and answers every query with an empty result, so the graph boots with no Cassandra.
|
Package stub is the default analytics store: it accepts every write and answers every query with an empty result, so the graph boots with no Cassandra. |
|
clients/cache
Package cache builds a ByteCache from config: the always-miss stub (the default) or Redis.
|
Package cache builds a ByteCache from config: the always-miss stub (the default) or Redis. |
|
clients/cache/redis
Package redis provides a Redis-backed cache implementation with singleflight support.
|
Package redis provides a Redis-backed cache implementation with singleflight support. |
|
clients/cache/stub
Package stub is the zero-infrastructure ByteCache: every Get misses and every write is dropped, so a graph boots and runs uncached without Redis.
|
Package stub is the zero-infrastructure ByteCache: every Get misses and every write is dropped, so a graph boots and runs uncached without Redis. |
|
clients/client
Package client is the client tier: the one decoration every outbound client (an SDK, a datastore, an HTTP API) runs its operations through.
|
Package client is the client tier: the one decoration every outbound client (an SDK, a datastore, an HTTP API) runs its operations through. |
|
clients/connector
Package connector builds ConnectorSources for the connector family: S3 now, a future GDrive/Salesforce kind added as a kinds-table entry + sibling package (ARCHITECTURE.md#the-one-idea).
|
Package connector builds ConnectorSources for the connector family: S3 now, a future GDrive/Salesforce kind added as a kinds-table entry + sibling package (ARCHITECTURE.md#the-one-idea). |
|
clients/connector/s3
Package s3 implements the S3 ConnectorSource for the connector family: a "test connection" probe that lists a bounded page under a connector's bucket/prefix using the connector's OWN credentials, over the shared StorageClient (INJECTED — this package never imports go/clients/storage, so there is no lateral same-layer import).
|
Package s3 implements the S3 ConnectorSource for the connector family: a "test connection" probe that lists a bounded page under a connector's bucket/prefix using the connector's OWN credentials, over the shared StorageClient (INJECTED — this package never imports go/clients/storage, so there is no lateral same-layer import). |
|
clients/cql
Package cql builds a CQL (Cassandra-protocol) Session — self-managed Apache Cassandra (KindCassandra, local and dev) or Amazon Keyspaces (KindKeyspaces, staging) — behind the consumer-side Session seam, so the choice is a config change.
|
Package cql builds a CQL (Cassandra-protocol) Session — self-managed Apache Cassandra (KindCassandra, local and dev) or Amazon Keyspaces (KindKeyspaces, staging) — behind the consumer-side Session seam, so the choice is a config change. |
|
clients/cql/cassandra
Package cassandra is the self-managed Apache Cassandra kind of clients/cql (local Compose, dev): a token-aware, DC-aware host policy, optional password auth and TLS, and speculative execution on idempotent reads.
|
Package cassandra is the self-managed Apache Cassandra kind of clients/cql (local Compose, dev): a token-aware, DC-aware host policy, optional password auth and TLS, and speculative execution on idempotent reads. |
|
clients/cql/driver
Package driver adapts the gocql driver to the CQLSession contract: Dial opens a session on a cluster configuration (built by a cql kind's subpackage), and every driver error is coded at this boundary.
|
Package driver adapts the gocql driver to the CQLSession contract: Dial opens a session on a cluster configuration (built by a cql kind's subpackage), and every driver error is coded at this boundary. |
|
clients/cql/keyspaces
Package keyspaces is the Amazon Keyspaces kind of clients/cql: the regional endpoint on port 9142, mandatory TLS with host verification, SigV4 authentication (the AWS SigV4 gocql plugin, credentials from an AWS credentials provider — the default chain, IRSA in-cluster) and DisableInitialHostLookup, per AWS guidance.
|
Package keyspaces is the Amazon Keyspaces kind of clients/cql: the regional endpoint on port 9142, mandatory TLS with host verification, SigV4 authentication (the AWS SigV4 gocql plugin, credentials from an AWS credentials provider — the default chain, IRSA in-cluster) and DisableInitialHostLookup, per AWS guidance. |
|
clients/database
Package database builds the SQL connection pool from config: PostgreSQL (postgres/) today, behind interfaces.DatabasePool, so switching engines is a config change.
|
Package database builds the SQL connection pool from config: PostgreSQL (postgres/) today, behind interfaces.DatabasePool, so switching engines is a config change. |
|
clients/database/postgres
Package postgres is the PostgreSQL backend of the database client tier: a lifecycle-managed *sql.DB pool (Client) plus ORM-independent pool utilities — a boot-time reachability probe (Ping) and a Prometheus pool-stats collector.
|
Package postgres is the PostgreSQL backend of the database client tier: a lifecycle-managed *sql.DB pool (Client) plus ORM-independent pool utilities — a boot-time reachability probe (Ping) and a Prometheus pool-stats collector. |
|
clients/jobs/river
Package river is the River job-queue client: the worker Runtime (pool + work loop), the InsertClient that enqueues from outside a worker, and Migrate for River's schema.
|
Package river is the River job-queue client: the worker Runtime (pool + work loop), the InsertClient that enqueues from outside a worker, and Migrate for River's schema. |
|
clients/lock
Package lock builds a config-selected core/interfaces.Locker — a DistributedLock (keyed, token-fenced, best-effort) plus Hold, the managed acquire-with-watchdog helper a single-active-operation guard depends on.
|
Package lock builds a config-selected core/interfaces.Locker — a DistributedLock (keyed, token-fenced, best-effort) plus Hold, the managed acquire-with-watchdog helper a single-active-operation guard depends on. |
|
clients/lock/local
Package local provides an in-process DistributedLock backend for single-pod deployments (dev / replica=1), where cross-process coordination is unnecessary.
|
Package local provides an in-process DistributedLock backend for single-pod deployments (dev / replica=1), where cross-process coordination is unnecessary. |
|
clients/lock/redis
Package redis provides a cross-pod DistributedLock backend over Redis, for multi-replica deployments (staging/prod).
|
Package redis provides a cross-pod DistributedLock backend over Redis, for multi-replica deployments (staging/prod). |
|
clients/messaging
Package messaging builds the message publisher and consumer from config: the in-process memory broker (the default), SQS, or Redis Pub/Sub.
|
Package messaging builds the message publisher and consumer from config: the in-process memory broker (the default), SQS, or Redis Pub/Sub. |
|
clients/messaging/memory
Package memory provides an in-process messaging backend for no-infra builds and tests.
|
Package memory provides an in-process messaging backend for no-infra builds and tests. |
|
clients/messaging/redis
Package redis provides a Redis Pub/Sub messaging integration (publisher + subscriber) implementing the core MessagePublisher / MessageConsumer contracts over PUBLISH / SUBSCRIBE / PSUBSCRIBE.
|
Package redis provides a Redis Pub/Sub messaging integration (publisher + subscriber) implementing the core MessagePublisher / MessageConsumer contracts over PUBLISH / SUBSCRIBE / PSUBSCRIBE. |
|
clients/messaging/sqs
Package sqs provides an AWS-SDK-backed SQS messaging integration (publisher + subscriber) usable against real AWS SQS and against ElasticMQ for local dev.
|
Package sqs provides an AWS-SDK-backed SQS messaging integration (publisher + subscriber) usable against real AWS SQS and against ElasticMQ for local dev. |
|
clients/outbox
Package outbox builds the OutboxSink a transactional-outbox relay delivers through: the zero-infrastructure stub (the default), an SQS queue, or an S3 bucket, selected by Kind.
|
Package outbox builds the OutboxSink a transactional-outbox relay delivers through: the zero-infrastructure stub (the default), an SQS queue, or an S3 bucket, selected by Kind. |
|
clients/outbox/s3
Package s3 is the S3 OutboxSink: it writes each record as one object, keyed by a template, with a SHA-256 integrity checksum (Object Lock buckets require an integrity checksum and accept it in place of Content-MD5) and SSE-KMS when the record names a key.
|
Package s3 is the S3 OutboxSink: it writes each record as one object, keyed by a template, with a SHA-256 integrity checksum (Object Lock buckets require an integrity checksum and accept it in place of Content-MD5) and SSE-KMS when the record names a key. |
|
clients/outbox/sqs
Package sqs is the SQS OutboxSink: it routes each record to a queue, sends each queue's records with SendMessageBatch (at most 10 per request) and maps each entry's result back to its record.
|
Package sqs is the SQS OutboxSink: it routes each record to a queue, sends each queue's records with SendMessageBatch (at most 10 per request) and maps each entry's result back to its record. |
|
clients/outbox/stub
Package stub is the zero-infrastructure OutboxSink: it accepts every record and delivers nothing, so the graph boots with no queue or bucket.
|
Package stub is the zero-infrastructure OutboxSink: it accepts every record and delivers nothing, so the graph boots with no queue or bucket. |
|
clients/prometheus
Package prometheus builds the MetricsQuerier from config: the zero-infrastructure stub (the default) or the HTTP query API, its requests run through the client tier.
|
Package prometheus builds the MetricsQuerier from config: the zero-infrastructure stub (the default) or the HTTP query API, its requests run through the client tier. |
|
clients/prometheus/http
Package http is the HTTP backend of the Prometheus querier: instant queries reduced to one sample, and range queries reduced to one series, over a Prometheus server's HTTP query API.
|
Package http is the HTTP backend of the Prometheus querier: instant queries reduced to one sample, and range queries reduced to one series, over a Prometheus server's HTTP query API. |
|
clients/prometheus/stub
Package stub is the zero-infrastructure metrics querier: every instant query is 0 and every range query an empty series, with no network I/O.
|
Package stub is the zero-infrastructure metrics querier: every instant query is 0 and every range query an empty series, with no network I/O. |
|
clients/replaybuffer
Package replaybuffer builds a ReplayBuffer from config — the in-process memory buffer (single pod; the default) or Redis (cross-pod) — and runs it through the client tier (Decorate).
|
Package replaybuffer builds a ReplayBuffer from config — the in-process memory buffer (single pod; the default) or Redis (cross-pod) — and runs it through the client tier (Decorate). |
|
clients/replaybuffer/memory
Package memory provides an in-process ReplayBuffer backend for single-pod deployments (dev / replica=1), where a reconnect lands on the same pod so cross-pod sharing is unnecessary.
|
Package memory provides an in-process ReplayBuffer backend for single-pod deployments (dev / replica=1), where a reconnect lands on the same pod so cross-pod sharing is unnecessary. |
|
clients/replaybuffer/redis
Package redis provides a cross-pod ReplayBuffer backend over Redis, for multi-replica deployments (staging/prod) where a reconnect may land on a different pod than the one that buffered the tail.
|
Package redis provides a cross-pod ReplayBuffer backend over Redis, for multi-replica deployments (staging/prod) where a reconnect may land on a different pod than the one that buffered the tail. |
|
clients/rpc
Package rpc builds the RPC client from config: gRPC (grpc/) today, behind RPCClient, so switching RPC frameworks is a config change.
|
Package rpc builds the RPC client from config: gRPC (grpc/) today, behind RPCClient, so switching RPC frameworks is a config change. |
|
clients/rpc/connect/interceptors
Package interceptors runs a generated Connect client's calls through the client tier (client.Tier): ClientOptions installs it as a Connect client interceptor.
|
Package interceptors runs a generated Connect client's calls through the client tier (client.Tier): ClientOptions installs it as a Connect client interceptor. |
|
clients/rpc/grpc
Package grpc dials gRPC client connections: NewClient builds one, and Client manages it as a lifecycle-managed, health-checkable client (interfaces.Client).
|
Package grpc dials gRPC client connections: NewClient builds one, and Client manages it as a lifecycle-managed, health-checkable client (interfaces.Client). |
|
clients/rpc/grpc/interceptors
Package interceptors runs a gRPC client connection's calls through the client tier (client.Tier): ClientOptions installs it as dial options, carrying the trace context and the service token on every call.
|
Package interceptors runs a gRPC client connection's calls through the client tier (client.Tier): ClientOptions installs it as dial options, carrying the trace context and the service token on every call. |
|
clients/rpc/wire
Package wire is the codec between RPC status codes (Connect and gRPC share one code space) and the domain's apperr codes.
|
Package wire is the codec between RPC status codes (Connect and gRPC share one code space) and the domain's apperr codes. |
|
clients/secrets
Package secrets is the credential-source client tier: it selects a secret backend — an environment variable (env/), a git-ignored local file (file/), or AWS Secrets Manager (awssm/) — by Kind (env when empty) and returns the interfaces.Source contract, so switching where a credential is read is a configuration change, not a caller edit.
|
Package secrets is the credential-source client tier: it selects a secret backend — an environment variable (env/), a git-ignored local file (file/), or AWS Secrets Manager (awssm/) — by Kind (env when empty) and returns the interfaces.Source contract, so switching where a credential is read is a configuration change, not a caller edit. |
|
clients/secrets/awssm
Package awssm implements the AWS Secrets Manager credential Source: it reads a credential from a Secrets Manager secret (a JSON object) named by a caller-supplied Ref→location mapping, extracting the configured JSON field.
|
Package awssm implements the AWS Secrets Manager credential Source: it reads a credential from a Secrets Manager secret (a JSON object) named by a caller-supplied Ref→location mapping, extracting the configured JSON field. |
|
clients/secrets/env
Package env implements the environment-variable credential Source: it resolves a Ref to a configured environment variable name and reads its value via os.Getenv.
|
Package env implements the environment-variable credential Source: it resolves a Ref to a configured environment variable name and reads its value via os.Getenv. |
|
clients/secrets/file
Package file implements the file-backed credential Source: it reads and writes credentials in a git-ignored local file (a dotenv-style KEY=value store) named by a caller-supplied Ref→location mapping.
|
Package file implements the file-backed credential Source: it reads and writes credentials in a git-ignored local file (a dotenv-style KEY=value store) named by a caller-supplied Ref→location mapping. |
|
clients/storage
Package storage builds the StorageClient from config: the in-memory backend (the default) or an S3-compatible one (AWS S3 or MinIO), run through the client tier (Decorate).
|
Package storage builds the StorageClient from config: the in-memory backend (the default) or an S3-compatible one (AWS S3 or MinIO), run through the client tier (Decorate). |
|
clients/storage/memory
Package memory provides an in-memory StorageClient backend for no-infra builds and tests.
|
Package memory provides an in-memory StorageClient backend for no-infra builds and tests. |
|
clients/storage/s3
Package s3 implements the StorageClient against S3-compatible object stores (AWS S3 and MinIO), selected by configuration.
|
Package s3 implements the StorageClient against S3-compatible object stores (AWS S3 and MinIO), selected by configuration. |
|
core/errors
Package errors provides application error types and codes, and the request-scoped error sink that keeps a domain error's real cause behind its sanitized transport error.
|
Package errors provides application error types and codes, and the request-scoped error sink that keeps a domain error's real cause behind its sanitized transport error. |
|
core/events
Package events is the generic event contract: the Event interface, the metadata every event carries, and a Registry a consumer fills with its own event catalog to parse payloads.
|
Package events is the generic event contract: the Event interface, the metadata every event carries, and a Registry a consumer fills with its own event catalog to parse payloads. |
|
core/interfaces
Package interfaces defines the core contracts for the Tech AI Knowledge Engine.
|
Package interfaces defines the core contracts for the Tech AI Knowledge Engine. |
|
core/principal
Package principal carries the request's resolved principal — the consumer's own caller type (its authorization model) — in a context.Context.
|
Package principal carries the request's resolved principal — the consumer's own caller type (its authorization model) — in a context.Context. |
|
core/tenant
Package tenant carries the request-scoped tenant (organization) identity in a context.Context.
|
Package tenant carries the request-scoped tenant (organization) identity in a context.Context. |
|
core/types
Package types provides shared domain types used across the platform.
|
Package types provides shared domain types used across the platform. |
|
execution/engine
Package engine provides execution infrastructure: bounded concurrent fan-out, observer dispatch, job-group execution, and simple function steps.
|
Package engine provides execution infrastructure: bounded concurrent fan-out, observer dispatch, job-group execution, and simple function steps. |
|
execution/gate
Package gate composes AnyJobs into a Gate of ordered phases, run with stop-on-failure and error aggregation.
|
Package gate composes AnyJobs into a Gate of ordered phases, run with stop-on-failure and error aggregation. |
|
execution/job
Package job builds interfaces.AnyJob values: Base decorates every job it builds with the job tier (recovery, logging, tracing, metrics, timeout) once, so a job is a function plus its name and group, never a hand-written AnyJob type.
|
Package job builds interfaces.AnyJob values: Base decorates every job it builds with the job tier (recovery, logging, tracing, metrics, timeout) once, so a job is a function plus its name and group, never a hand-written AnyJob type. |
|
foundation/cache
Package cache provides caching utilities for the foundation layer.
|
Package cache provides caching utilities for the foundation layer. |
|
foundation/cache/decorators
Package decorators provides cache decorators for cross-cutting concerns.
|
Package decorators provides cache decorators for cross-cutting concerns. |
|
foundation/chartspec
Package chartspec is the pure visual-algebra compiler of the analytics engine.
|
Package chartspec is the pure visual-algebra compiler of the analytics engine. |
|
foundation/config
Package config provides a factory for ConfigLoader implementations with multiple backends.
|
Package config provides a factory for ConfigLoader implementations with multiple backends. |
|
foundation/config/schema/decoration
Package decoration is the config surface for every decorated component — controllers, clients, workflows, pipelines, services, repositories, stores and jobs.
|
Package decoration is the config surface for every decorated component — controllers, clients, workflows, pipelines, services, repositories, stores and jobs. |
|
foundation/config/schema/infra
Package infra provides typed configuration structures for Tech AI Knowledge Engine services.
|
Package infra provides typed configuration structures for Tech AI Knowledge Engine services. |
|
foundation/config/schema/listquery
Package listquery is the config surface for the list-query value-suggestion facet: the row threshold above which a facet-eligible field is served from the maintained distinct_values table instead of a live DISTINCT, and the interval at which the reconcile job recomputes refcounts.
|
Package listquery is the config surface for the list-query value-suggestion facet: the row threshold above which a facet-eligible field is served from the maintained distinct_values table instead of a live DISTINCT, and the interval at which the reconcile job recomputes refcounts. |
|
foundation/config/schema/pipelines
Package pipelines is the config surface for the pipeline (orchestration) layer: the multipart-upload sizing (a pipeline's timeout is a decoration layer, decoration.pipelines).
|
Package pipelines is the config surface for the pipeline (orchestration) layer: the multipart-upload sizing (a pipeline's timeout is a decoration layer, decoration.pipelines). |
|
foundation/config/schema/repos
Package repos is the config surface for the repository (data-access) layer: the entity cache's TTL, version and key prefix.
|
Package repos is the config surface for the repository (data-access) layer: the entity cache's TTL, version and key prefix. |
|
foundation/config/schema/resilience
Package resilience is the tunable config surface for the foundation resilience decorators (retry, breaker, rate-limit, bulkhead, timeout, budget) — the per-environment knobs threaded through the repo/client/interceptor providers Each struct mirrors its primitive's tunable fields with defaults identical to the primitive's DefaultConfig(), so adopting the config surface is behavior-preserving; the providers convert these into the primitive's options.
|
Package resilience is the tunable config surface for the foundation resilience decorators (retry, breaker, rate-limit, bulkhead, timeout, budget) — the per-environment knobs threaded through the repo/client/interceptor providers Each struct mirrors its primitive's tunable fields with defaults identical to the primitive's DefaultConfig(), so adopting the config surface is behavior-preserving; the providers convert these into the primitive's options. |
|
foundation/config/schema/stores
Package stores is the config surface for the data (stores) layer: the ORM the stores are built on (Kind), the page-size cap, the per-resource default page sizes and the list-query caches, so a deployer can tune pagination without a rebuild.
|
Package stores is the config surface for the data (stores) layer: the ORM the stores are built on (Kind), the page-size cap, the per-resource default page sizes and the list-query caches, so a deployer can tune pagination without a rebuild. |
|
foundation/config/schema/transport
Package transport is the config surface for the edge (transport) layer: WebSocket, gRPC, CORS and HTTP tuning.
|
Package transport is the config surface for the edge (transport) layer: WebSocket, gRPC, CORS and HTTP tuning. |
|
foundation/config/schema/workers
Package workers is the shared config surface for background workers: the common tunables (name, telemetry, relay and sweep cadences, batching) plus the sweep-timing cross-field invariant, so every worker inherits the typed-field + Validate shape.
|
Package workers is the shared config surface for background workers: the common tunables (name, telemetry, relay and sweep cadences, batching) plus the sweep-timing cross-field invariant, so every worker inherits the typed-field + Validate shape. |
|
foundation/config/viper
Package viper provides a Viper-backed ConfigLoader implementation.
|
Package viper provides a Viper-backed ConfigLoader implementation. |
|
foundation/convert
Package convert maps values across an API boundary: EnumMapper pairs a proto enum with a domain enum from one table, ParseUUID parses a request id as invalid input when malformed, and Int32OrPanic / CountToInt32 narrow counts to a proto int32.
|
Package convert maps values across an API boundary: EnumMapper pairs a proto enum with a domain enum from one table, ParseUUID parses a request id as invalid input when malformed, and Int32OrPanic / CountToInt32 narrow counts to a proto int32. |
|
foundation/crypto
Package crypto provides authenticated symmetric encryption for data at rest — e.g.
|
Package crypto provides authenticated symmetric encryption for data at rest — e.g. |
|
foundation/decorate
Package decorate is the type-agnostic operation-decoration mechanism: a common OpMiddleware interface, a Chain composer, and a generic Exec[R] runner that wraps ANY named operation with the cross-cutting decorator stack, regardless of the operation's return type.
|
Package decorate is the type-agnostic operation-decoration mechanism: a common OpMiddleware interface, a Chain composer, and a generic Exec[R] runner that wraps ANY named operation with the cross-cutting decorator stack, regardless of the operation's return type. |
|
foundation/dtovalidate
Package dtovalidate is the leaf rule library the generated DTO Validate() methods compose Each rule is a small, hand-written check that REUSES the standard library (google/uuid, net/url, path/filepath, net/mail) rather than a hand-rolled regex, and returns a core/errors CodeInvalidInput AppError on violation so the failure classifies to 400 / InvalidArgument at the boundary.
|
Package dtovalidate is the leaf rule library the generated DTO Validate() methods compose Each rule is a small, hand-written check that REUSES the standard library (google/uuid, net/url, path/filepath, net/mail) rather than a hand-rolled regex, and returns a core/errors CodeInvalidInput AppError on violation so the failure classifies to 400 / InvalidArgument at the boundary. |
|
foundation/fsys
Package fsys provides a real-OS filesystem implementation of the FS interface defined in core/interfaces.
|
Package fsys provides a real-OS filesystem implementation of the FS interface defined in core/interfaces. |
|
foundation/git
Package git provides reusable git operations for the CLI and tooling layers.
|
Package git provides reusable git operations for the CLI and tooling layers. |
|
foundation/jsonapi
Package jsonapi is the [JSON:API](https://jsonapi.org/format/) codec: it rewrites a proto-JSON REST response into the {data, meta, links, errors} envelope (TransformSingleResource, TransformCollection, TransformMeta, TransformError).
|
Package jsonapi is the [JSON:API](https://jsonapi.org/format/) codec: it rewrites a proto-JSON REST response into the {data, meta, links, errors} envelope (TransformSingleResource, TransformCollection, TransformMeta, TransformError). |
|
foundation/lifecycle
Package lifecycle provides a Manager that starts a set of lifecycle-managed clients in registration order and stops them in reverse for graceful shutdown, so a composition root wires its clients once and coordinates their startup and teardown centrally instead of by hand.
|
Package lifecycle provides a Manager that starts a set of lifecycle-managed clients in registration order and stops them in reverse for graceful shutdown, so a composition root wires its clients once and coordinates their startup and teardown centrally instead of by hand. |
|
foundation/listquery
Package listquery is the pure, ORM-free half of the list-query SDK a per-resource filter allow-list (Map) + sort allow-list (Set), a JSON filter-DSL parser that compiles a client filter string into the core/types Filter contract (gated by the allow-list so only allow-listed fields with type-checked values reach a query), and the shared Compile/CompileOrder fold that walks that IR through a core/interfaces.QueryBackend algebra.
|
Package listquery is the pure, ORM-free half of the list-query SDK a per-resource filter allow-list (Map) + sort allow-list (Set), a JSON filter-DSL parser that compiles a client filter string into the core/types Filter contract (gated by the allow-list so only allow-listed fields with type-checked values reach a query), and the shared Compile/CompileOrder fold that walks that IR through a core/interfaces.QueryBackend algebra. |
|
foundation/logger
Package logger provides structured logging with multiple backend implementations.
|
Package logger provides structured logging with multiple backend implementations. |
|
foundation/logger/logctx
Package logctx provides shared context utilities for logger sub-packages.
|
Package logctx provides shared context utilities for logger sub-packages. |
|
foundation/logger/stdlib
Package stdlib provides a pure stdlib slog logger implementation.
|
Package stdlib provides a pure stdlib slog logger implementation. |
|
foundation/logger/zap
Package zap provides a zap-backed logger implementation with PII redaction.
|
Package zap provides a zap-backed logger implementation with PII redaction. |
|
foundation/metrics
Package metrics provides a builder for the Prometheus metrics implementation.
|
Package metrics provides a builder for the Prometheus metrics implementation. |
|
foundation/metrics/noop
Package noop provides a no-op metrics implementation that silently discards all metric operations.
|
Package noop provides a no-op metrics implementation that silently discards all metric operations. |
|
foundation/metrics/prom
Package prom provides a Prometheus-backed metrics implementation.
|
Package prom provides a Prometheus-backed metrics implementation. |
|
foundation/middleware
Package middleware provides stdlib net/http middleware for composing request processing pipelines without external web frameworks.
|
Package middleware provides stdlib net/http middleware for composing request processing pipelines without external web frameworks. |
|
foundation/options
Package options provides a generic functional options pattern for configuring types.
|
Package options provides a generic functional options pattern for configuring types. |
|
foundation/resilience
Package resilience provides fault-tolerance primitives as standalone subpackages.
|
Package resilience provides fault-tolerance primitives as standalone subpackages. |
|
foundation/resilience/adaptivethrottle
Package adaptivethrottle provides a client-side adaptive throttler (Google SRE "Handling Overload"): it sheds a growing fraction of outbound requests locally when a backend's accept-rate drops, protecting the backend from a client-side retry storm.
|
Package adaptivethrottle provides a client-side adaptive throttler (Google SRE "Handling Overload"): it sheds a growing fraction of outbound requests locally when a backend's accept-rate drops, protecting the backend from a client-side retry storm. |
|
foundation/resilience/budget
Package budget tracks retry consumption across a call chain via context propagation.
|
Package budget tracks retry consumption across a call chain via context propagation. |
|
foundation/resilience/bulkhead
Package bulkhead provides a builder for Bulkhead implementations with multiple backends.
|
Package bulkhead provides a builder for Bulkhead implementations with multiple backends. |
|
foundation/resilience/bulkhead/adaptive
Package adaptive provides a self-tuning (AIMD) concurrency-limiting Bulkhead.
|
Package adaptive provides a self-tuning (AIMD) concurrency-limiting Bulkhead. |
|
foundation/resilience/bulkhead/channel
Package channel provides a channel-based semaphore bulkhead implementation.
|
Package channel provides a channel-based semaphore bulkhead implementation. |
|
foundation/resilience/bulkhead/keyed
Package keyed is the in-process KeyedLimiter: a per-key counting semaphore that refuses, rather than waits, once a key holds its cap.
|
Package keyed is the in-process KeyedLimiter: a per-key counting semaphore that refuses, rather than waits, once a key holds its cap. |
|
foundation/resilience/circuitbreaker
Package circuitbreaker provides a builder for CircuitBreaker implementations with multiple backends.
|
Package circuitbreaker provides a builder for CircuitBreaker implementations with multiple backends. |
|
foundation/resilience/circuitbreaker/gobreaker
Package gobreaker provides a circuit breaker backed by sony/gobreaker/v2.
|
Package gobreaker provides a circuit breaker backed by sony/gobreaker/v2. |
|
foundation/resilience/deadletter
Package deadletter provides the dead-letter-queue facade — the Go mirror of techai_webutils.foundation.resilience.dlq.
|
Package deadletter provides the dead-letter-queue facade — the Go mirror of techai_webutils.foundation.resilience.dlq. |
|
foundation/resilience/dedup
Package dedup provides message deduplication for exactly-once processing under at-least-once delivery.
|
Package dedup provides message deduplication for exactly-once processing under at-least-once delivery. |
|
foundation/resilience/hedge
Package hedge provides a builder for Hedger implementations that reduce tail latency by racing a backup attempt.
|
Package hedge provides a builder for Hedger implementations that reduce tail latency by racing a backup attempt. |
|
foundation/resilience/leader
Package leader provides leader election for single-instance job execution.
|
Package leader provides leader election for single-instance job execution. |
|
foundation/resilience/pausegate
Package pausegate stops a consumer pulling work while its downstream is unavailable.
|
Package pausegate stops a consumer pulling work while its downstream is unavailable. |
|
foundation/resilience/ratelimiter
Package ratelimiter provides a builder for RateLimiter implementations with multiple backends.
|
Package ratelimiter provides a builder for RateLimiter implementations with multiple backends. |
|
foundation/resilience/ratelimiter/token
Package token provides a token-bucket rate limiter backed by golang.org/x/time/rate.
|
Package token provides a token-bucket rate limiter backed by golang.org/x/time/rate. |
|
foundation/resilience/reswire
Package reswire builds the resilience primitives that are not decoration layers — the tail-latency hedger and the adaptive throttle — from a ConfigLoader and the schema/resilience config surface, starting from the schema defaults so an absent overlay is behavior-preserving.
|
Package reswire builds the resilience primitives that are not decoration layers — the tail-latency hedger and the adaptive throttle — from a ConfigLoader and the schema/resilience config surface, starting from the schema defaults so an absent overlay is behavior-preserving. |
|
foundation/resilience/retry
Package retry provides a builder for Retrier implementations with multiple backends.
|
Package retry provides a builder for Retrier implementations with multiple backends. |
|
foundation/resilience/retry/exponential
Package exponential provides an exponential backoff retrier using cenkalti/backoff/v5.
|
Package exponential provides an exponential backoff retrier using cenkalti/backoff/v5. |
|
foundation/stream
Package stream is the engine's substrate for large, pull-based results over interfaces.Stream.
|
Package stream is the engine's substrate for large, pull-based results over interfaces.Stream. |
|
foundation/system
Package system provides CommandRunner implementations for executing external commands.
|
Package system provides CommandRunner implementations for executing external commands. |
|
foundation/tracer
Package tracer provides distributed tracing with swappable backends.
|
Package tracer provides distributed tracing with swappable backends. |
|
foundation/tracer/nooptracer
Package nooptracer provides a no-op tracer that satisfies interfaces.Tracer.
|
Package nooptracer provides a no-op tracer that satisfies interfaces.Tracer. |
|
foundation/tracer/oteltracer
Package oteltracer provides an OpenTelemetry-backed tracer implementation.
|
Package oteltracer provides an OpenTelemetry-backed tracer implementation. |
|
foundation/validator
Package validator provides a small, generic business-rule validator for the service layer — the below-transport-boundary complement to the proto/request validation performed at the transport edge (connect ValidateInterceptor).
|
Package validator provides a small, generic business-rule validator for the service layer — the below-transport-boundary complement to the proto/request validation performed at the transport edge (connect ValidateInterceptor). |
|
pipelines/pipeline
Package pipeline is the pipeline tier: a Base builds a consumer's decorated pipeline stages from functions, and stages compose with Stage.Then into a staged pipeline or run side by side with Base.FanOut.
|
Package pipeline is the pipeline tier: a Base builds a consumer's decorated pipeline stages from functions, and stages compose with Stage.Then into a staged pipeline or run side by side with Base.FanOut. |
|
repos/repository
Package repository is the repository tier: a Base is the embeddable core of a consumer's repository, running its custom queries through the tier's decoration (Tier) with the promoted Exec, and building the decorated CRUD repositories the tier offers over a store.
|
Package repository is the repository tier: a Base is the embeddable core of a consumer's repository, running its custom queries through the tier's decoration (Tier) with the promoted Exec, and building the decorated CRUD repositories the tier offers over a store. |
|
services/service
Package service is the service tier: a Base is the embeddable core of a consumer's service, running its custom operations through the tier's decoration (Tier) with the promoted Exec, and building the decorated CRUD services and single-operation services the tier offers.
|
Package service is the service tier: a Base is the embeddable core of a consumer's service, running its custom operations through the tier's decoration (Tier) with the promoted Exec, and building the decorated CRUD services and single-operation services the tier offers. |
|
stores/ent
Package ent is the Ent kind of the store tier (store.KindEnt): the Ent/SQL backend of the list-query compiler — it interprets the list-query IR into Ent dialect/sql predicates and order options, the func(*sql.Selector) shape every Ent query accepts via query.Where(predicate.T(pred)) and query.Order(opt) — plus Apply and the FoldFilters/FoldOrder glue an Ent store's list adapters share.
|
Package ent is the Ent kind of the store tier (store.KindEnt): the Ent/SQL backend of the list-query compiler — it interprets the list-query IR into Ent dialect/sql predicates and order options, the func(*sql.Selector) shape every Ent query accepts via query.Where(predicate.T(pred)) and query.Order(opt) — plus Apply and the FoldFilters/FoldOrder glue an Ent store's list adapters share. |
|
stores/errors
Package errors maps a store's database errors to domain errors: each mapper translates driver-specific errors (constraint violations, not-found, schema errors) into a coded core/errors.AppError.
|
Package errors maps a store's database errors to domain errors: each mapper translates driver-specific errors (constraint violations, not-found, schema errors) into a coded core/errors.AppError. |
|
stores/errors/postgres
Package postgres provides PostgreSQL-specific DB error mapping.
|
Package postgres provides PostgreSQL-specific DB error mapping. |
|
stores/gorm
Package gorm is the GORM kind of the store tier (store.KindGORM): a lifecycle-managed opener for the *gorm.DB a GORM store runs its statements on, its SQL dialect selected by Kind (lazily, in Start), so a consumer can put its stores on GORM, or switch another ORM it owns for this one, as a composition-root wiring change rather than a rewrite.
|
Package gorm is the GORM kind of the store tier (store.KindGORM): a lifecycle-managed opener for the *gorm.DB a GORM store runs its statements on, its SQL dialect selected by Kind (lazily, in Start), so a consumer can put its stores on GORM, or switch another ORM it owns for this one, as a composition-root wiring change rather than a rewrite. |
|
stores/outbox
Package outbox builds the durable side of a transactional outbox from config: the store the relay (workflows/outbox) leases rows from and finalizes them in.
|
Package outbox builds the durable side of a transactional outbox from config: the store the relay (workflows/outbox) leases rows from and finalizes them in. |
|
stores/outbox/postgres
Package postgres is the PostgreSQL kind of the outbox store (stores/outbox): the relay's claim, lease and finalize operations over a product's own outbox table, named by configuration.
|
Package postgres is the PostgreSQL kind of the outbox store (stores/outbox): the relay's claim, lease and finalize operations over a product's own outbox table, named by configuration. |
|
stores/outbox/stub
Package stub is the stub kind of the outbox store (stores/outbox): an outbox with no rows, so a relay graph boots and runs with no database.
|
Package stub is the stub kind of the outbox store (stores/outbox): an outbox with no rows, so a relay graph boots and runs with no database. |
|
stores/sqltx
Package sqltx is the database/sql transaction manager: the interfaces.TransactionManager a workflow opens its transactions through when its stores run raw SQL on a *sql.DB (the postgres outbox store).
|
Package sqltx is the database/sql transaction manager: the interfaces.TransactionManager a workflow opens its transactions through when its stores run raw SQL on a *sql.DB (the postgres outbox store). |
|
stores/store
Package store is the store tier: the persistence layer a repository is built on.
|
Package store is the store tier: the persistence layer a repository is built on. |
|
tests/fixtures
Package fixtures provides thin, hand-written factories over the GENERATED gomock mocks in the sibling mocks package.
|
Package fixtures provides thin, hand-written factories over the GENERATED gomock mocks in the sibling mocks package. |
|
tests/fixtures/dbtest/cassandra
Package cassandra provides a testcontainers-based Apache Cassandra 5 instance for integration tests.
|
Package cassandra provides a testcontainers-based Apache Cassandra 5 instance for integration tests. |
|
tests/fixtures/dbtest/elasticmq
Package elasticmq provides a testcontainers-based ElasticMQ (SQS-compatible) instance for integration tests.
|
Package elasticmq provides a testcontainers-based ElasticMQ (SQS-compatible) instance for integration tests. |
|
tests/fixtures/dbtest/minio
Package minio provides a testcontainers-based MinIO instance for integration tests.
|
Package minio provides a testcontainers-based MinIO instance for integration tests. |
|
tests/fixtures/dbtest/postgres
Package postgres provides a generic testcontainers-based PostgreSQL database for integration tests.
|
Package postgres provides a generic testcontainers-based PostgreSQL database for integration tests. |
|
tests/fixtures/dbtest/redis
Package redis provides a testcontainers-based Redis instance for integration tests.
|
Package redis provides a testcontainers-based Redis instance for integration tests. |
|
tests/fixtures/decorgen
Package decorgen is the fixture tools/decorgen generates from: Store has every method shape a decorator handles, and decorated_gen.go is its committed output.
|
Package decorgen is the fixture tools/decorgen generates from: Store has every method shape a decorator handles, and decorated_gen.go is its committed output. |
|
tests/fixtures/e2ekit
Package e2ekit is the gen-free shared test-kit for endpoint E2E + telemetry suites.
|
Package e2ekit is the gen-free shared test-kit for endpoint E2E + telemetry suites. |
|
tests/fixtures/outboxtest
Package outboxtest is the reusable OutboxStore conformance suite (Run) and the routing-sink suite (RunRouting) a consumer runs against its own store and sink over real infrastructure; the engine runs them against the postgres outbox-store kind.
|
Package outboxtest is the reusable OutboxStore conformance suite (Run) and the routing-sink suite (RunRouting) a consumer runs against its own store and sink over real infrastructure; the engine runs them against the postgres outbox-store kind. |
|
tests/fixtures/store
Package store (testutil) provides a interfaces.Store implementation that supports call counting and error injection for use in tests.
|
Package store (testutil) provides a interfaces.Store implementation that supports call counting and error injection for use in tests. |
|
tests/fixtures/suite
Package suite provides test suite constructors for unit and integration tests.
|
Package suite provides test suite constructors for unit and integration tests. |
|
tests/fixtures/suite/base
Package base provides the base test suite with shared utilities.
|
Package base provides the base test suite with shared utilities. |
|
tests/integration
Package integration holds the Go integration tests: each file exercises a clients-tier backend against the real service it wraps, started in a throwaway testcontainers container by the fixtures under go/tests/fixtures/dbtest — Redis (cache, lock, messaging, replay buffer), MinIO (S3 storage and the S3 connector), ElasticMQ (SQS messaging), and PostgreSQL (River's runtime and migrations).
|
Package integration holds the Go integration tests: each file exercises a clients-tier backend against the real service it wraps, started in a throwaway testcontainers container by the fixtures under go/tests/fixtures/dbtest — Redis (cache, lock, messaging, replay buffer), MinIO (S3 storage and the S3 connector), ElasticMQ (SQS messaging), and PostgreSQL (River's runtime and migrations). |
|
tests/mocks
Package mocks is a generated GoMock package.
|
Package mocks is a generated GoMock package. |
|
tools/decorgen
Package decorgen generates decorators: for each named interface, a Decorated<Name> type whose every operation — a method taking a context.Context first and returning an error last — runs through a decorate.Subject (the Exec every tier's Base embeds), named by the method, and whose other methods pass straight through.
|
Package decorgen generates decorators: for each named interface, a Decorated<Name> type whose every operation — a method taking a context.Context first and returning an error last — runs through a decorate.Subject (the Exec every tier's Base embeds), named by the method, and whose other methods pass straight through. |
|
tools/decorgen/cmd/decorgen
command
Command decorgen writes the decorators decorgen.Generate builds, for a go:generate directive:
|
Command decorgen writes the decorators decorgen.Generate builds, for a go:generate directive: |
|
transport
Package transport is the controller tier: the edge every inbound request enters by — a REST or RPC handler.
|
Package transport is the controller tier: the edge every inbound request enters by — a REST or RPC handler. |
|
transport/connect
Package connect provides a Connect/gRPC server factory that mounts proto-generated handlers on a standard net/http.ServeMux.
|
Package connect provides a Connect/gRPC server factory that mounts proto-generated handlers on a standard net/http.ServeMux. |
|
transport/connect/interceptors
Package interceptors provides Connect RPC interceptors for cross-cutting concerns.
|
Package interceptors provides Connect RPC interceptors for cross-cutting concerns. |
|
transport/docs
Package docs provides reusable HTTP handlers for serving OpenAPI specs and Redoc UI documentation pages.
|
Package docs provides reusable HTTP handlers for serving OpenAPI specs and Redoc UI documentation pages. |
|
transport/grpc
Package grpc builds a configured gRPC server (NewServer) and registers the gRPC health service on it; interceptors/ runs its RPCs through the controller tier.
|
Package grpc builds a configured gRPC server (NewServer) and registers the gRPC health service on it; interceptors/ runs its RPCs through the controller tier. |
|
transport/grpc/interceptors
Package interceptors runs a gRPC server's RPCs through the controller tier (transport.Tier): ServerOptions lays the tier out as unary and stream server interceptors, with gRPC-native tracing, metrics and logging standing in for the tier's own.
|
Package interceptors runs a gRPC server's RPCs through the controller tier (transport.Tier): ServerOptions lays the tier out as unary and stream server interceptors, with gRPC-native tracing, metrics and logging standing in for the tier's own. |
|
transport/jsonapi
Package jsonapi provides HTTP middleware that transforms Vanguard REST responses into [JSON:API](https://jsonapi.org/format/) envelopes with the foundation/jsonapi codec.
|
Package jsonapi provides HTTP middleware that transforms Vanguard REST responses into [JSON:API](https://jsonapi.org/format/) envelopes with the foundation/jsonapi codec. |
|
transport/rest
Package rest provides HTTP response helpers for REST API handlers.
|
Package rest provides HTTP response helpers for REST API handlers. |
|
transport/rpc
Package rpc provides Connect RPC utilities for bidirectional mapping between domain AppErrors and Connect/gRPC status codes.
|
Package rpc provides Connect RPC utilities for bidirectional mapping between domain AppErrors and Connect/gRPC status codes. |
|
transport/server
Package server builds the ServerTransport from config: an unserved stub (the default) or Connect over net/http.
|
Package server builds the ServerTransport from config: an unserved stub (the default) or Connect over net/http. |
|
transport/server/stub
Package stub is the zero-infrastructure ServerTransport: handlers register on a mux that is never served, so a graph boots and its routes can be exercised in process (httptest) without opening a port.
|
Package stub is the zero-infrastructure ServerTransport: handlers register on a mux that is never served, so a graph boots and its routes can be exercised in process (httptest) without opening a port. |
|
workflows/loop
Package loop runs background work as periodic workflow passes.
|
Package loop runs background work as periodic workflow passes. |
|
workflows/outbox
Package outbox is the transactional-outbox relay: it leases due rows from an interfaces.OutboxRepository, delivers them through an interfaces.OutboxSink in bounded-concurrency chunks, and finalizes each row by its own outcome — sent, retried with capped exponential backoff and full jitter, or parked once its attempts are exhausted.
|
Package outbox is the transactional-outbox relay: it leases due rows from an interfaces.OutboxRepository, delivers them through an interfaces.OutboxSink in bounded-concurrency chunks, and finalizes each row by its own outcome — sent, retried with capped exponential backoff and full jitter, or parked once its attempts are exhausted. |
|
workflows/workflow
Package workflow is the workflow tier: the one boundary observed in every environment and the only tier that opens a transaction.
|
Package workflow is the workflow tier: the one boundary observed in every environment and the only tier that opens a transaction. |
Click to show internal directories.
Click to hide internal directories.