kairosflux

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 25, 2026 License: Apache-2.0, MIT Imports: 13 Imported by: 0

README

KairosFlux

English | 中文

License Go Version Protocol Performance

KairosFlux (formerly BanDB) is an AI-native temporal data engine: every write is versioned and immutable, every read can ask "as of when", and every state can be replayed from the version ledger and checked against its own deterministic fingerprint. It is the temporal data-flow engine of ChronoBrew, an org built around one idea — Time-series is the substance, determinism is the differentiator, AI is the direction. QuantBrew (a deterministic A-share backtesting kernel) and ChronoScout (a full-market recon crawler) are its first two tenants — see ChronoBrew's architecture overview for how the three pieces fit together.

Underneath the temporal model sits a narrower, already-production substrate: high-frequency writes headed for a data warehouse are bursty upstream and easy to overwhelm downstream. KairosFlux absorbs the burst, validates and cleans each record against a machine-readable contract, buffers it durably, and (optionally) delivers it downstream at a pace the sink can handle.

Single binary. No third-party dependencies beyond gRPC/Protobuf, which is only used for an optional benchmarking endpoint (see Performance).

This repository has completed its product/protocol rename: BanDBKairosFlux, protocol brand BANLVKair. Module path, package name (bannetkairnet), cmd/* binaries, and docs have all moved. Frame format, opcodes, and cross-language test vector bytes are unchanged throughout, so nothing deployed today breaks. One file is deliberately still named client/python/bandb_client.py — it's slated to be retired outright in an upcoming cleanup rather than renamed in place, so its old identifiers (BanDBClient, BanDBError, ...) are left alone for now. Every command in this document was run against a real server while writing it — see Quick Start.

Why "AI-native"

Point-in-time correctness is the property an autonomous agent needs from its storage layer and the property a plain key-value store cannot give it:

  • Writes never overwrite. PUT_VERSIONED always creates a new immutable version of a logical key; nothing is ever mutated in place.
  • as_of(t) queries are point-in-time. GET_AS_OF(key, t) returns the latest version whose write time is <= t and never a version written after t — an agent re-running an old decision sees exactly what was known at that instant, not what's true now.
  • State is replayable and self-checkable. REPLAY_FINGERPRINT rebuilds the latest state of a key range from its version ledger and produces a deterministic SHA-256 over (LogicalKey, Seq, Payload), so any two processes that replay the same ledger can prove they landed on the same state without comparing raw bytes.
  • Every write carries who/when. Each version record carries an operation envelope (seq, write_ts, source, schema_ver, payload_hash) and is queryable via audit commands — "who wrote this key, and how many times, in this window" is an answered question, not a log-grepping exercise.
  • A data plane an agent can call directly is the direction, not a shipped wire feature yet. The embedded API (kairosflux.go) already exposes it: BuildContext (deterministic read bundles for agent research) and SubmitProposal (agent writes go through the same versioned/audited path). A network-visible agent plane remains roadmap.

Quick Start

Requires Go 1.26+. Every command below was run end-to-end against a real kairosflux-server instance while writing this document; exit codes are noted where they matter.

Build:

$ go build ./...

Start a server (another terminal):

$ cd cmd/kairosflux-server && go run .
...
2026/08/24 23:50:47 INFO kairnet server starting name=KairosFlux addr=127.0.0.1:8080

Write three versions of the same logical key — each call is a new, immutable version, not an overwrite:

$ go run ./cmd/kairosflux-cli -addr 127.0.0.1:8080 put-versioned order:1001 '{"amount":128}' scout
已写入版本 seq=1: order:1001 = {"amount":128}

$ go run ./cmd/kairosflux-cli -addr 127.0.0.1:8080 put-versioned order:1001 '{"amount":150}' scout
已写入版本 seq=2: order:1001 = {"amount":150}

$ go run ./cmd/kairosflux-cli -addr 127.0.0.1:8080 put-versioned order:1001 '{"amount":175}' scout
已写入版本 seq=3: order:1001 = {"amount":175}

(the trailing scout argument is the source field of the operation envelope — "who wrote this", see Roadmap / M2)

List the full version history, then read as of a timestamp that falls strictly between v2 and v3's write time:

$ go run ./cmd/kairosflux-cli -addr 127.0.0.1:8080 list-versions order:1001
seq=1 write_nanos=1787586686757950000 payload={"amount":128}
seq=2 write_nanos=1787586686829263000 payload={"amount":150}
seq=3 write_nanos=1787586686890053000 payload={"amount":175}

$ go run ./cmd/kairosflux-cli -addr 127.0.0.1:8080 get-as-of order:1001 1787586686850000000
seq=2 write_nanos=1787586686829263000 payload={"amount":150}

as_of returns v2, not v3 — even though v3 already exists in storage, the query time falls before its write time, so it is invisible to this read. This is the point-in-time guarantee, not an artifact of read timing.

Replay the ledger and self-check its fingerprint:

$ go run ./cmd/kairosflux-cli -addr 127.0.0.1:8080 fingerprint order:
逻辑键数=1 不一致数=0 指纹=2c4e10cc1ab683b5dbcec51920641b4765d737d809e57d623018c79d8aa56788

逻辑键数=1(1 logical key), 不一致数=0(0 mismatches against the :current pointer), followed by the 64-hex-char SHA-256 fingerprint of the replayed state. Two independent replays of the same ledger — including across a server restart — produce the same 64 characters.

Write and read a market-data snapshot with the Python client (v1 protocol, no version history)
$ python3 client/python/examples/write_quote.py --addr 127.0.0.1:8080 \
    --code 600000 --date 2026-08-17 \
    --open 10.0 --high 10.5 --low 9.8 --close 10.2 --volume 1000000 --prev-close 10.0
写入成功: key=quote:2026-08-17:600000
读回内容: {"code": "600000", "date": "2026-08-17", "open": 10.0, "high": 10.5, "low": 9.8, "close": 10.2, "volume": 1000000.0, "prev_close": 10.0}

A negative price is rejected by contract validation before it ever touches the buffer:

$ python3 client/python/examples/write_quote.py --addr 127.0.0.1:8080 \
    --code 600001 --date 2026-08-17 --open -1 --high 10.5 --low 9.8 --close 10.2 --volume 1000000
写入被拒绝(清洗/schema 校验未通过): dropped

Samples — deploy-as-code

KairosFlux is a library first: the same API surface (kairosflux.NewEmbedded / kairosflux.Open / kairosflux.Serve in kairosflux.go, the module-root top-level package) runs in-process or as a network shell. Three capability samples, each an independent main, each one command to run (all outputs below were captured by running them):

1. Embedded — in-process full flow. No network listener; put three versioned writes, read as-of a point strictly between versions (must see v1 only), list versions, replay the ledger fingerprint (self-check against :current, zero mismatches), then list audit writes.

$ go run ./cmd/kairosflux-sample-embedded -data-dir /tmp/kf-demo-data
写入版本 seq=1: {"code":"510300",...}
写入版本 seq=2: {"code":"510300",...}
写入版本 seq=3: {"code":"510300",...}
GET_AS_OF(9:30:00.5) → seq=1 payload={"code":"510300",...}
LIST_VERSIONS → 3 个版本(seq 升序)
REPLAY_FINGERPRINT → 逻辑键=1 指纹=26ae2bc46b88a30a4095810af8bed7002fcbf7d907ed64a1b0bf268f3dbbe668 对账不一致=0
LIST_WRITES → 3 条写入,按来源: sample-demo x3
[sample-embedded] 全链路通过(指纹对账零不一致)

2. Server + Python client — cross-language. kairosflux.Serve opens a real listening port; the Go leg does PUT_VERSIONED → GET_AS_OF over the wire via a thin v2 client, then the Python leg (repo client/python/bandb_client.py) does a v1 put/get round trip and a v2 ack=window batch write with FLUSH reconciliation and STAT counter check.

$ go run ./cmd/kairosflux-sample-server -port 19090 -data-dir /tmp/kf-demo-srv
[sample-server] 服务已就绪: 127.0.0.1:19090(数据目录 /tmp/kf-demo-srv)
[sample-server] Go 腿:PUT_VERSIONED 成功 seq=1
[sample-server] Go 腿:GET_AS_OF 成功 seq=1 payload={"code":"510300",...}
[sample-server] Python v1 腿: put/get 往返 OK, value = b'{"code":"600519",...}'
[sample-server] Python v2 腿: FLUSH/WINDOW_ACK received=3 accepted=3 rejected=0
[sample-server] Python v2 腿: STAT 累计 received=3 accepted=3
[sample-server] Python 腿全部通过
[sample-server] 两条腿全部通过

-python=off skips the Python leg (and a missing python3 skips it automatically with an honest note); exit code 0 means both legs passed.

3. Audit export. Multi-source writes, then a LIST_WRITES export to an append-only JSONL file with per-envelope hash self-check and a manifest line carrying a deterministic export_fingerprint (same shape as kairosflux-cli export-writes).

$ go run ./cmd/kairosflux-sample-audit -data-dir /tmp/kf-demo-audit -out /tmp/kf-audit.jsonl
[sample-audit] LIST_WRITES → 5 条写入,按来源: jobctl-reconcile x2 quantscout-crawler x3
[sample-audit] 导出完成: 5 条 → /tmp/kf-audit.jsonl(export_fingerprint=68d79402e6e431c578b992dfb1955faa1180c486c13473a3147874974f87910e,全部信封 hash 自检通过)

All three exit 0 on success, 1 on any failed step or detected inconsistency (e.g. an as-of read returning the wrong version, a fingerprint mismatch, a failed envelope hash check).

Architecture

   writers            Ingest         Clean              Temporal Store
 (Go / Python  ─Kair─▶ (kairnet TLV) ─▶ (contracts/  ─▶  (WAL + LSM, versioned)
  clients)                              schema, M1)
                                                           │
                                          PUT_VERSIONED ───┤──▶ new immutable version
                                          GET_AS_OF(k,t) ──┤──▶ latest version, write_ts<=t, never future
                                          LIST_VERSIONS ───┤──▶ full version history
                                          REPLAY_FINGERPRINT┤──▶ deterministic sha256 vs :current   (M0)
                                          LIST_WRITES /    ─┤──▶ audit: who wrote what, when          (M2,
                                          export-writes     │                                    merged)
                                                           │
                                    ┌──────────────────────┴───────────────────────┐
                                    ▼                                              ▼
                        Deliver (file / ClickHouse sink)                 AI Agent data plane
                        — existing v1 ingest pipeline,                   Context (read) / Proposal (write)
                          independent of the temporal opcodes            — embedded API shipped, wire surface
                                                                           remains roadmap

Two subsystems — a Multi-Raft sharded KV and a dubbo-go-inspired delivery governance layer — are fully implemented and tested but excluded from the default build (//go:build experimental); build with -tags experimental to include them. Wire-format details: docs/Kair-协议规范.md. Temporal key-space and semantics: docs/架构与语义总览.md.

Features

Status Capability
Implemented — M0 Versioned writes never overwrite; PUT_VERSIONED / GET_AS_OF / LIST_VERSIONS / REPLAY_FINGERPRINT opcodes wired end-to-end through server and CLI (shipped 2026-08-24).
Implemented — M1 Machine-readable per-record-type contracts (contracts/*.schema.json: key layout, PIT semantics, idempotency key, validation rules), fail-fast contract loading, structured validation sub-codes (0x30010x3004), timestamp-monotonicity checks dispatched by a declared time-kind instead of a colon-position heuristic on the key string.
Merged — M2 LIST_WRITES audit query (opcode 0x0D) and a per-version operation envelope (seq, write_ts, source, schema_ver, payload_hash) with an envelope version tag and lazily-migrated reads for pre-M2 records; COUNT-by-source aggregation; append-only, deterministically-ordered JSONL export; REPLAY_FINGERPRINT upgraded to a dataset/as-of-scoped callable service. Full test suite and race detector green (merged 2026-08-25).
Merged — M3 A declarative Job control plane (job:spec:{name} / job:status:{name} / job:events:{name}:v{seq}) built on the existing versioned opcodes, a single-process reconcile loop (internal/jobctl + cmd/kairosflux-jobctl), and an explicit strategy lifecycle state machine (Hypothesis → Gate → Candidate → Paper → Live/Retired). Verified by a 10,000-rerun idempotency test against a live server.
Implemented — M4 (embedded API) An AI-native data plane: a Context surface for agents to read point-in-time state and a Proposal surface to write through the same versioned/audited path — implemented as internal/aiplane and exposed via kairosflux.Engine.SubmitProposal / BuildContext. A network-visible agent plane over the wire remains roadmap.

Deployment Notes

For the current production shape — single machine, a single writer (QuantScout), daily batch writes — leave AdmissionEnabled and ShardRoutingEnabled off in config/config.json (both already default to false, see config/global.go):

  • AdmissionEnabled guards against concurrent-write overload with adaptive shedding. A daily batch job from one writer never produces the concurrent burst this exists to shed — there's no overload problem domain to defend against here, only the added latency-probing overhead.
  • ShardRoutingEnabled forwards keys that don't belong to the local node to their owner across a multi-node placement. A single node owns 100% of the keyspace, so there is no routing decision to make.

Flip either on only when the deployment actually grows into the shape they're for (concurrent multi-writer load, or a real multi-node placement) — leaving them on by default in this shape adds overhead and a probing/forwarding surface with nothing behind it to protect or route to.

Data Cleaning

Every write passes through a cleaning hook before it's buffered: frame and size checks, optional timestamp monotonicity (dispatched by the write's declared time-kind as of M1, not a heuristic over the key string), and a contract-driven schema registry keyed by record type (service/ingesthook/schema, contracts/*.schema.json). A rejected record returns dropped and never touches the buffer. The bundled market-data contract enforces required fields, positive prices, OHLC consistency, and a ±21% sanity bound on daily price change (21%, not 20% — ChiNext/STAR-market limit-up is ±20% off the previous close, but the limit-up price itself can be up to ~20.02% above it after rounding, so a 20% threshold clips legitimate limit-up prints; see service/ingesthook/schema/quote.go). Adding a new record type is one contract file plus one Validator implementation — see the package docs.

Temporal Core — Semantics

See docs/架构与语义总览.md for the full write-up: the temporal key space (logical key / version key / :current pointer), the as_of(t) contract and its point-in-time guarantee, the fingerprint definition, and the bitemporal roadmap (M0 unifies valid-time and write-time; separating them is M2+ scope). The RFC this was built from — docs/rfc/时态内核-M0-版本化与as-of.md — has the wiring decisions and their rationale in full.

The protocol side of this (RFC stage, zero code shipped — see docs/rfc/Kair-2.md, whose header literally says "design document, no code changes yet") sketches what production usage actually needs: write-heavy, read-light. The real load is QuantScout's daily batch export of ~5000 rows, not interactive request/response, so v2 designs three ack tiers selectable per connection — every (today's behavior, one response per write), window (batched acknowledgment every N writes or on FLUSH), and none (fully fire-and-forget). Dropping per-write ack on none also drops the guarantee that a lost connection tells you what got lost — so the design makes reconciliation mandatory for that tier: a client on ack=none must be able to replay/diff what it sent against what the server actually has, or it must not use that tier. None of this exists in code yet; v1's ack=every remains the only shipped behavior.

Roadmap

The full four-milestone plan (M0–M4) lives in QuantBrew's 方案-BanDB-时态内核与AI数据平面.md. In short: M0/M1 are shipped (see Features); M2 (replay-as-a- service, the audit envelope, LIST_WRITES), M3 (declarative Job control plane) and M4 (the Context/Proposal AI data plane, internal/aiplane, exposed via kairosflux.Engine.SubmitProposal/BuildContext) are implemented; what remains on the roadmap is the network-visible agent plane over the wire.

Performance

发布口径基准(2026-08-25 实测,darwin/arm64 8 核,本机 fsync 地板约 250–530 次/s;完整 矩阵与已知测量缺陷见 docs/bench/01.md):

  • 载入:100w 版本化写入 16 并发载入耗时 19m8.61s(871 w/s),载入后 REPLAY_FINGERPRINT 对账零不一致(逻辑键=100000,对账不一致=0)
  • 写路径(数据量=1000000,每行 50000 采样,全 0 错误):
    • server v2 PUT_VERSIONED(ack=every/window/none):QPS 460–474,p50 约 16ms—— standalone 模式每次写 2 次 WAL append + fsync,吞吐被磁盘 fsync 速率钉住
    • embedded 进程内直调:QPS 452(与网络路径同一 fsync 地板,进程内 vs 网络无吞吐差)
    • server v1 PUTQPS 931 ≈ 2×v2(1 次 append vs 2 次,物理自洽检查通过)
  • 读路径GET_AS_OF 100w 档 p50 2.66 ms(embedded)/ 3.01 ms(server),10w 档 数百微秒级;前缀扫描与 LIST_WRITES 完整行以 10w 档为准(100w 档扫描路径因引擎读路径 文件句柄耗尽未能测——已知缺陷,转 M5-C)
  • 耐久:kill -9 复验恢复数 ≥ 已 ack 数(ack-after-fsync 契约在代码审查与 kill 复验中 均未破坏)

Reproduce with the stage B harness in cmd/kairosflux-bench(perf / footprint / adversarial / soak100 四个子命令)。

Robustness

go test -fuzz against the 4 frame-parsing entry points ran for a combined 300 seconds (5 minutes) and logged ~37.7 million executions with zero crashes (kairnet.FuzzUnPack 369,985 / proto.FuzzDecodeScanRequest 15,108,651 / proto.FuzzDecodeScanResponse 12,065,019 / ingesthook.FuzzParsePut 10,203,443). Full write-up, including the malformed-frame test matrix (truncated frames, oversized length claims, non-UTF8 msgIDs, slow-client half-writes) it grew out of: docs/iteration-2026-08-20-bannet-robustness-audit.md.

Who's using this

QuantScout (soon ChronoScout), a Python market-data crawler, writes full-market daily snapshots into KairosFlux as its first production tenant — see the Python client example above. A real 5241-row full-market export: 5222 rows accepted, 19 rejected (all explainable: 17 halted/ delisted/warning-flag stocks with no trade that day, 2 legitimate ChiNext limit-up prints tripping the ±21% sanity bound) — cross-checked by reading every row back with the Go client and diffing field-by-field against the Python-side source.

Documentation

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Documentation

Overview

Package kairosflux 是 KairosFlux 的可导入双模式引擎 API(顶层包,发布批次 阶段 A):embedded(纯进程内)与 server(同一套 API 的网络壳)共用同一个 Engine,时态内核五操作 + 审计 + Context/Proposal 访问口在两种模式下逐字节 同语义。跨仓调用方(ChronoBrew/ChronoBrew 的 sample 与 E2E 测试)用 `import kairosflux "github.com/ChronoBrew/KairosFlux"` 即可获得全部能力, 不需要 import 任何 internal 包。

三种构造形态:

  • NewEmbedded:embedded 模式。DataDir 不存在会创建,不开网络监听, 适合"把 KairosFlux 当库嵌进另一个进程"(E2E 测试、样例、CI 装置)。
  • Open:打开既有数据目录(embedded/server 由 Options.Port 决定,Port<=0 为 embedded)。DataDir 必须已存在——重启恢复(kill -9 后一致性验证、 长期数据复用)用这个形态。
  • Serve:server 模式。DataDir 不存在会创建,构造即绑定监听(Port 必填), Addr() 返回监听地址,Close() 优雅关停。server 模式的网络壳与生产装配 Node(service/node.go)是同一个核心接线(v1 Router + v2 RouterV2 + 采集过滤钩子),差别是 Node 额外挂分片路由/网关准入/下游投递/周期指标 这些按配置门控的生产子系统(Engine 的 server 模式不挂,接口面以"真的 会换实现"为准,不造插件注册表)。

与 Node 的关系:Node 仍是生产服务器装配(cmd/kairosflux-server 的薄壳), 读进程级 config.G;Engine 是给"把 KairosFlux 当库/当嵌入式服务"的调用方 用的实例级 API,每个 Engine 持有自己的数据目录与监听配置,互不干扰。 Engine 只支持 standalone 模式(WAL 持久化);Raft 集群是另一个部署形态, 不在本 API 的范围内。

已知边界(诚实标注,不是遗漏):

  • Engine 的进程内写路径(PutVersioned)不经过协议层角色强制 (service/router_v2.go 的 handlePutVersioned 对 source 的 agent 身份 校验只覆盖 PUT_VERSIONED 线协议帧)——进程内调用方本身就是可信代码, 角色强制是"线协议上防越权"的机制,不适用于同进程直调。server 模式下 经网络的写入仍受完整协议层强制。
  • kairnet.Server 的连接数上限(config.G.MaxConn)仍读进程级全局配置 (kairnet/server.go 的 acceptLoop,改动会牵动既有行为,未在本批次触碰): Engine 的 server 模式沿用进程默认值 1000。
  • Engine 不加载 contracts/*.schema.json(service 的 schema.LoadContractsDefault 是生产契约校验的前提;Engine 的采集过滤钩子对未注册前缀走默认放行 回退路径——见 service/ingesthook/filter.go 的 validate 回退),嵌入式 场景不需要仓库内契约文件即可运行。
  • 存储引擎无独立 Close(flush/compaction 工作协程随进程退出);Engine.Close 负责排空并关闭 WAL 文件句柄与(server 模式下)网络壳,保证同进程内 Open 复用同一数据目录安全。

Index

Constants

View Source
const (
	ProposalFactor         = aiplane.ProposalFactor
	ProposalHypothesis     = aiplane.ProposalHypothesis
	ProposalExperiment     = aiplane.ProposalExperiment
	ProposalRecommendation = aiplane.ProposalRecommendation
	ProposalReview         = aiplane.ProposalReview
)

ProposalKind 常量(值即 aiplane 的枚举值,跨仓调用方不需要 import internal 包即可引用)。

Variables

This section is empty.

Functions

This section is empty.

Types

type ContextBundle

type ContextBundle = aiplane.ContextBundle

ContextBundle 是 BuildContext 的输出(确定性上下文包,字段契约见 contracts/aiplane/context.schema.json)。

type ContextRequest

type ContextRequest = aiplane.ContextRequest

ContextRequest 是 BuildContext 的唯一输入参数(as-of 语义)。

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine 是 KairosFlux 的双模式引擎:embedded 与 server 共用同一个时态内核 (PUT_VERSIONED/GET_AS_OF/LIST_VERSIONS/REPLAY_FINGERPRINT/LIST_WRITES), server 模式额外持有网络壳(kairnet.Server,v1+v2 双协议)。Engine 的方法 在两种模式下逐字节同语义——这是"server 模式 = 同一 API 的网络壳"的落点。

func NewEmbedded

func NewEmbedded(opts Options) (*Engine, error)

NewEmbedded 构造 embedded 模式引擎:DataDir 不存在则创建,不开网络监听。 失败以 error 返回(不 panic、不静默降级),调用方据此决定如何处理。

func Open

func Open(opts Options) (*Engine, error)

Open 打开既有数据目录构造引擎(embedded/server 由 Options.Port 决定): DataDir 必须已存在,否则报错。这是"重启恢复"形态——kill -9 之后用同一 DataDir 重新 Open,WAL 重放 + SSTable 加载使数据回到崩溃前状态 (崩溃安全顺序见 internal/temporal 包文档:先落版本键、再落 :current 指针)。

func Serve

func Serve(opts Options) (*Engine, error)

Serve 构造 server 模式引擎:DataDir 不存在则创建,Port 必须 >0;构造完成 即已绑定监听(kairnet.Server.Start 同步绑定,绑定失败以 error 返回), Addr() 可读,调用方后续只需 Close() 优雅关停。网络壳与 Node 共用同一套 核心接线(v1 Router + v2 RouterV2 + 采集过滤钩子),Node 的分片路由/准入/ 投递/指标等按配置门控的生产子系统不挂载(见本文件顶部文档)。

func (*Engine) Addr

func (e *Engine) Addr() string

Addr 返回 server 模式的监听地址(Host:Port)。embedded 模式返回空串。

func (*Engine) BuildContext

func (e *Engine) BuildContext(req ContextRequest, contractsDir, redlinesPath string) (ContextBundle, error)

BuildContext 是 Context 访问口:为研究员 agent 组装确定性上下文包 (数据集契约 + 摘要、测过哪些因子、策略状态、风控红线 + 各自摘要), 同一请求(同一 as_of + 同一底层账本/契约/红线文件)两次调用逐字节相同。 contractsDir 通常是 contracts/ 目录,redlinesPath 通常是 riskredlines/redlines.json(委托 internal/aiplane.BuildContext)。

func (*Engine) Close

func (e *Engine) Close() error

Close 优雅关停引擎(幂等):server 模式先按 kairnet 三段式排空在途请求并 关闭监听,随后排空并关闭 WAL 文件句柄(service.KVServer.Close 负责)—— 此后同进程内用同一 DataDir 重新 Open 是安全的(WAL 文件唯一写者不变量 恢复)。存储引擎的 flush/compaction 工作协程随进程退出,无需也不存在单独 的关闭入口(见 storage.Engine 文档)。

func (*Engine) GetAsOf

func (e *Engine) GetAsOf(logical string, asOfNanos int64) (Version, bool, error)

GetAsOf 返回 logical 在 asOfNanos 时刻可见的最新版本(as-of 语义:绝不 返回 asOfNanos 之后写入的版本)。该时刻无可见版本时 found=false。

func (*Engine) ListVersions

func (e *Engine) ListVersions(logical string) ([]Version, error)

ListVersions 返回 logical 的全部版本,按 seq 升序(从未写过返回空切片)。

func (*Engine) ListWrites

func (e *Engine) ListWrites(prefix string, tFromNanos, tToNanos int64, sourceFilter string) (ListWritesResult, error)

ListWrites 是审计查询:扫描 prefix 下的全部版本键,按 [tFromNanos, tToNanos](<=0 表示对应方向无界)与 sourceFilter(""=不过滤)筛出每次 历史写入,返回信封列表(按 LogicalKey,Seq 升序,确定性输出)与按来源 聚合计数。temporal.Store 的幂等/崩溃安全保证使 HashOK 可作"数据在写入 之后是否发生过漂移"的逐条自检。

func (*Engine) PutVersioned

func (e *Engine) PutVersioned(logical string, payload []byte, writeNanos int64, source string, schemaVersion uint32) (uint64, error)

PutVersioned 写入一个不可变版本,返回分配到的 seq。与网络协议 PUT_VERSIONED 同一语义(source/schemaVersion 落进操作元数据信封,供 LIST_WRITES 审计 按来源/契约版本过滤);进程内直调不经过协议层 agent 角色强制(见本文件 顶部"已知边界")。writeNanos 是写入时刻(unix 纳秒),调用方控制——E2E 测试用可控时间戳构造 as-of 定点语义;生产通常传 time.Now().UnixNano()。

func (*Engine) ReplayFingerprint

func (e *Engine) ReplayFingerprint(prefix string, asOfNanos int64) (ReplayResult, error)

ReplayFingerprint 对 prefix(数据集)重放出每个逻辑键的最新状态并计算 确定性指纹(sha256 over (LogicalKey, Seq, Payload) 集合)。asOfNanos<=0 无时间上界并核对 :current 指针(Mismatches 是真实对账结果);asOfNanos>0 按此刻重放(Bounded=true,不做 :current 对账,见 ReplayResult.Bounded 的 文档——调用方必须区分"核对通过"与"没核对")。

func (*Engine) SubmitProposal

func (e *Engine) SubmitProposal(p Proposal) (fingerprint string, seq uint64, err error)

SubmitProposal 是 agent 写提议的唯一入口(Proposal 访问口):字段校验 + 角色强制(只接受 Proposal 对象,写落 proposal: 键空间)委托 internal/aiplane.SubmitProposal,返回提议指纹与本次写入分配到的 seq。 与 jobctl.V2Store(网络瘦客户端)是同一个 ReadWriter 语义的不同实现—— 这里是进程内直连 TemporalStore。

type ListWritesResult

type ListWritesResult = service.ListWritesResult

ListWritesResult 是 LIST_WRITES 的结果(Entries + BySource 聚合, 见 service.ListWritesResult)。

type Options

type Options struct {
	// DataDir 是数据目录(WAL 与 SSTable 的落盘处)。必填。
	DataDir string

	// Host 是 server 模式的监听主机(默认 "127.0.0.1")。embedded 模式不使用。
	Host string

	// Port 是 server 模式的监听端口。<=0 构造 embedded 模式(不开网络监听);
	// Serve 要求 >0。
	Port int

	// WindowSafetyValveN 透传给 RouterV2 的 ack=window 安全阀 N
	// (service/router_v2.go 的 windowSafetyValveN 文档);<=0 用生产默认值
	// 1000。embedded 模式下无网络连接,不参与 ack 协商,本字段无意义。
	WindowSafetyValveN uint32
}

Options 是 Engine 的唯一构造参数(NewEmbedded/Open/Serve 三形态共用)。

type Proposal

type Proposal = aiplane.Proposal

Proposal 是 agent 唯一能写入的对象(字段契约见 contracts/aiplane/proposal.schema.json)。

type ProposalKind

type ProposalKind = aiplane.ProposalKind

ProposalKind 枚举 agent 能提交的提议种类(factor/hypothesis/ experiment/recommendation/review)。

type ReplayResult

type ReplayResult = service.ReplayResult

ReplayResult 是 REPLAY_FINGERPRINT 的结果(KeyCount/Fingerprint/ Mismatches/Bounded,见 service.ReplayResult)。

type Version

type Version = temporal.Version

Version 是 PUT_VERSIONED 写入的一条不可变版本记录(LogicalKey/Seq/ WriteNanos/Source/SchemaVer/PersistedHash/Payload,见 internal/temporal.Version)。

type WriteEnvelope

type WriteEnvelope = service.WriteEnvelope

WriteEnvelope 是 LIST_WRITES 返回的单条审计信封 (LogicalKey/Seq/WriteNanos/Source/SchemaVer/PersistedHash/Payload/ HashOK,见 service.WriteEnvelope)。

Directories

Path Synopsis
Package client 是 KairosFlux 的 Go 客户端 SDK。
Package client 是 KairosFlux 的 Go 客户端 SDK。
Package cluster 是分片集群的控制面:决定「一个 key 归属哪个分片、哪个物理节点」, 以及节点间的读择优与转发连接复用。
Package cluster 是分片集群的控制面:决定「一个 key 归属哪个分片、哪个物理节点」, 以及节点间的读择优与转发连接复用。
cmd
kairosflux-bench command
命令 kairosflux-bench 是压测工具集:
命令 kairosflux-bench 是压测工具集:
kairosflux-cli command
命令 kairosflux-cli 是 KairosFlux 的命令行客户端,基于 client SDK。
命令 kairosflux-cli 是 KairosFlux 的命令行客户端,基于 client SDK。
kairosflux-crashsim command
命令 kairosflux-crashsim 是找茬面的崩溃模拟器(阶段 B):独立进程打开 embedded 引擎连续写入 N 条版本化记录,每 50 条用原子 rename 更新一次 progress 文件(记录"已拿到 fsync 确认的最大下标"),供外部观察者在任意 时刻 SIGKILL(或 rlimit 触发信号)后验证崩溃一致性:
命令 kairosflux-crashsim 是找茬面的崩溃模拟器(阶段 B):独立进程打开 embedded 引擎连续写入 N 条版本化记录,每 50 条用原子 rename 更新一次 progress 文件(记录"已拿到 fsync 确认的最大下标"),供外部观察者在任意 时刻 SIGKILL(或 rlimit 触发信号)后验证崩溃一致性:
kairosflux-grpc-server command
命令 kairosflux-grpc-server 是 Kair(kairnet TLV) 协议的基准测试/协议对照服务端, 不是生产摄入入口——生产摄入用 cmd/kairosflux-server(Kair)。
命令 kairosflux-grpc-server 是 Kair(kairnet TLV) 协议的基准测试/协议对照服务端, 不是生产摄入入口——生产摄入用 cmd/kairosflux-server(Kair)。
kairosflux-ingest command
命令 ingest 是 M1 的 A1 层压测:进程内直接驱动存储引擎,证明存储引擎 在内存封顶下扛得住高频顺序写。
命令 ingest 是 M1 的 A1 层压测:进程内直接驱动存储引擎,证明存储引擎 在内存封顶下扛得住高频顺序写。
kairosflux-jobctl command
kairosflux-jobctl 是 M3 声明式 Job 控制面的独立命令行入口(docs/方案- BanDB-时态内核与AI数据平面.md §M3)。
kairosflux-jobctl 是 M3 声明式 Job 控制面的独立命令行入口(docs/方案- BanDB-时态内核与AI数据平面.md §M3)。
kairosflux-sample-audit command
命令 kairosflux-sample-audit 是"audit 审计导出"能力样例(发布批次阶段 A): embedded 引擎写入多来源数据后,用 LIST_WRITES 导出 append-only JSONL 审计文件(每行一条写入信封:logical_key/seq/write_ts/source/schema_ver/ payload_hash/payload_b64/hash_ok),末尾追加清单行(导出条数 + export_fingerprint——导出内容的确定性摘要,见 exportManifestRecord 的 文档:它是"本次导出文件"的完整性校验值,不是数据集状态指纹,二者不可 互相比较)。
命令 kairosflux-sample-audit 是"audit 审计导出"能力样例(发布批次阶段 A): embedded 引擎写入多来源数据后,用 LIST_WRITES 导出 append-only JSONL 审计文件(每行一条写入信封:logical_key/seq/write_ts/source/schema_ver/ payload_hash/payload_b64/hash_ok),末尾追加清单行(导出条数 + export_fingerprint——导出内容的确定性摘要,见 exportManifestRecord 的 文档:它是"本次导出文件"的完整性校验值,不是数据集状态指纹,二者不可 互相比较)。
kairosflux-sample-embedded command
命令 kairosflux-sample-embedded 是"embedded 模式进程内全流程"能力样例 (发布批次阶段 A):把 KairosFlux 当库嵌进进程,跑通 合成数据 → PUT_VERSIONED 版本化写入 → GET_AS_OF 定点读取 → LIST_VERSIONS 版本清单 → REPLAY_FINGERPRINT 重放指纹(:current 对账)→ LIST_WRITES 审计 全链路,全部走 kairosflux.go(仓库根)的可导入 API,不开任何网络监听。
命令 kairosflux-sample-embedded 是"embedded 模式进程内全流程"能力样例 (发布批次阶段 A):把 KairosFlux 当库嵌进进程,跑通 合成数据 → PUT_VERSIONED 版本化写入 → GET_AS_OF 定点读取 → LIST_VERSIONS 版本清单 → REPLAY_FINGERPRINT 重放指纹(:current 对账)→ LIST_WRITES 审计 全链路,全部走 kairosflux.go(仓库根)的可导入 API,不开任何网络监听。
kairosflux-sample-server command
命令 kairosflux-sample-server 是"server 模式 + Python 客户端"能力样例 (发布批次阶段 A):kairosflux.Serve 起一个真实监听端口,同一套 API 作为 网络壳对外服务——先由 Go 侧 v2 瘦客户端(kairnet/codec + negotiate + proto 拼帧,与 kairosflux-cli 同一模式)经真实线协议完成 PUT_VERSIONED → GET_AS_OF 往返,再由 Python 客户端(仓库 client/python/bandb_client.py:v1 直写直读 + v2 ack=window 批量写 + FLUSH 对账 + STAT 累计计数)演示跨语言访问。
命令 kairosflux-sample-server 是"server 模式 + Python 客户端"能力样例 (发布批次阶段 A):kairosflux.Serve 起一个真实监听端口,同一套 API 作为 网络壳对外服务——先由 Go 侧 v2 瘦客户端(kairnet/codec + negotiate + proto 拼帧,与 kairosflux-cli 同一模式)经真实线协议完成 PUT_VERSIONED → GET_AS_OF 往返,再由 Python 客户端(仓库 client/python/bandb_client.py:v1 直写直读 + v2 ack=window 批量写 + FLUSH 对账 + STAT 累计计数)演示跨语言访问。
kairosflux-server command
与 server_pprof.go(//go:build pprof) 互斥:两者各自定义 main,缺少本约束会使 `go build -tags pprof` 因 main 重复声明而失败,pprof 构建不可用。
与 server_pprof.go(//go:build pprof) 互斥:两者各自定义 main,缺少本约束会使 `go build -tags pprof` 因 main 重复声明而失败,pprof 构建不可用。
Package config 定义 KairosFlux 的全局运行配置及其加载。
Package config 定义 KairosFlux 的全局运行配置及其加载。
internal
admission
Package admission 提供网关入口的「自适应并发限流 / 准入控制」。
Package admission 提供网关入口的「自适应并发限流 / 准入控制」。
aiplane
Package aiplane 实现 M4 AI 数据平面(docs/方案-BanDB-时态内核与AI数据平面.md §M4):让"Agent 只读真相、只写提议、引擎裁决"成为正式接口。
Package aiplane 实现 M4 AI 数据平面(docs/方案-BanDB-时态内核与AI数据平面.md §M4):让"Agent 只读真相、只写提议、引擎裁决"成为正式接口。
credit
Package credit 提供字节级信用池(令牌桶式背压):写入方 Acquire 占用字节信用, 持久化方 Release 归还信用;信用不足时 Acquire 阻塞,从而把未持久化数据的内存占用 限制在预算之内。
Package credit 提供字节级信用池(令牌桶式背压):写入方 Acquire 占用字节信用, 持久化方 Release 归还信用;信用不足时 Acquire 阻塞,从而把未持久化数据的内存占用 限制在预算之内。
identity
Package identity 是"写请求发起方角色"这一条规则的唯一真相来源:把 internal/aiplane.Role(agent 只能写 Proposal、引擎不受限)从 API 层的一道 闸门升级为协议层强制(M4 上报缺口,任务书:"把 M4 的 WriteAsAgent API 层 闸门升级为协议层强制")时,service/router_v2.go 需要在 handlePutVersioned 里按解出的 source 字段做角色校验——但 service 不能 import internal/aiplane: internal/aiplane/integration_test.go 与 internal/jobctl/v2store_integration_test.go 都是同包内部测试(package aiplane / package jobctl,不是 _test 后缀的外部 测试包)且都 import service 起真实服务端做端到端验证,若 service 反过来 import internal/aiplane,会在 `go test ./internal/aiplane/...` 编译测试 二进制时触发"import cycle not allowed in test"(已用最小复现验证过这个 编译期错误,不是理论风险)。
Package identity 是"写请求发起方角色"这一条规则的唯一真相来源:把 internal/aiplane.Role(agent 只能写 Proposal、引擎不受限)从 API 层的一道 闸门升级为协议层强制(M4 上报缺口,任务书:"把 M4 的 WriteAsAgent API 层 闸门升级为协议层强制")时,service/router_v2.go 需要在 handlePutVersioned 里按解出的 source 字段做角色校验——但 service 不能 import internal/aiplane: internal/aiplane/integration_test.go 与 internal/jobctl/v2store_integration_test.go 都是同包内部测试(package aiplane / package jobctl,不是 _test 后缀的外部 测试包)且都 import service 起真实服务端做端到端验证,若 service 反过来 import internal/aiplane,会在 `go test ./internal/aiplane/...` 编译测试 二进制时触发"import cycle not allowed in test"(已用最小复现验证过这个 编译期错误,不是理论风险)。
jobctl
Package jobctl 实现 M3 对象模型与声明式 Job 控制面(docs/方案-BanDB- 时态内核与AI数据平面.md §M3):daily 流水线从 shell 串联升级为"声明式 任务 + 本地 reconcile 循环"。
Package jobctl 实现 M3 对象模型与声明式 Job 控制面(docs/方案-BanDB- 时态内核与AI数据平面.md §M3):daily 流水线从 shell 串联升级为"声明式 任务 + 本地 reconcile 循环"。
kvgrpc
Package kvgrpc 是 KairosFlux 的 gRPC 传输实现,位于 internal/ 之下:它是内部传输, 不是对外契约。
Package kvgrpc 是 KairosFlux 的 gRPC 传输实现,位于 internal/ 之下:它是内部传输, 不是对外契约。
metrics
Package metrics 提供零依赖的进程内可观测性:一组原子计数器 + 仪表回调, 以「周期性 slog 快照」作为暴露出口——headless 边缘设备直接 tail 日志即可观测, 无需开端口、无需 Prometheus/Grafana 等外部基础设施。
Package metrics 提供零依赖的进程内可观测性:一组原子计数器 + 仪表回调, 以「周期性 slog 快照」作为暴露出口——headless 边缘设备直接 tail 日志即可观测, 无需开端口、无需 Prometheus/Grafana 等外部基础设施。
temporal
Package temporal 定义 KairosFlux 时态内核(M0)的版本化记录语义。
Package temporal 定义 KairosFlux 时态内核(M0)的版本化记录语义。
codec
Package codec 是 Kair 帧格式的编解码层:字节 ↔ Message 的转换,只认字节, 不认 msgID 该分派给谁、不认连接生命周期——这是重构 RFC (docs/rfc/bannet-重构.md)第一步迁移的目标包,从根包 kairnet 的 message.go/datapack.go 原样搬入,本步不改变任何字节布局或行为,只搬家。
Package codec 是 Kair 帧格式的编解码层:字节 ↔ Message 的转换,只认字节, 不认 msgID 该分派给谁、不认连接生命周期——这是重构 RFC (docs/rfc/bannet-重构.md)第一步迁移的目标包,从根包 kairnet 的 message.go/datapack.go 原样搬入,本步不改变任何字节布局或行为,只搬家。
dispatch
Package dispatch 是分发层:路由表(msgID → Handler)、worker 池调度、 panic 隔离——见 docs/rfc/bannet-重构.md C.2/C.5,msghandle.go 整体迁入。
Package dispatch 是分发层:路由表(msgID → Handler)、worker 池调度、 panic 隔离——见 docs/rfc/bannet-重构.md C.2/C.5,msghandle.go 整体迁入。
handler
Package handler 是 kairnet 面向业务代码的契约层:用户实际要实现/使用的 唯一一组类型(Handler/Request/HookAction/Conn),边界不因本次重构而 变复杂——这是重构 RFC(docs/rfc/bannet-重构.md)C.2 所说的"业务层"。
Package handler 是 kairnet 面向业务代码的契约层:用户实际要实现/使用的 唯一一组类型(Handler/Request/HookAction/Conn),边界不因本次重构而 变复杂——这是重构 RFC(docs/rfc/bannet-重构.md)C.2 所说的"业务层"。
lifecycle
Package lifecycle 是连接生命周期层:一个显式的状态机(Idle→Active→ Closing→Closed),取代此前散落在 connection.go 里的裸 context.Context+sync.Once 组合——见 docs/rfc/bannet-重构.md B.2/C.1: 重构前没有任何地方能回答"这个连接现在处于什么状态",状态只存在于一堆 副作用的组合里。
Package lifecycle 是连接生命周期层:一个显式的状态机(Idle→Active→ Closing→Closed),取代此前散落在 connection.go 里的裸 context.Context+sync.Once 组合——见 docs/rfc/bannet-重构.md B.2/C.1: 重构前没有任何地方能回答"这个连接现在处于什么状态",状态只存在于一堆 副作用的组合里。
negotiate
Package negotiate 实现 Kair v1/v2 的 HELLO 协商(docs/rfc/Kair-2.md §5/§5.1):v2 客户端连接后先发一个 v1 格式的 HELLO 帧,按是否在超时内 收到 v2 格式的响应判断对端版本,零破坏地与 v1 共存于同一条连接。
Package negotiate 实现 Kair v1/v2 的 HELLO 协商(docs/rfc/Kair-2.md §5/§5.1):v2 客户端连接后先发一个 v1 格式的 HELLO 帧,按是否在超时内 收到 v2 格式的响应判断对端版本,零破坏地与 v1 共存于同一条连接。
transport
Package transport 是传输层:只认原始字节的收发(Reader/Writer 循环、 连接注册表),不认 Kair 帧内容该分派给谁——见 docs/rfc/bannet-重构.md C.2/C.5,是重构第五步(最后一步)的迁移目标。
Package transport 是传输层:只认原始字节的收发(Reader/Writer 循环、 连接注册表),不认 Kair 帧内容该分派给谁——见 docs/rfc/bannet-重构.md C.2/C.5,是重构第五步(最后一步)的迁移目标。
Package predicate 提供一个最小的字段谓词,用于边缘查询的服务端下推: 对 JSON 值取出指定字段,与操作数按算子比较,只让命中的行通过—— 从而「只回传命中切片」而非整段原始流。
Package predicate 提供一个最小的字段谓词,用于边缘查询的服务端下推: 对 JSON 值取出指定字段,与操作数按算子比较,只让命中的行通过—— 从而「只回传命中切片」而非整段原始流。
Package proto 定义客户端/服务端的命名协议常量。
Package proto 定义客户端/服务端的命名协议常量。
本文件是 Raft 的类型、构造、生命周期与只读访问器。
本文件是 Raft 的类型、构造、生命周期与只读访问器。
delivery
Package delivery 是数仓写入前置缓冲的「下游投递层」骨架:把本地缓冲的数据 按批投递到一个或多个下游 sink(ClickHouse / Doris / 湖仓 / 文件)。
Package delivery 是数仓写入前置缓冲的「下游投递层」骨架:把本地缓冲的数据 按批投递到一个或多个下游 sink(ClickHouse / Doris / 湖仓 / 文件)。
delivery/governance
Package governance 是投递层的「治理」子包,借鉴 dubbo-go 的服务治理模型但落在数据面: 把多个下游 sink 当作一组要被治理的后端——熔断(breaker)、健康感知路由(router)、 健康探测(health)、退避重试(retry)。
Package governance 是投递层的「治理」子包,借鉴 dubbo-go 的服务治理模型但落在数据面: 把多个下游 sink 当作一组要被治理的后端——熔断(breaker)、健康感知路由(router)、 健康探测(health)、退避重试(retry)。
delivery/offset
Package offset 是投递层的「强一致 offset」子包:把每个 sink 的投递进度(游标) 持久化为一条 KV,从而在进程崩溃/重启后从已提交位置续投,而非从头重投。
Package offset 是投递层的「强一致 offset」子包:把每个 sink 的投递进度(游标) 持久化为一条 KV,从而在进程崩溃/重启后从已提交位置续投,而非从头重投。
ingesthook
Package ingesthook 提供一个挂在采集入口的真实 PreHandle 过滤钩子示例: 在数据落盘前完成「丢弃畸形帧 + 时间戳单调性校验 + schema 校验 + 字段脱敏」四件事, 把「可编程边缘采集缓冲网关」从挂载点变成有内容的演示。
Package ingesthook 提供一个挂在采集入口的真实 PreHandle 过滤钩子示例: 在数据落盘前完成「丢弃畸形帧 + 时间戳单调性校验 + schema 校验 + 字段脱敏」四件事, 把「可编程边缘采集缓冲网关」从挂载点变成有内容的演示。
ingesthook/schema
Package schema 提供「按数据类型注册校验规则」的最小注册表:每种业务数据类型 (如行情快照)注册一个 Validator,落盘前按 key 前缀分派到对应校验器。
Package schema 提供「按数据类型注册校验规则」的最小注册表:每种业务数据类型 (如行情快照)注册一个 Validator,落盘前按 key 前缀分派到对应校验器。
本文件是 SSTable 的类型与共用定义:磁盘布局常量、块索引、以及 SSTable 本身。
本文件是 SSTable 的类型与共用定义:磁盘布局常量、块索引、以及 SSTable 本身。

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL