projectsync

Documentation

Overview

Package projectsync is the cross-machine project-data transport (project-sync): allowlist-selected bundle packing/unpacking with a checksummed manifest, and the machine-local import ledger. The identity layer (repo-born project ID) and the merge semantics (datamerge) live in their own packages; this package only moves bytes safely.

Bundle layout (tar.gz):

manifest.json          — first entry, format-guarded, lists every file + sha256
data/<rel>             — file payloads at their DataDir-relative paths

Trust model: sha256 protects against corruption, NOT malice (no signature). The execution safety line is elsewhere — verify-acceptance --trust-foreign and the import-time lineage check (same key ⇒ same developer's other machine).

Chinese strings use raw string literals (Windows quote-corruption rule).

Package projectsync 是项目数据跨机器传输层（project-sync）：allowlist 圈定的 bundle 打包/解包 + 校验和 manifest + 机器本地导入账本。身份层（repo-born project ID）与合并语义（datamerge）在各自包里；本包只负责安全搬运字节。

Bundle 布局（tar.gz）：

manifest.json          — 首条目，版本守卫，列出每个文件 + sha256
data/<rel>             — 文件载荷，按 DataDir 相对路径摆放

信任模型：sha256 防损坏不防恶意（无签名）。执行安全线在别处—— verify-acceptance --trust-foreign 与导入时 lineage 判定（同 key ⇒ 同一开发者的 另一台机器）。

Index

• Constants
• func AppendImportRecord(dataDir string, rec ImportRecord) error
• func ExportFiles(dataDir string, extra []string) ([]string, error)
• func HasImportedBundle(dataDir, bundleID string) (bool, error)
• func HasImportedSHA(dataDir, sha string) (bool, error)
• func NewBundleID() (string, error)
• func StripNonAllowlisted(dataDir string) ([]string, error)
• type FileEntry
• type ImportRecord
• type Manifest
  • func Pack(in PackInput, w io.Writer) (*Manifest, error)
  • func Unpack(r io.Reader, destDir string) (*Manifest, error)
• type Origin
• type PackInput

Constants

const (
	IncludeQuarantine = `quarantine`
	IncludeHazards    = `hazards`
)

IncludeQuarantine / IncludeHazards are the opt-in sensitive stores.

IncludeQuarantine / IncludeHazards 是显式选入的敏感 store。

const FormatVersion = 1

FormatVersion is the bundle format version. Import refuses a manifest whose version is 0 (missing/hand-edited) or greater than this (future format) — the same forward-compatibility guard task_port.go applies to task bundles: a future format must never be silently mis-parsed as the current one.

FormatVersion 是 bundle 格式版本。import 拒绝版本为 0（缺失/手改）或大于此值 （未来格式）的 manifest——与 task_port.go 对 task bundle 的前向兼容守卫同款： 未来格式绝不能被静默误解析为当前格式。

const MaxBundleBytes = 2 << 30

MaxBundleBytes is the total-payload ceiling Unpack enforces (2 GiB): the per-entry size is bounded by the manifest, but a forged manifest could declare a huge total and stream gigabytes to disk (a filler DoS against /tmp). Cheap bound; legitimate project bundles are orders of magnitude below it.

MaxBundleBytes 是 Unpack 强制的总载荷上限（2 GiB）：单条目尺寸被 manifest 约束， 但伪造的 manifest 可声明巨大总量向磁盘流数据（对 /tmp 的填充 DoS）。廉价上限； 合法项目 bundle 低于它若干数量级。

Functions

func AppendImportRecord

func AppendImportRecord(dataDir string, rec ImportRecord) error

AppendImportRecord appends one record to the machine-local ledger.

AppendImportRecord 向机器本地账本追加一条记录。

func ExportFiles

func ExportFiles(dataDir string, extra []string) ([]string, error)

ExportFiles walks dataDir and returns the slash-separated DataDir-relative paths that a bundle carries. extra is a list of opt-in includes (IncludeQuarantine / IncludeHazards); unknown values are an error (fail-closed on typos — silently dropping a typo'd "quarentine" would later surprise the user with a missing store).

Portable classes:

• tasks/*.json (NOT *.lock — per-task lock residue)
• checklog*.jsonl, toollog*.jsonl (active + rotated)
• sessions.jsonl, sessions/*.json
• act/conclusions.jsonl
• stamps/* (NOT stamps/hook-deploy — machine-local epoch stamp)
• protocol.yml

ExportFiles 遍历 dataDir，返回 bundle 携带的斜杠分隔 DataDir 相对路径。extra 是 选入的敏感 store（IncludeQuarantine / IncludeHazards）；未知值报错（对 typo fail-closed——静默丢掉拼错的 "quarentine" 会在日后以缺 store 惊吓用户）。

可移植类：

• tasks/*.json （不含 *.lock——per-task 锁残留）
• checklog*.jsonl、toollog*.jsonl（active + rotated）
• sessions.jsonl、sessions/*.json
• act/conclusions.jsonl
• stamps/* （不含 stamps/hook-deploy——机器本地 epoch 戳）
• protocol.yml

func HasImportedBundle

func HasImportedBundle(dataDir, bundleID string) (bool, error)

HasImportedBundle reports whether this machine already imported the given bundle id. Corrupt lines are skipped (a damaged ledger must never block imports), so a read error only surfaces when the file is unreadable.

HasImportedBundle 报告本机是否已导入过给定 bundle id。坏行跳过（损坏的账本 绝不阻塞导入），只有文件不可读才报错。

func HasImportedSHA

func HasImportedSHA(dataDir, sha string) (bool, error)

HasImportedSHA reports whether this machine already imported a bundle with this whole-file sha256 (same corrupt-line tolerance as HasImportedBundle). The digest is computable from the file ALONE, so the import flow can skip on it BEFORE unpacking — bundle ids live inside the tar, but a repeated pull should not pay the full tar parse to learn it already imported these bytes.

HasImportedSHA 报告本机是否已导入过带此整文件 sha256 的 bundle（坏行容忍与 HasImportedBundle 一致）。摘要仅凭文件本身即可计算，导入流程可在解包之前据其 跳过——bundle id 在 tar 内部，而重复 pull 不该为「已导入过这些字节」付整趟 tar 解析。

func NewBundleID

func NewBundleID() (string, error)

NewBundleID generates a random bundle id (16 bytes hex). Bundle ids need uniqueness, not secrecy.

NewBundleID 生成随机 bundle id（16 字节 hex）。bundle id 需要唯一性，不需要保密性。

func StripNonAllowlisted

func StripNonAllowlisted(dataDir string) ([]string, error)

StripNonAllowlisted removes every file under dataDir that the allowlist does not admit, returning the removed rel paths. THE IMPORT-SIDE ENFORCEMENT of default-deny: Unpack validates manifest↔tar consistency, but the manifest itself is untrusted (no signature) — a forged bundle can list imports.jsonl / active-task-ref-* / hooks/* / quarantine/** / hazards/** and datamerge would faithfully move them into the live DataDir (polluting the ledger, hijacking session anchors, bypassing the --include gate for sensitive stores). Opt-in stores are stripped unconditionally here: import has no --include flag in v1, so even a bundle honestly declaring Includes loses those payloads rather than gaining an unrequested sensitive store.

StripNonAllowlisted 删除 dataDir 下 allowlist 不放行的每个文件，返回被删的 rel 路径。这是 allowlist 默认拒绝在导入侧的执行：Unpack 只校验 manifest↔tar 一致， 而 manifest 本身不可信（无签名）——伪造 bundle 可在清单里列 imports.jsonl / active-task-ref-* / hooks/* / quarantine/** / hazards/**， datamerge 会忠实地把它们搬进活 DataDir（污染账本、劫持会话锚、绕过敏感 store 的 --include 门槛）。选入型 store 在此无条件剥除：v1 的 import 没有 --include flag，即使 bundle 诚实声明了 Includes，也是丢掉这些载荷而非换来一个未经请求 的敏感 store。

Types

type FileEntry

type FileEntry struct {
	Path   string `json:"path"` // DataDir-relative, slash-separated
	SHA256 string `json:"sha256"`
	Size   int64  `json:"size"`
}

FileEntry is one bundled file with its integrity fields.

FileEntry 是一个被打包的文件及其完整性字段。

type ImportRecord

type ImportRecord struct {
	BundleID   string    `json:"bundle_id"`
	SHA256     string    `json:"sha256"` // whole-bundle-file hash at import time
	ImportedAt time.Time `json:"imported_at"`
	FromKey    string    `json:"from_key,omitempty"`
	ToKey      string    `json:"to_key,omitempty"`
	Counts     string    `json:"counts,omitempty"` // human-readable action summary
}

ImportRecord is one ledger line.

ImportRecord 是一条账本行。

type Manifest

type Manifest struct {
	FormatVersion int         `json:"format_version"`
	ForgeVersion  string      `json:"forge_version,omitempty"`
	BundleID      string      `json:"bundle_id"`
	ExportedAt    time.Time   `json:"exported_at"`
	Origin        Origin      `json:"origin"`
	Files         []FileEntry `json:"files"`
	// Includes records the extra includes used at export (quarantine/hazards), so a
	// consumer can tell a sensitive bundle from a default one.
	//
	// Includes 记录导出时的额外包含（quarantine/hazards），使消费方能区分敏感
	// bundle 与默认 bundle。
	Includes []string `json:"includes,omitempty"`
}

Manifest is the bundle envelope. Every file entry is checksummed; Origin carries the source machine's identity derivation so the importer can decide lineage (same key ⇒ trusted sync) without any network round-trip.

Manifest 是 bundle 信封。每个文件条目带校验和；Origin 携带源机器的身份推导， 使导入方无需任何网络往返即可判定 lineage（同 key ⇒ 受信同步）。

func Pack

func Pack(in PackInput, w io.Writer) (*Manifest, error)

Pack writes the bundle (manifest.json + data/<rel>) as a gzip tar stream to w and returns the manifest it wrote. Two passes over each file: hash pass then stream pass — bundle payloads (rotated logs) can be large, so nothing is held fully in memory except the manifest. File order is sorted for deterministic bundles.

Pack 把 bundle（manifest.json + data/<rel>）以 gzip tar 流写到 w，返回写出的 manifest。每个文件两遍：先 hash 后流式写——bundle 载荷（rotated 日志）可能很 大，除 manifest 外不整体驻内存。文件排序保证 bundle 确定。

func Unpack

func Unpack(r io.Reader, destDir string) (*Manifest, error)

Unpack reads a bundle stream, validates it against its own manifest, and writes the payloads under <destDir>/data/. Security posture (mirrors cli/update.go extractBinary): regular files only (symlink/hardlink headers rejected), no absolute paths, no `..` traversal, every tar entry must be listed in the manifest and every listed entry must be present, format version double-guarded (0 and >FormatVersion both refused), per-file sha256+size verified while streaming. destDir must be outside FORGE_DATA_HOME (the caller owns that decision — staging must not be discoverable by DataDir scanners).

Unpack 读取 bundle 流，对照其 manifest 校验，把载荷写到 <destDir>/data/ 下。 安全姿态（镜像 cli/update.go extractBinary）：仅普通文件（拒绝 symlink/hardlink header）、拒绝绝对路径、拒绝 `..` 穿越、tar 条目必须在 manifest 列表内且列表 条目必须齐全、格式版本双拒（0 与 >FormatVersion）、流式校验逐文件 sha256+size。 destDir 必须在 FORGE_DATA_HOME 之外（该决策归调用方——staging 不能被 DataDir 扫描器发现）。

type Origin

type Origin struct {
	Hostname  string `json:"hostname,omitempty"`
	User      string `json:"user,omitempty"`
	Root      string `json:"root,omitempty"`
	Key       string `json:"key"`
	KeyMode   string `json:"key_mode"` // path | id
	ProjectID string `json:"project_id,omitempty"`
}

Origin is the source-machine provenance block.

Origin 是源机器溯源块。

type PackInput

type PackInput struct {
	DataDir      string
	Extra        []string // opt-in sensitive stores (IncludeQuarantine / IncludeHazards)
	Origin       Origin
	ForgeVersion string
	Now          time.Time // ExportedAt stamp (injected for test determinism)
}

PackInput parameterizes Pack.

PackInput 参数化 Pack。