Documentation
¶
Overview ¶
Package blob handles content-addressed blob storage in Fossil repository databases.
Fossil's blob format is a 4-byte big-endian uncompressed-size prefix followed by zlib-compressed data. Compress and Decompress handle this encoding transparently.
Store compresses content, computes its SHA1 hash, and inserts it into the blob table. Load retrieves and decompresses a blob by RID. StoreDeltaRaw stores content that arrived over the wire already delta-encoded, without expanding it. EncodeForStorage is the shared decision point for whether a receive path can skip Compress entirely and write already-encoded wire bytes straight to blob.content.
This package does not decide what gets delta-encoded. Deltifying an artifact that is already stored means rewriting an existing row, not inserting a new one, and the decision needs to expand content, which this package cannot do without importing internal/content. Both live in content.Deltify, which is the only place the policy is stated. StoreDelta inserts a new row as a delta and has no callers outside tests; it is not the commit path's primitive.
Index ¶
- func Compress(data []byte) (result []byte, err error)
- func Decompress(data []byte) (result []byte, err error)
- func EncodeForStorage(data []byte, verbatim []byte) ([]byte, error)
- func Exists(q db.Querier, uuid string) (libfossil.FslID, bool)
- func Load(q db.Querier, rid libfossil.FslID) (result []byte, err error)
- func Store(q db.Querier, content []byte) (rid libfossil.FslID, uuid string, err error)
- func StoreDelta(q db.Querier, content []byte, srcRid libfossil.FslID) (rid libfossil.FslID, uuid string, err error)
- func StoreDeltaRaw(q db.Querier, uuid string, deltaBytes []byte, srcRid libfossil.FslID, ...) (rid libfossil.FslID, err error)
- func StorePhantom(q db.Querier, uuid string) (rid libfossil.FslID, err error)
- type Inflater
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Compress ¶
Compress produces Fossil-compatible compressed blob content: [4-byte big-endian uncompressed size][zlib-compressed data]. This matches Fossil's blob_compress() in src/blob.c.
func Decompress ¶
Decompress handles Fossil's compressed blob format: [4-byte big-endian uncompressed size][zlib-compressed data]. The 4-byte prefix is skipped before decompressing.
func EncodeForStorage ¶
EncodeForStorage returns the bytes to write into blob.content for data. When verbatim is non-nil, it is already data re-expressed in Fossil's on-disk blob format (e.g. bytes received over the wire that were already encoded that way) and is returned as-is, with no zlib pass of our own. When verbatim is nil, data is compressed fresh via Compress.
Centralizing the choice here -- rather than in each receive-path caller -- means a caller with wire-verbatim bytes on hand never has to decide whether re-encoding is safe to skip; it just offers the bytes and this function decides. Locally authored content, which never has verbatim bytes to offer, is unaffected: it always takes the Compress path.
func Exists ¶
Exists reports whether a blob row with the given uuid is present, and returns its rid. It answers existence only.
It returns true for a phantom — a real blob row with size = -1 and NULL content, standing in for an artifact we know of but have not received. A phantom's content cannot be read, and neither can that of a delta whose chain bottoms out in one. Callers that are about to read content must use content.AvailableByUUID instead, which is transitive over the delta chain.
Mirrors Fossil's rid_from_uuid (src/xfer.c:70).
func StoreDelta ¶
func StoreDelta(q db.Querier, content []byte, srcRid libfossil.FslID) (rid libfossil.FslID, uuid string, err error)
StoreDelta inserts a NEW row holding content delta-encoded against srcRid. It has no callers outside tests and is not what the commit path uses: deltifying an already-stored artifact rewrites an existing row rather than inserting one, and the decision of what to deltify is content.Deltify's, which holds the whole policy in one place. Kept for the delta-chain fixtures several packages build with it.
func StoreDeltaRaw ¶
func StoreDeltaRaw(q db.Querier, uuid string, deltaBytes []byte, srcRid libfossil.FslID, storedBlob []byte) (rid libfossil.FslID, err error)
StoreDeltaRaw stores delta-encoded content exactly as given — without expanding it against its source — and records the delta-to-source link. Unlike StoreDelta, which computes a delta from full target content and verifies the round-trip against an already-readable source, StoreDeltaRaw accepts content that arrived already delta-encoded (e.g. over the wire during a transfer) whose source may itself still be a phantom. The target's size is read from the delta's own header (delta.OutputSize), so no source content is needed to store it.
If a real (non-phantom) blob already exists for uuid, this is a no-op. If a phantom row exists, it is filled in place. Otherwise a new row is created. In every case the row's declared size is real, never -1: the target is not phantomized just because its source might be — mirrors Fossil's content_put_ex (src/content.c:557-620), which stores delta content unconditionally and records REPLACE INTO delta(rid,srcid) whether or not the source is currently available. Availability is a live, transitively-recomputed property (content.IsAvailable), not something this function or any caller needs to update or be notified about; content.Expand verifies the expanded result against its claimed UUID on every read, since nothing here can verify a delta whose source isn't readable yet.
deltaBytes is wire data, not a programmer-controlled argument: malformed or empty input (including from a hostile peer) is reported as an error, never a panic. Argument shapes StoreDeltaRaw's own caller controls (q, uuid, srcRid) are still asserted, matching this package's usual nil/invalid-argument convention.
storedBlob, when non-nil, is deltaBytes already re-expressed in Fossil's on-disk blob format (e.g. bytes received over the wire already encoded that way) and is written verbatim via EncodeForStorage instead of being recompressed. Pass nil when no such bytes are available (e.g. a delta computed locally rather than received).
Types ¶
type Inflater ¶
type Inflater struct {
// contains filtered or unexported fields
}
Inflater loads a run of blobs through a single reusable zlib reader. The zero value is ready to use. It is not safe for concurrent use.
It exists for delta-chain replay, where one Expand inflates every delta on a chain in sequence. Building a fresh zlib reader per link re-allocates a ~32 KiB flate history window and its machinery each time, and that per-link setup was the single largest allocation source in sub-64 KiB chain expansion. Resetting one reader instead pays the setup once for the whole chain. Each Load still returns a freshly allocated buffer the caller owns — only the reader is shared, so results never alias one another.
Source Files