delta

Documentation

Overview

Package delta implements rsync-style rolling checksums and block matching for efficient partial file transfers.

Index

• Constants
• func Apply(basis []byte, d *Delta) ([]byte, error)
• func MarshalDelta(d *Delta) ([]byte, error)
• func MarshalSignature(sig *Signature) ([]byte, error)
• type BlockChecksum
• type Delta
  • func Encode(r io.Reader, sig *Signature) (*Delta, error)
  • func EncodeBytes(target []byte, sig *Signature) (*Delta, error)
  • func UnmarshalDelta(data []byte) (*Delta, error)
• type Op
• type OpKind
• type Signature
  • func Sign(r io.Reader, blockSize int) (*Signature, error)
  • func SignBytes(data []byte, blockSize int) (*Signature, error)
  • func UnmarshalSignature(data []byte) (*Signature, error)

Constants

const DefaultBlockSize = 4096

DefaultBlockSize is used when a caller does not specify a block size.

const MaxDecodeOps = 16 << 20 // 16M

MaxDecodeOps caps ops/blocks allocated when unmarshalling untrusted wire data. Bounded well below MaxMessageSize-scale payloads (each op is at least 1 byte).

const MaxOutputSize = 64 << 20 // 64 MiB

MaxOutputSize caps reconstructed file size from a delta (matches typical MaxFileBytes / proto framing). Rejects malicious OutputSize before allocate.

Functions

func Apply

func Apply(basis []byte, d *Delta) ([]byte, error)

Apply reconstructs the target file by applying d against basis. basis must be the full original file contents that produced the signature.

func MarshalDelta

func MarshalDelta(d *Delta) ([]byte, error)

MarshalDelta serializes a Delta to a compact binary form. Format: magic(4) blockSize(u32) outputSize(u64) nOps(u32) then ops. OpLiteral: kind(u8) len(u32) data OpCopy: kind(u8) blockIndex(u32)

func MarshalSignature

func MarshalSignature(sig *Signature) ([]byte, error)

MarshalSignature serializes a Signature for the wire protocol.

Types

type BlockChecksum

type BlockChecksum struct {
	// Index is the block number in the basis file.
	Index int `json:"index"`
	// Weak is the Adler-32-like rolling checksum.
	Weak uint32 `json:"weak"`
	// Strong is MD5 of the block for match-only candidate confirmation (rsync-like).
	// MD5 is not a security boundary: after apply, the full file is verified with
	// SHA-256 against the manifest hash before commit.
	Strong [md5.Size]byte `json:"strong"`
}

BlockChecksum holds weak (rolling) and strong checksums for one block.

type Delta

type Delta struct {
	BlockSize  int
	Ops        []Op
	OutputSize int64
}

Delta is a sequence of ops that reconstruct a target file from a basis.

func Encode

func Encode(r io.Reader, sig *Signature) (*Delta, error)

Encode computes a delta from basis signature to the full target contents in r.

func EncodeBytes

func EncodeBytes(target []byte, sig *Signature) (*Delta, error)

EncodeBytes is Encode over an in-memory buffer.

func UnmarshalDelta

func UnmarshalDelta(data []byte) (*Delta, error)

UnmarshalDelta parses a binary delta produced by MarshalDelta.

type Op

type Op struct {
	Kind OpKind
	// Data is set for OpLiteral.
	Data []byte
	// BlockIndex is set for OpCopy (index into the basis signature).
	BlockIndex int
}

Op is one instruction in a delta: either a literal run or a basis block copy.

type OpKind

type OpKind byte

OpKind is the kind of a delta operation.

const (
	// OpLiteral copies raw bytes from the delta stream.
	OpLiteral OpKind = iota
	// OpCopy copies a block from the basis file by block index.
	OpCopy
)

type Signature

type Signature struct {
	BlockSize int             `json:"block_size"`
	Blocks    []BlockChecksum `json:"blocks"`
}

Signature describes the block layout of a basis file (the version the receiver already has).

func Sign

func Sign(r io.Reader, blockSize int) (*Signature, error)

Sign reads r and produces a block signature for use as a basis.

func SignBytes

func SignBytes(data []byte, blockSize int) (*Signature, error)

SignBytes is Sign over an in-memory buffer.

func UnmarshalSignature

func UnmarshalSignature(data []byte) (*Signature, error)

UnmarshalSignature parses a signature from MarshalSignature.