Documentation
¶
Overview ¶
Package rootfsdiff compares recorded contents and metadata of root filesystem tar archives without extracting or executing their contents.
Scan reads tar/tar.gz streams, Compare produces deterministic changes, and ParsePolicy/Evaluate apply explicit required-path and payload-growth gates. The separate public report package renders text, JSON and Markdown; the CLI composes these operations. See docs/v0.1-contract.md for their semantics.
Call Validate on caller-built models or after decoding model JSON. Validation does not normalize, sort, mutate, print or access the filesystem. Manifests may have arbitrary input order; produced manifests and diffs are sorted. Complete manifests validate hardlink closure. Individual change entries need not include their target because comparison ignores can remove that target. Nil model slices marshal as empty arrays, and absent change sides as null. Warning path "." preserves metadata warnings on an ignored root directory record without retaining or synthesizing a root entry.
Core functions never extract or execute archive contents. Scan accepts a caller-owned io.Reader and does not close it. Context cancellation is checked between bounded reads; it cannot interrupt an arbitrary reader already blocked inside Read. Independent calls have no mutable global options; callers must not concurrently mutate input models while a call reads them. Sizes, counters and numeric ownership use int64; JSON modes are decimal 07777 bit masks, while human renderers use octal. See the contract for exclusions, limits, error categories and the complete schema.
Index ¶
- Constants
- Variables
- func MetadataExclusions() []string
- func ValidatePath(p string) error
- type Change
- type ChangeKind
- type CompareOptions
- type Diff
- type Entry
- type EntryKind
- type Error
- type ErrorKind
- type Field
- type Growth
- type LimitError
- type Manifest
- type Policy
- type PolicyRule
- type Report
- type ScanOptions
- type Summary
- type Violation
- type Warning
- type WarningCode
- type WarningSource
Examples ¶
Constants ¶
const ( WarningUncomparedMetadata WarningCode = "uncompared_metadata" SourceBefore WarningSource = "before" SourceAfter WarningSource = "after" )
const ( ChangeAdded ChangeKind = "added" ChangeRemoved ChangeKind = "removed" ChangeModified ChangeKind = "modified" FieldKind Field = "kind" FieldMode Field = "mode" FieldUID Field = "uid" FieldGID Field = "gid" FieldSize Field = "size" FieldSHA256 Field = "sha256" FieldLinkTarget Field = "link_target" FieldDeviceMajor Field = "device_major" FieldDeviceMinor Field = "device_minor" )
const ( DefaultMaxEntries int64 = 100_000 DefaultMaxArchiveBytes int64 = 8 << 30 DefaultMaxFileBytes int64 = 2 << 30 DefaultMaxPathBytes int64 = 4_096 )
const SchemaVersion = 1
Variables ¶
var ( ErrInvalidInput = errors.New("invalid input") ErrUnsupportedInput = errors.New("unsupported input") ErrLimitExceeded = errors.New("limit exceeded") )
Error categories support errors.Is through wrapping. I/O and cancellation errors retain their original identity through Error.Unwrap.
Functions ¶
func MetadataExclusions ¶
func MetadataExclusions() []string
MetadataExclusions returns a fresh, fixed-order slice for every report.
func ValidatePath ¶
ValidatePath checks a canonical POSIX root-relative model path. It does not normalize input, impose scan limits, interpret backslashes or glob syntax, or consult the host filesystem. Root records are absent from manifests.
Types ¶
type Change ¶
type Change struct {
Path string `json:"path"`
Kind ChangeKind `json:"kind"`
Before *Entry `json:"before"`
After *Entry `json:"after"`
ChangedFields []Field `json:"changed_fields"`
}
Change has exactly the applicable sides. Modified fields are complete and in the fixed contract order. Added/removed records use an empty field array.
func (Change) MarshalJSON ¶
type ChangeKind ¶
type ChangeKind string
type CompareOptions ¶
type CompareOptions struct {
IgnorePaths []string `json:"ignore_paths"`
IgnorePrefixes []string `json:"ignore_prefixes"`
}
CompareOptions applies literal canonical paths to both sides. Prefixes match themselves and descendants separated by '/', never neighboring names. Input order and duplicate rules are allowed and have no semantic effect.
func (CompareOptions) MarshalJSON ¶
func (o CompareOptions) MarshalJSON() ([]byte, error)
func (CompareOptions) Validate ¶
func (o CompareOptions) Validate() error
type Diff ¶
type Diff struct {
Summary Summary `json:"summary"`
Changes []Change `json:"changes"`
PositiveGrowth []Growth `json:"positive_growth"`
Warnings []Warning `json:"warnings"`
}
Diff records every retained change and every warning, even for ignored paths. PositiveGrowth is complete, ordered by decreasing bytes then increasing path.
func Compare ¶
func Compare(before, after Manifest, opts CompareOptions) (Diff, error)
Compare returns the complete deterministic difference between two manifests. It validates both complete manifests and opts before applying ignore rules. Entries may arrive in any order. Inputs are never mutated, and returned change sides are independent copies; callers must not mutate inputs during a call.
Exact ignores match only their literal canonical path. A prefix matches itself and descendants separated by '/', so "var/log" does not match "var/logger". Both sides are filtered alike. Every warning survives filtering and receives its before/after source. No implicit directory entries are manufactured.
Changes sort by path and list every differing field in this order: kind, mode, uid, gid, size, sha256, link_target, device_major, device_minor. Type changes are single modified records. Only Entry fields are compared: timestamps, user/group names, directory sizes and excluded metadata are not represented as comparable fields. Hardlinks use their validated effective inode metadata.
Payload totals sum retained regular-entry sizes with checked int64 arithmetic; hardlink aliases and all other kinds contribute zero. Net growth is after minus before, including additions, removals and type transitions. This is uncompressed regular payload, not flash size or filesystem disk usage. PositiveGrowth contains every positive per-path regular-payload delta, sorted by descending bytes then path, without a presentation limit. Failures return a zero Diff and a typed validation error, never a successful partial result.
Example ¶
package main
import (
"crypto/sha256"
"fmt"
rd "github.com/Azhovan/rootfs-diff"
)
func main() {
file := func(path, contents string) rd.Entry {
return rd.Entry{
Path: path, Kind: rd.KindRegular, Mode: 0644,
Size: int64(len(contents)), SHA256: fmt.Sprintf("%x", sha256.Sum256([]byte(contents))),
}
}
// Caller-built manifests may be unordered. No implicit directories are needed.
before := rd.Manifest{Entries: []rd.Entry{
file("usr/bin/agent", "v1"),
file("var/log/boot", "old log"),
file("etc/device.conf", "enabled\n"),
}}
after := rd.Manifest{
Entries: []rd.Entry{
file("usr/bin/agent", "version2"),
file("etc/new", "x"),
file("var/log/boot", "new log contents"),
file("etc/device.conf", "enabled\n"),
},
Warnings: []rd.Warning{{
Path: "var/log/boot", Code: rd.WarningUncomparedMetadata, Key: "SCHILY.xattr.user.build",
}},
}
diff, err := rd.Compare(before, after, rd.CompareOptions{IgnorePrefixes: []string{"var/log"}})
if err != nil {
panic(err)
}
for _, change := range diff.Changes {
fmt.Printf("%s %s %v\n", change.Kind, change.Path, change.ChangedFields)
}
// Totals count retained regular payloads, not filesystem disk usage.
fmt.Printf("payload: %d -> %d bytes (%+d)\n", diff.Summary.BeforePayloadBytes,
diff.Summary.AfterPayloadBytes, diff.Summary.NetGrowthBytes)
for _, growth := range diff.PositiveGrowth {
fmt.Printf("growth: %s +%d bytes\n", growth.Path, growth.Bytes)
}
// Ignoring changes and payload at a path never suppresses its warnings.
for _, warning := range diff.Warnings {
fmt.Printf("warning: %s %s %s\n", warning.Source, warning.Path, warning.Key)
}
}
Output: added etc/new [] modified usr/bin/agent [size sha256] payload: 10 -> 17 bytes (+7) growth: usr/bin/agent +6 bytes growth: etc/new +1 bytes warning: after var/log/boot SCHILY.xattr.user.build
func (Diff) MarshalJSON ¶
type Entry ¶
type Entry struct {
Path string `json:"path"`
Kind EntryKind `json:"kind"`
Mode uint32 `json:"mode"`
UID int64 `json:"uid"`
GID int64 `json:"gid"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
LinkTarget string `json:"link_target"`
DeviceMajor int64 `json:"device_major"`
DeviceMinor int64 `json:"device_minor"`
}
Entry records explicit archive metadata, with numeric mode bits (07777 mask) and lowercase hexadecimal SHA-256. Inapplicable fields are zero or empty. Hardlinks retain their own path/kind/target and use terminal regular-entry mode, ownership, size and digest. UID/GID preserve signed numeric tar IDs.
type Error ¶
Error adds operation and archive-path context without losing an underlying I/O error. Path is an archive path, never an absolute host filename.
type ErrorKind ¶
type ErrorKind string
ErrorKind identifies invalid or unsupported input. An empty kind leaves the underlying error's identity intact without assigning an input category.
type LimitError ¶
LimitError identifies the exhausted budget. Actual is the first observed value beyond Maximum, not necessarily the full size of an unread input. Err may retain an underlying read error when both facts are known.
func (*LimitError) Error ¶
func (e *LimitError) Error() string
func (*LimitError) Is ¶
func (e *LimitError) Is(target error) bool
func (*LimitError) Unwrap ¶
func (e *LimitError) Unwrap() error
type Manifest ¶
Manifest contains only explicit entries. Caller-built entries and warnings may have any order; producers return sorted slices. Nil means empty.
Example ¶
package main
import (
"fmt"
rd "github.com/Azhovan/rootfs-diff"
)
func main() {
manifest := rd.Manifest{Entries: []rd.Entry{{
Path: "etc/device.conf", Kind: rd.KindRegular, Mode: 0640,
UID: 123, GID: 456, Size: 0,
SHA256: "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
}}}
if err := manifest.Validate(); err != nil {
panic(err)
}
fmt.Printf("%s mode=%04o uid=%d gid=%d\n", manifest.Entries[0].Path,
manifest.Entries[0].Mode, manifest.Entries[0].UID, manifest.Entries[0].GID)
}
Output: etc/device.conf mode=0640 uid=123 gid=456
func Scan ¶
Scan reads an uncompressed or gzip-compressed tar stream, detected by magic bytes. It never extracts entries, resolves host links, or closes r. Only explicit entries are returned, sorted by canonical POSIX path. Payloads are hashed in bounded chunks and are not retained.
Zero options select documented defaults. The decoded-byte budget includes headers, extensions and padding; the logical-entry budget includes ignored root directory records. Path limits apply to decoded names before path normalization. Hardlinks retain their canonical target and inherit effective data from their terminal regular entry. Uncompared PAX metadata produces sorted warnings containing keys, never values. Standard metadata exclusions are available through MetadataExclusions for the final report.
Scan checks ctx between reads and during link resolution. Cancellation cannot interrupt an arbitrary reader already blocked in Read. The caller owns any deadline or Close needed to unblock that reader. On error Scan returns a zero Manifest, never a successful partial result. Independent calls share no state.
Example ¶
package main
import (
"archive/tar"
"bytes"
"context"
"fmt"
rootfsdiff "github.com/Azhovan/rootfs-diff"
)
func main() {
// Archive creation is only for this self-contained example. In an
// application, pass an opened local tar or tar.gz file as the reader.
var archive bytes.Buffer
w := tar.NewWriter(&archive)
if err := w.WriteHeader(&tar.Header{Name: "etc/device.conf", Typeflag: tar.TypeReg, Mode: 0640, Uid: 100, Gid: 200, Size: 5}); err != nil {
panic(err)
}
if _, err := w.Write([]byte("hello")); err != nil {
panic(err)
}
if err := w.Close(); err != nil {
panic(err)
}
manifest, err := rootfsdiff.Scan(context.Background(), &archive, rootfsdiff.ScanOptions{})
if err != nil {
panic(err)
}
for _, entry := range manifest.Entries {
fmt.Printf("%s %s %04o %d:%d %d %s\n", entry.Path, entry.Kind, entry.Mode, entry.UID, entry.GID, entry.Size, entry.SHA256)
}
}
Output: etc/device.conf regular 0640 100:200 5 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
func (Manifest) MarshalJSON ¶
type Policy ¶
type Policy struct {
SchemaVersion int `json:"schema_version"`
RequiredPaths []string `json:"required_paths"`
MaxTotalGrowthBytes *int64 `json:"max_total_growth_bytes,omitempty"`
}
func ParsePolicy ¶
ParsePolicy reads one strict JSON policy of at most 1 MiB (1,048,576 bytes), including whitespace. It does not close the caller-owned reader. The object must contain schema_version (the integer 1) and required_paths (an array of exact canonical paths). The optional max_total_growth_bytes is a nonnegative int64 integer; omission disables the growth rule, while zero forbids positive net growth. Unknown or duplicate object keys, nulls, invalid Unicode, noninteger numbers and trailing values are errors. Duplicate required paths are allowed and have no extra effect. Paths are validated, never normalized.
Invalid JSON or policy values return an error matching ErrInvalidInput. Input over the byte budget returns *LimitError matching ErrLimitExceeded, with Limit "policy_bytes". Underlying reader errors remain reachable through errors.Is or errors.As. Every failure returns a zero Policy, never a partial policy.
Example ¶
package main
import (
"errors"
"fmt"
"strings"
rd "github.com/Azhovan/rootfs-diff"
)
func main() {
policy, err := rd.ParsePolicy(strings.NewReader(`{
"schema_version": 1,
"required_paths": ["etc/init.d/S40network", "usr/bin/device-agent"],
"max_total_growth_bytes": 1048576
}`))
if err != nil {
panic(err)
}
fmt.Println(policy.SchemaVersion, policy.RequiredPaths)
fmt.Printf("maximum net growth: %d bytes\n", *policy.MaxTotalGrowthBytes)
_, err = rd.ParsePolicy(strings.NewReader(`{"schema_version":1,"required_paths":[],"typo":0}`))
fmt.Println("invalid policy:", errors.Is(err, rd.ErrInvalidInput))
}
Output: 1 [etc/init.d/S40network usr/bin/device-agent] maximum net growth: 1048576 bytes invalid policy: true
func (Policy) MarshalJSON ¶
type PolicyRule ¶
type PolicyRule string
const ( RuleRequiredPath PolicyRule = "required_path" RuleMaxTotalGrowthBytes PolicyRule = "max_total_growth_bytes" )
type Report ¶
type Report struct {
SchemaVersion int `json:"schema_version"`
Summary Summary `json:"summary"`
Changes []Change `json:"changes"`
PositiveGrowth []Growth `json:"positive_growth"`
Violations []Violation `json:"violations"`
Warnings []Warning `json:"warnings"`
MetadataExclusions []string `json:"metadata_exclusions"`
}
Report is the complete schema-version-1 output. No filenames, timestamps or run-dependent values belong here. A renderer must validate before writing.
Example ¶
package main
import (
"encoding/json"
"fmt"
rd "github.com/Azhovan/rootfs-diff"
)
func main() {
report := rd.Report{SchemaVersion: rd.SchemaVersion, MetadataExclusions: rd.MetadataExclusions()}
if err := report.Validate(); err != nil {
panic(err)
}
encoded, err := json.Marshal(report)
if err != nil {
panic(err)
}
fmt.Println(string(encoded))
}
Output: {"schema_version":1,"summary":{"before_entries":0,"after_entries":0,"added":0,"removed":0,"modified":0,"unchanged":0,"before_payload_bytes":0,"after_payload_bytes":0,"net_growth_bytes":0},"changes":[],"positive_growth":[],"violations":[],"warnings":[],"metadata_exclusions":["acls","directory_sizes","file_capabilities","selinux_labels","timestamps","unrecognized_pax_metadata","user_group_names","xattrs"]}
func (Report) MarshalJSON ¶
type ScanOptions ¶
type ScanOptions struct {
MaxEntries int64 `json:"max_entries"`
MaxArchiveBytes int64 `json:"max_archive_bytes"`
MaxFileBytes int64 `json:"max_file_bytes"`
MaxPathBytes int64 `json:"max_path_bytes"`
}
ScanOptions limits decoded input, including headers, extensions and padding. Zero fields select defaults. Positive values override them; negatives fail. MaxPathBytes applies independently to final entry paths and link targets.
func (ScanOptions) Validate ¶
func (o ScanOptions) Validate() error
func (ScanOptions) WithDefaults ¶
func (o ScanOptions) WithDefaults() (ScanOptions, error)
WithDefaults validates options and returns an independent effective value.
type Summary ¶
type Summary struct {
BeforeEntries int64 `json:"before_entries"`
AfterEntries int64 `json:"after_entries"`
Added int64 `json:"added"`
Removed int64 `json:"removed"`
Modified int64 `json:"modified"`
Unchanged int64 `json:"unchanged"`
BeforePayloadBytes int64 `json:"before_payload_bytes"`
AfterPayloadBytes int64 `json:"after_payload_bytes"`
NetGrowthBytes int64 `json:"net_growth_bytes"`
}
Summary counts entries after ignore filtering, not inode identities. Payload totals count regular entries only. All counts and byte values are int64.
type Violation ¶
type Violation struct {
Rule PolicyRule `json:"rule"`
Path string `json:"path"`
Limit *int64 `json:"limit"`
Actual *int64 `json:"actual"`
}
Violation has a path only for required_path and numeric values only for the growth rule. Renderers generate descriptions from these stable fields.
func Evaluate ¶
Evaluate validates all inputs and returns unique violations sorted by rule, then path. It does not mutate inputs; returned numeric pointers are independent copies. An empty rule set or a passing policy returns an empty slice.
Required paths inspect the full after manifest, including comparison-ignored entries. Only explicit entries count: an implicit parent does not satisfy a required directory. Presence is sufficient for every kind, including symlinks; no links are resolved and no runtime usability or executability is implied. A required_path violation identifies the missing path and has nil numeric data.
The growth rule uses diff.Summary.NetGrowthBytes, the comparison's signed net uncompressed regular-payload growth after ignore filtering. It does not sum PositiveGrowth or recompute totals from the full candidate. Equality with the byte limit and shrinking pass, including a zero-byte baseline. A growth failure has rule max_total_growth_bytes, empty path, Limit and Actual byte values. The caller must supply the diff for its intended comparison and candidate.
Invalid policies, manifests or diffs return a typed ErrInvalidInput error and no partial violations. A violation is a successful evaluation, not an error.
Example ¶
package main
import (
"fmt"
"strings"
rd "github.com/Azhovan/rootfs-diff"
)
func main() {
// These manifests contain only explicit entries. An implicit "etc" parent
// is not required here; a symlink's presence is enough, without resolving it.
before := rd.Manifest{Entries: []rd.Entry{{Path: "obsolete", Kind: rd.KindDirectory}}}
after := rd.Manifest{Entries: []rd.Entry{
{Path: "etc/device.conf", Kind: rd.KindSymlink, LinkTarget: "../settings"},
{Path: "var/log/boot", Kind: rd.KindFIFO},
}}
diff, err := rd.Compare(before, after, rd.CompareOptions{IgnorePrefixes: []string{"var/log"}})
if err != nil {
panic(err)
}
policy, err := rd.ParsePolicy(strings.NewReader(`{
"schema_version": 1,
"required_paths": ["var/log/boot", "usr/bin/device-agent", "etc/device.conf"],
"max_total_growth_bytes": 0
}`))
if err != nil {
panic(err)
}
violations, err := rd.Evaluate(after, diff, policy)
if err != nil {
panic(err)
}
// Ignored var/log/boot still satisfies presence. The symlink also passes.
// Added/removed nonregular entries have zero payload growth, so zero passes.
fmt.Printf("net growth: %d bytes\n", diff.Summary.NetGrowthBytes)
for _, violation := range violations {
fmt.Println(violation.Rule, violation.Path)
}
}
Output: net growth: 0 bytes required_path usr/bin/device-agent
type Warning ¶
type Warning struct {
Source WarningSource `json:"source"`
Path string `json:"path"`
Code WarningCode `json:"code"`
Key string `json:"key"`
}
Warning identifies a metadata key, never its value. Manifest warnings have empty Source; comparison/report warnings have a before or after Source. Path is canonical, except "." identifies an ignored root directory record.
type WarningCode ¶
type WarningCode string
type WarningSource ¶
type WarningSource string
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
rootfs-diff
command
|
|
|
examples
|
|
|
library
command
Command library demonstrates the public rootfsdiff and report packages using tiny in-memory tar inputs.
|
Command library demonstrates the public rootfsdiff and report packages using tiny in-memory tar inputs. |
|
synthetic
Package synthetic supplies the small, offline release-review demonstration.
|
Package synthetic supplies the small, offline release-review demonstration. |
|
synthetic/cmd/demo
command
Command demo generates deterministic inputs and checks a compiled rootfs-diff.
|
Command demo generates deterministic inputs and checks a compiled rootfs-diff. |
|
internal
|
|
|
cli
Package cli composes the library and report renderer for the rootfs-diff command.
|
Package cli composes the library and report renderer for the rootfs-diff command. |
|
testfixture
Package testfixture builds small in-memory archives for tests.
|
Package testfixture builds small in-memory archives for tests. |
|
Package report renders validated rootfsdiff reports without scanning archives or evaluating policies.
|
Package report renders validated rootfsdiff reports without scanning archives or evaluating policies. |