filter

package
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 22, 2026 License: MPL-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package filter implements the Git process boundary of FXVCS: the long-running clean/smudge filter (Git filter protocol version 2), the pointer-aware diff and merge drivers, the pre-push publication barrier, and launcher installation into repository-local Git config.

Normative behaviour is documented in docs/spec/filter-process.md, docs/spec/diff-merge-drivers.md, docs/spec/hook-integration.md, docs/spec/launcher-installation.md and docs/spec/failure-semantics.md.

Index

Constants

View Source 
const (
	ConflictOurs   = "<<<<<<< ours\n"
	ConflictSep    = "=======\n"
	ConflictTheirs = ">>>>>>> theirs\n"
)

Conflict layout markers.

View Source 
const HookMarker = "# fxvcs-managed pre-push hook"

HookMarker identifies a hook file written by fxvcs.

View Source 
const (
	// MaxPacketPayload is the largest payload one packet may carry.
	MaxPacketPayload = 65516
)

pkt-line framing as used by Git's long-running filter protocol: each packet is a 4-hex-digit length (including the 4 length bytes) followed by payload. "0000" is the flush packet that terminates a list.

View Source 
const (
	ProtocolVersion = format.FilterProtocolVersion
)

Protocol constants (git/Documentation/gitattributes: "Long Running Filter Process").

Variables

View Source 
var (
	// ErrUnsupportedPointer wraps pointer.ErrUnsupportedVersion with the
	// installed/required detail required at the byte boundary.
	ErrUnsupportedPointer = errors.New("filter: unsupported pointer format version")
	// ErrMalformedPointer is a blob that starts like a pointer but is invalid.
	ErrMalformedPointer = errors.New("filter: malformed pointer")
	// ErrObjectMismatch is a cached object whose bytes/size disagree with the
	// pointer; the working file is never written from it.
	ErrObjectMismatch = errors.New("filter: cached object does not match pointer")
)

Errors surfaced by the engine.

View Source 
var ErrHookExists = errors.New("filter: a pre-push hook that is not managed by fxvcs already exists")

ErrHookExists is returned when a user hook already occupies pre-push.

View Source 
var ErrNoPublicationLedger = errors.New("filter: publication ledger unavailable; cannot verify that pushed objects are published")

ErrNoPublicationLedger is returned when the hook runs without an engine Backend: publication cannot be judged, so the push is refused rather than waved through.

View Source 
var ErrNotPointerInput = errors.New("filter: input is not an fxvcs pointer")

ErrNotPointerInput is returned by drivers when they need pointer bytes and receive something else.

View Source 
var ErrObjectUnavailable = errors.New("filter: object not available locally")

ErrObjectUnavailable is returned by smudge for a path recorded as hydrated whose object is neither cached nor obtainable from a configured remote.

View Source 
var ErrProtocol = errors.New("filter: protocol error")

ErrProtocol is wrapped by every handshake/framing failure.

Functions

func DefaultLauncherPath 

func DefaultLauncherPath() (string, error)

DefaultLauncherPath returns the stable per-user launcher path for this OS.

func DefaultLegacyLauncherPath 

func DefaultLegacyLauncherPath() (string, bool, error)

DefaultLegacyLauncherPath is LegacyLauncherPath for the running OS.

func FormatUpgradeRequired 

func FormatUpgradeRequired(e *format.UpgradeRequiredError) string

FormatUpgradeRequired renders the multi-line stderr block every Git entry point prints when the repository requires a newer client.

func HookScript 

func HookScript(launcher string) string

HookScript renders the pre-push hook script that chains to the launcher.

func Install 

func Install(ctx context.Context, git *gitclient.Client, launcher string) error

Install writes GitConfigEntries(launcher) into the repository-local config of the checkout that git operates on. Values are set with `git config --local`, which is atomic per key; the set is idempotent.

func LauncherPath 

func LauncherPath(goos string, getenv func(string) string, home string) (string, error)

LauncherPath computes the launcher path for goos using getenv and home. It is separated from DefaultLauncherPath so all three shapes can be tested on one machine.

func LegacyLauncherPath 

func LegacyLauncherPath(goos string, getenv func(string) string, home string) (string, bool, error)

LegacyLauncherPath returns the exact former default launcher path when this OS has one. It is used only to recognize fxvcs-owned defaults during migration; arbitrary recorded paths remain custom and are never moved.

func PrePush 

func PrePush(ctx context.Context, co *Checkout, remoteName string, refs []PushRef, log io.Writer) error

PrePush implements the publication barrier for one push. remoteName is the first hook argument (used to select commits not yet on that remote when the remote ref is new). It scans the added/modified paths of every commit being pushed, keeps those whose filter attribute is fxvcs, decodes each pointer, publishes every object that is not yet recorded on a requiredForPublication remote (Backend.Publish uploads to bound remotes and records the result), and fails only when an object still lacks a publication record afterwards — because a required remote is unbound or unreachable, or the object is no longer available locally. Git refs therefore never become remotely reachable before their assets are durable, and the common path needs no separate `fxvcs objects publish`.

func PrePushRefusalMessage 

func PrePushRefusalMessage(res HookInstallResult, launcher string) string

PrePushRefusalMessage is the stderr text printed when a hook must be chained manually.

func QuoteCommandPath 

func QuoteCommandPath(p, goos string) string

QuoteCommandPath quotes an executable path for use inside a git config command value. Git runs these values through `sh -c` (Git for Windows ships its own sh), so:

  • on Windows backslashes become forward slashes, which every Git for Windows sh and the Windows loader accept, avoiding sh escaping rules;
  • the path is wrapped in double quotes when it contains a space or any shell-special character, and `"`, `$`, backtick, and backslash are escaped inside the quotes.

func RenderDiff 

func RenderDiff(w io.Writer, path string, oldSide, newSide Side) error

RenderDiff writes the side-by-side pointer metadata view and a one-line summary. Output is deterministic and contains no host paths.

func RunDiffDriver 

func RunDiffDriver(args []string, installed format.Version, stdout io.Writer) error

RunDiffDriver is the `fxvcs diff-driver` entry point (GIT_EXTERNAL_DIFF argument convention: path oldSide-file oldSide-hex oldSide-mode newSide-file newSide-hex newSide-mode). With a single argument (unmerged path) it prints a note.

func RunOneShot 

func RunOneShot(ctx context.Context, opts ProcessOptions, command, pathname string, stdin io.Reader, stdout, stderr io.Writer) error

RunOneShot is the `fxvcs clean-filter <path>` / `smudge-filter <path>` entry point for debugging and `git cat-file --filters`-style use: it reads the entire content from stdin and writes the result to stdout.

func RunPrePush 

func RunPrePush(ctx context.Context, opts ProcessOptions, remoteName string, stdin io.Reader, stderr io.Writer) error

RunPrePush is the `fxvcs pre-push <remote> <url>` entry point.

func RunProcess 

func RunProcess(ctx context.Context, opts ProcessOptions, stdin io.Reader, stdout, stderr io.Writer) error

RunProcess is the `fxvcs filter-process` entry point. It preflights the checkout, then serves the long-running filter protocol on stdin/stdout. Diagnostics go to stderr, which Git forwards to the user. It returns *format.UpgradeRequiredError (unwrapped) when the repository requires a newer client so the CLI can choose the upgrade-required exit code.

func Serve 

func Serve(ctx context.Context, r io.Reader, w io.Writer, h Handler, log io.Writer) error

Serve speaks the long-running filter protocol on r/w until Git closes the stream. It returns nil on a clean EOF and a wrapped ErrProtocol on framing or handshake violations. Handler errors are reported to Git per command and do not stop the server; they are also written to log (may be nil).

func Uninstall 

func Uninstall(ctx context.Context, git *gitclient.Client) error

Uninstall removes the repository-local driver configuration.

func WholeManifest 

func WholeManifest(domainID, oid string, size int64) domain.ObjectManifest

WholeManifest builds the whole/1, uncompressed manifest for an object.

func WholePointer 

func WholePointer(domainID, oid string, size int64) (pointer.Pointer, error)

WholePointer builds the deterministic pointer for a whole/1 object.

Types

type Backend 

type Backend interface {
	// Ingest records content as pending publication and returns the
	// deterministic pointer (identical bytes → identical pointer).
	Ingest(ctx context.Context, content io.Reader, path string) (pointer.Pointer, error)
	// HydrationIntent reports the intent recorded for path. known is false
	// when no row (or no state database) exists: the unhydrated default.
	HydrationIntent(ctx context.Context, path string) (hydrated, known bool, err error)
	// RecordHydrated notes, best effort, that the working file at path holds
	// the real bytes of digest (the clean filter just ingested them from it).
	RecordHydrated(ctx context.Context, path, digest string, size int64) error
	// FetchObject brings digest into the local cache from a configured
	// remote, verifying while streaming. objectstore.ErrNotFound when none
	// has it.
	FetchObject(ctx context.Context, digest string) error
	// ContentObjects returns every object digest the pointer depends on: the
	// file itself for whole/1, the manifest and its chunks for a chunked
	// encoding. It may need the manifest, so it can touch a remote.
	ContentObjects(ctx context.Context, ptr pointer.Pointer) ([]string, error)
	// OpenContent returns the whole file behind a pointer, fetching whatever
	// is missing and verifying every byte against the manifest and the
	// pointer. Any error means "no valid content"; the caller must not write
	// a partial result.
	OpenContent(ctx context.Context, ptr pointer.Pointer) (io.ReadCloser, error)
	// MissingRemotes returns the requiredForPublication remotes that hold no
	// publication record for the pointer's object.
	MissingRemotes(ctx context.Context, ptr pointer.Pointer) ([]string, error)
	// MissingRemotesMany is MissingRemotes for many pointers at once, keyed by
	// pointer oid. The barrier uses it instead of a loop: against a network
	// object store, one query per asset is one sequential round trip per
	// asset, paid before any upload starts.
	MissingRemotesMany(ctx context.Context, ptrs []pointer.Pointer) (map[string][]string, error)
	// RequiredRemotes lists the requiredForPublication remote names.
	RequiredRemotes() []string
	// Publish uploads the given pending objects (by digest) to every
	// requiredForPublication remote bound on this machine and records the
	// result in the ledger. It returns per-object failures instead of
	// stopping at the first, and never pushes Git refs. log receives
	// human-readable progress (Git shows hook stderr to the user).
	Publish(ctx context.Context, digests []string, log io.Writer) []PublishFailure
	// Close releases the backend (databases, locks).
	Close() error
}

Backend is the engine the Git process boundary delegates to for everything that must agree with the SDK operations: ingestion (cache pin + publication ledger row through the single ingestion path), hydration intent recorded in state.db, remote fetches, and the publication ledger consulted by the pre-push barrier. It is implemented over app.Repo by internal/filter/wire; this package never imports the engine so the dependency stays one-way.

Every method is bounded local work except FetchObject and MissingRemotes, which may talk to configured remotes; the caller holds no lock around them.

type Checkout 

type Checkout struct {
	Root   string // working-tree top level
	Repo   *domain.Repository
	Git    *gitclient.Client
	Cache  *cache.Cache
	Domain string
	// Backend is the engine (nil in cache-only mode, see ProcessOptions).
	Backend Backend
	// ConfigSource says where repository.yaml came from: "worktree",
	// "index" or "HEAD".
	ConfigSource string
	// contains filtered or unexported fields
}

Checkout is the resolved Git/FXVCS context of a filter or driver process.

func OpenCheckout 

func OpenCheckout(ctx context.Context, opts ProcessOptions) (*Checkout, error)

OpenCheckout resolves the working tree through Git, loads .fxvcs/repository.yaml (working tree first, then the index, then HEAD so a clone whose checkout is still in progress can be served), and runs the mandatory preflight. On *format.UpgradeRequiredError the caller must stop before answering any Git command.

func (*Checkout) Close 

func (co *Checkout) Close() error

Close releases the backend.

type ConfigEntry 

type ConfigEntry struct {
	Key   string
	Value string
}

ConfigEntry is one repository-local git config key/value.

func GitConfigEntries 

func GitConfigEntries(launcher string) []ConfigEntry

GitConfigEntries returns the exact repository-local configuration that routes Git's filter, diff, and merge drivers to launcher. The launcher path is quoted for Git's shell so spaces and (on Windows) backslashes survive.

type Engine 

type Engine struct {
	Cache *cache.Cache
	// Domain is the storageDomainID new objects are ingested under (clean).
	Domain string
	// Installed is the running binary version, for diagnostics.
	Installed format.Version
	// Backend is optional (see type comment).
	Backend Backend
}

Engine performs clean and smudge. With a Backend it ingests through the engine's single ingestion path (cache pin + publication ledger row), consults the recorded hydration intent, and may fetch from configured remotes; without one (unit tests, degraded clone-in-progress mode) it works over the local object cache alone and never records anything.

func (*Engine) Clean 

func (e *Engine) Clean(ctx context.Context, req Request, content io.Reader, out io.Writer) error

Clean converts working-tree bytes into a pointer, ingesting the object into the cache and pinning it as pending publication. Content that already is a valid pointer passes through unchanged (idempotent). It streams: memory is bounded by pointer.MaxSize plus copy buffers regardless of file size.

func (*Engine) Smudge 

func (e *Engine) Smudge(ctx context.Context, req Request, content io.Reader, out io.Writer) error

Smudge converts a pointer into working-tree bytes according to the recorded hydration intent of the path:

  • no Backend, no state row, or intent unhydrated: emit the pointer bytes unchanged (the unhydrated default; a fresh clone holds pointers even when the host cache happens to contain the object);
  • intent hydrated: reconstruct from the cache, else fetch from a configured remote into the cache, else fail the Git operation (ErrObjectUnavailable) — never silently substitute a pointer for a path the user hydrated.

Non-pointer input passes through unchanged. It never emits partial or unverified content as success.

type Handler 

type Handler interface {
	Clean(ctx context.Context, req Request, content io.Reader, out io.Writer) error
	Smudge(ctx context.Context, req Request, content io.Reader, out io.Writer) error
}

Handler transforms content for one command. It reads the whole input from content and writes the result to out. Writing anything to out commits the response to status=success unless the handler then returns an error, in which case the protocol reports status=error after the partial content and Git discards the result. Returning an error before writing reports status=error immediately.

type HookInstallResult 

type HookInstallResult struct {
	Path      string
	HooksPath string // directory (honors core.hooksPath)
	Updated   bool   // an fxvcs-managed hook was refreshed
}

HookInstallResult reports where the hook went.

func InstallPrePushHook 

func InstallPrePushHook(ctx context.Context, git *gitclient.Client, launcher string) (HookInstallResult, error)

InstallPrePushHook writes the managed hook into the hooks directory Git reports (which honors core.hooksPath and linked worktrees). It never overwrites a hook it did not write: ErrHookExists is returned instead, and the caller should instruct the user to chain `fxvcs pre-push "$@"`.

type MergeOutcome 

type MergeOutcome int

MergeOutcome classifies a merge. 

const (
	// MergeClean means result holds the merged pointer (exit 0).
	MergeClean MergeOutcome = iota
	// MergeConflict means result holds the conflict layout (exit 1).
	MergeConflict
)

func Merge 

func Merge(base, ours, theirs []byte, installed format.Version) ([]byte, MergeOutcome, error)

Merge performs the pointer-aware three-way merge. Rules:

  1. ours == theirs (bytes) -> ours, clean
  2. ours or theirs is not a valid pointer -> ErrNotPointerInput
  3. base absent or not a pointer -> conflict (both sides added)
  4. ours == base -> theirs, clean
  5. theirs == base -> ours, clean
  6. otherwise -> conflict layout, both pointers kept

It never picks by timestamp or size.

type MergeResult 

type MergeResult struct {
	Outcome MergeOutcome
	Path    string
}

MergeResult is what RunMergeDriver did.

func RunMergeDriver 

func RunMergeDriver(basePath, oursPath, theirsPath, pathname string, installed format.Version) (MergeResult, error)

RunMergeDriver is the `fxvcs merge-driver %O %A %B %P` entry point. It rewrites the %A file with the result (or the conflict layout) and returns the outcome. On ErrNotPointerInput (or any error) %A is left untouched and the caller must exit non-zero so Git records the path as unmerged.

type PendingObject 

type PendingObject struct {
	Digest  string
	Domain  string
	Path    string // first path where it was seen
	Commit  string
	Missing []string // required remotes without a publication record
	Reason  string   // why automatic publication did not succeed, when known
	// Pointer is the index pointer the object belongs to; a chunked encoding
	// expands to a manifest object plus its chunks, all of which must be
	// published before the push may proceed.
	Pointer pointer.Pointer
}

PendingObject is a pointer reachable from a pushed commit whose object is not published to every required remote.

type ProcessOptions 

type ProcessOptions struct {
	// Installed is the running binary version (required).
	Installed format.Version
	// Cwd is the directory Git started the process in ("" = os.Getwd).
	Cwd string
	// Git overrides the git client (tests). nil = gitclient.New(Cwd).
	Git *gitclient.Client
	// CacheDir overrides the object cache root ("" = cache.DefaultDir()).
	CacheDir string
	// Connect opens the engine Backend for the resolved working tree
	// (internal/filter/wire supplies it). nil runs the cache-only engine:
	// clean pins in the cache but records no ledger row or intent, smudge
	// always emits pointers, pre-push cannot judge publication.
	Connect func(ctx context.Context, root string) (Backend, error)
	// Log receives diagnostics that are not errors (nil = discard).
	Log io.Writer
}

ProcessOptions configures the filter process and one-shot filters.

type PublicationBarrierError 

type PublicationBarrierError struct {
	Pending []PendingObject
}

PublicationBarrierError is returned when a push must be refused.

func (*PublicationBarrierError) Error 

func (e *PublicationBarrierError) Error() string

type PublishFailure 

type PublishFailure struct {
	Digest string
	Remote string
	Err    error
}

PublishFailure reports one object×remote upload that did not succeed.

type PushRef 

type PushRef struct {
	LocalRef, LocalOID, RemoteRef, RemoteOID string
}

PushRef is one line of the pre-push stdin protocol: 

<local ref> SP <local oid> SP <remote ref> SP <remote oid> LF

func ParsePushRefs 

func ParsePushRefs(r io.Reader) ([]PushRef, error)

ParsePushRefs parses the pre-push stdin lines.

type Request 

type Request struct {
	Command  string // clean | smudge
	Pathname string
	Ref      string // optional (smudge)
	Treeish  string // optional (smudge)
	Blob     string // optional (smudge)
	CanDelay bool   // Git offered delay; this implementation never delays
}

Request is one filter command from Git.

type Side 

type Side struct {
	Absent  bool
	Pointer *pointer.Pointer
	// For raw sides: computed identity so pointer and hydrated file compare.
	OID  string
	Size int64
	Mode string
}

Side describes one side of a diff: absent, a pointer, or raw bytes (a hydrated working file or a historical blob).

func ReadSide 

func ReadSide(path, hexOID, mode string, installed format.Version) (Side, error)

ReadSide classifies a file handed to the diff driver. path=="", "/dev/null", "nul", or mode "." marks an absent side. The hex is not used for that decision because Git reports an all-zero hex for a working-tree file that does exist. Raw sides are hashed while streaming so memory stays bounded.

Source Files

View all Source files

Directories

Path Synopsis
Package wire connects the Git process boundary (internal/filter) to the engine (internal/app) so that the clean filter ingests through the same path as every SDK operation, smudge honours the hydration intent recorded in state.db, and the pre-push barrier consults the publication ledger.
 Package wire connects the Git process boundary (internal/filter) to the engine (internal/app) so that the clean filter ingests through the same path as every SDK operation, smudge honours the hydration intent recorded in state.db, and the pre-push barrier consults the publication ledger.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.