replicate

Documentation

Overview

Package replicate implements pull-mode replication of a zonegit repository from a primary daemon to one or more secondaries.

The protocol is intentionally simple: it's a content-addressed Merkle-walk over plain HTTP. A secondary asks the primary "which branches do you have?", then for each branch where its local hash differs, asks "what objects do you have reachable from <root> that I don't have given I already have <known>?", then fetches each missing object by its content hash and finally moves the local ref to match.

Why HTTP and not gRPC: gRPC pulls in ~150 transitive modules and a protobuf compiler step. For the v0.8 demo, plain HTTP with content addressing is enough — every object body is identified by its SHA-256 hash so transport correctness is verifiable end-to-end with no extra schema. v0.9+ can add a streaming-gRPC backend behind the same Client interface if throughput becomes the constraint.

Endpoint layout:

GET  /v0/refs                  → JSON: {zones, branches}
POST /v0/objects/walk          → JSON: {missing: []hash}
GET  /v0/objects/<hex-hash>    → binary object body; X-Object-Kind header

The protocol is one-way: clients never push to primaries. Multi-master replication is a v0.9+ design problem deliberately out of scope here.

Index

• Constants
• type Branch
• type Client
  • func NewClient(primaryURL string, local *repo.Repo) *Client
  • func (c *Client) Pull(ctx context.Context) error
  • func (c *Client) Run(ctx context.Context)
  • func (c *Client) Stop()
• type RefsResponse
• type Server
  • func (s *Server) RegisterHandlers(mux *http.ServeMux)
• type WalkRequest
• type WalkResponse

Constants

const BasePath = "/v0"

BasePath is the URL prefix every replication endpoint sits under. Kept as a constant so client and server agree byte-for-byte.

Types

type Branch

type Branch struct {
	Zone string `json:"zone"`
	Name string `json:"name"`
	Hash string `json:"hash"` // hex-encoded
}

Branch identifies one (zone, name) pair plus the commit hash it points at. Exchanged in the /v0/refs response so the secondary can compute which branches have diverged.

type Client

type Client struct {
	PrimaryURL string
	Local      *repo.Repo
	Interval   time.Duration
	HTTP       *http.Client
	// contains filtered or unexported fields
}

Client is a secondary's pull-replication client. Construct one per (primary, local-repo) pair and call Run to start the polling loop.

One Pull() invocation fetches every diverged branch in one pass:

1. GET /v0/refs to learn the primary's current branch tips.
2. For each (zone, branch) where the local hash differs, request a Merkle walk to determine what objects are missing.
3. Fetch each missing object and store it locally.
4. Move local refs to match the primary atomically.

The local repo must be writable (not opened ReadOnly). The polling loop is single-threaded — one Pull at a time — which avoids ref race-conditions without needing a multi-writer protocol.

func NewClient

func NewClient(primaryURL string, local *repo.Repo) *Client

NewClient builds a Client with sensible defaults: 5s poll interval, 30s per-request HTTP timeout.

func (*Client) Pull

func (c *Client) Pull(ctx context.Context) error

Pull performs one full sync cycle: refs → missing-walk → fetch → ref advance. Safe to call directly (without Run) for tests or one-shot secondary catchups.

func (*Client) Run

func (c *Client) Run(ctx context.Context)

Run starts the background polling loop. Returns when Stop is called. Errors per cycle are logged but don't terminate the loop — transient network failures shouldn't take down a secondary.

func (*Client) Stop

func (c *Client) Stop()

Stop terminates the polling loop and waits for the goroutine to exit.

type RefsResponse

type RefsResponse struct {
	Zones    []string `json:"zones"`
	Branches []Branch `json:"branches"`
}

RefsResponse is the body returned by GET /v0/refs.

type Server

type Server struct {
	// SnapshotFn returns a fresh read-only Repo for the current state.
	// Used per request so secondaries see the latest commits the
	// writer has produced. Implementations typically wire this to
	// resolve.PollingSnapshotter or open a fresh handle on every call.
	SnapshotFn func() (*repo.Repo, error)
}

Server exposes a zonegit repo to secondaries over HTTP. It is stateless in the sense that every request opens against the current snapshot of the underlying repo; correctness comes from content-addressed object bodies, not from session state.

The repo handle must be opened read-only — Server never mutates.

func (*Server) RegisterHandlers

func (s *Server) RegisterHandlers(mux *http.ServeMux)

RegisterHandlers wires the protocol endpoints onto mux under BasePath. Callers can mount the same server alongside their existing /metrics endpoint by passing http.DefaultServeMux or any sub-mux.

type WalkRequest

type WalkRequest struct {
	Roots []string `json:"roots"` // hex
	Known []string `json:"known"` // hex
}

WalkRequest is the body POSTed to /v0/objects/walk.

Roots are the object hashes the secondary wants to reach (typically the primary's branch tips after diffing /v0/refs).

Known are object hashes the secondary already has in its local store. The server expands Known into its full reachable set, then walks Roots and returns whatever the secondary still needs.

type WalkResponse

type WalkResponse struct {
	Missing []string `json:"missing"` // hex
}

WalkResponse is the body returned by /v0/objects/walk. Hashes are in arbitrary order; clients can fetch them in any order since each object is content-addressable on its own.