lib

package
v0.0.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 23, 2026 License: MIT Imports: 35 Imported by: 0

Documentation

Overview

Content-Defined Chunking

Implementation of the GearCDC algorithm to re-use blocks when the contents changed slightly (data added or removed somewhere in the middle).

See https://joshleeb.com/posts/gear-hashing.html

Git-style glob pattern matching.

A minimal protobuf implementation that only supports what this project needs.

A revision snapshot represents a sorted list of all effective RevisionEntries for a given revision. It is created by reading all revisions from the given revision to the root revision, and then merging the revisions together.

A sorted, chunked, on-disk temporary storage of entries.

Index

Constants

View Source 
const (
	RawKeySize = 32
	SaltSize   = RawKeySize

	TotalCipherOverhead = nonceSize + 16
	EncryptedKeySize    = RawKeySize + TotalCipherOverhead
)
View Source 
const (
	// MaxBlockDataSize is the largest plaintext payload a block can carry.
	// It is chosen so that Padmé padding is a no-op at the maximum (i.e.
	// `Padme(MaxBlockDataSize) == MaxBlockDataSize`) and leaves enough room
	// for the encrypted header and the protobuf envelope to fit within
	// `MaxBlockSize`.
	MaxBlockDataSize           = MaxBlockSize - 128*1024
	UpdateHeadRevisionLockName = "head"
)
View Source 
const (
	EncryptionVersion uint16 = 1
	StorageVersion    uint16 = 1
)
View Source 
const (
	MaxBlockSize       = 8 * 1024 * 1024
	MaxControlFileSize = 1 * 1024 * 1024
)
View Source 
const BlockIdSize = 32
View Source 
const DefaultTempChunkSize = 4 * 1024 * 1024
View Source 
const MaxPathLen = 4096

MaxPathLen is the maximum allowed length (in bytes) for a `Path`. 4096 matches Linux PATH_MAX (including its terminating NUL) and is well above macOS (1024) and the Windows non-extended limit (260).

View Source 
const PathDelim = "/"
View Source 
const PathSeparator = string(os.PathSeparator)
View Source 
const RevisionMagic = "cling-revision"

RevisionMagic is the constant string stored as the first field of every marshalled `Revision`. It lets a disaster-recovery tool tell revision blocks apart from data blocks by decrypting each block and reading the first field as a string.

Variables

View Source 
var (
	ErrRootRevision = errors.New("root revision cannot be read")
	ErrHeadChanged  = Errorf("head changed during commit")
)
View Source 
var (
	ErrStorageNotFound      = Errorf("storage not found")
	ErrStorageAlreadyExists = Errorf("storage already exists")
	ErrBlockNotFound        = Errorf("block not found")
	ErrControlFileNotFound  = Errorf("control file not found")
	ErrLockNotFound         = Errorf("lock not found")
)
View Source 
var ErrCancel = Errorf("operation cancelled")
View Source 
var ErrDuplicateTempEntry = Errorf("duplicate entry")

Returned by `TempWriter.Add` and `TempWriter.CloseAndSort` when a writer that does not ignore duplicates sees the same entry twice.

View Source 
var ErrEmptyCommit = Errorf("empty commit")
View Source 
var ErrIsSymlink = errors.New("path is a symlink")
View Source 
var ErrLockAlreadyAcquired = errors.New("lock already acquired")
View Source 
var RepositoryConfigHeaderComment = strings.Trim(`
DO NOT DELETE OR MODIFY THIS FILE.

This file contains the configuration of your cling repository including
the master key information.
You need your passphrase to unlock the repository so this file in itself
is not enough to access your data. But without this file all your data is
lost. Forever.

So please back this file up. 
Copy it to a secure place (a password manager might be a good choice) or 
even print it out and keep it somewhere safe.
`, "\n ")

Functions

func AtomicWriteFile 

func AtomicWriteFile(fs FS, name string, perm fs.FileMode, data ...[]byte) error

Try to write the data to the target file in the most safe way possible:

  • Write the data to a temporary file.
  • fsync the temporary file.
  • Set the permissions of the temporary file.
  • Rename the temporary file to the target file.
  • fsync the parent directory of the target file.

In case of an error, the temporary file is deleted.

func AtomicWriteTempFilename 

func AtomicWriteTempFilename(name string) string

func BlockIdCompare 

func BlockIdCompare(a, b BlockId) int

func CheckHealth 

func CheckHealth(ctx context.Context, repository *Repository, tempFS FS, opts HealthCheckOptions) error

CheckHealth verifies the integrity of `repository`.

It always traverses the entire revision chain (head to root), checking that every revision can be read and that every revision's path entries are strictly sorted. Additional checks can be enabled via `opts`.

func CheckPassphraseStrength 

func CheckPassphraseStrength(phrase []byte) error

func ComparePathKey added in v0.0.4 

func ComparePathKey(a, b PathKey) int

func Compress 

func Compress(data []byte, target []byte) (n int, ok bool, err error)

Compress writes Deflate-compressed `data` into `target` and returns the number of bytes written. `ok` is false when the compressed output would not fit in `target`, which the caller treats as "not worth compressing".

func Decompress 

func Decompress(data []byte) ([]byte, error)

func Decrypt 

func Decrypt(ciphertext []byte, cipher cryptoCipher.AEAD, associatedData []byte, dst []byte) ([]byte, error)

dst - must be large enough to hold the plaintext (`len(ciphertext) - TotalCipherOverhead`).

func DecryptInPlace 

func DecryptInPlace(ciphertext []byte, cipher cryptoCipher.AEAD, associatedData []byte) ([]byte, error)

Re-uses the ciphertext buffer.

func Encrypt 

func Encrypt(plaintext []byte, cipher cryptoCipher.AEAD, associatedData []byte, dst []byte) ([]byte, error)

dst - must be large enough to hold the ciphertext, nonce, and cipher overhead 

('len(plaintext) + TotalCipherOverhead')

func EnhanceMetadata 

func EnhanceMetadata(md *PathMetadata, fileInfo fs.FileInfo)

func FormatRecoveryCode 

func FormatRecoveryCode(data []byte) string

If the data length is not divisible by 4 then the last block will be shortened.

func GlobMatch 

func GlobMatch(pattern GlobPattern, text []byte, isDir bool) bool

Match Git-style glob pattern according to https://git-scm.com/docs/gitignore#_pattern_format and https://github.com/git/git/blob/master/wildmatch.c

func IsAlnum 

func IsAlnum(c byte) bool

func IsAlpha 

func IsAlpha(c byte) bool

func IsAtomicWriteTempFile 

func IsAtomicWriteTempFile(name string) bool

func IsBlank 

func IsBlank(c byte) bool

func IsCntrl 

func IsCntrl(c byte) bool

func IsCompressible 

func IsCompressible(data []byte) bool

Calculate the "entropy" of the data using the Shannon entropy formula to decide whether it should be compressed. (https://en.wikipedia.org/wiki/Entropy_(information_theory)

We only look at the first `compressionCheckSize` bytes.

func IsDigit 

func IsDigit(c byte) bool

func IsGraph 

func IsGraph(c byte) bool

func IsLower 

func IsLower(c byte) bool

func IsPrint 

func IsPrint(c byte) bool

func IsPunct 

func IsPunct(c byte) bool

func IsSpace 

func IsSpace(c byte) bool

func IsUpper 

func IsUpper(c byte) bool

func IsXDigit 

func IsXDigit(c byte) bool

func NewCipher 

func NewCipher(key RawKey) (cryptoCipher.AEAD, error)

Create an XChaChaPoly1305 cipher from the given raw key.

func Padme 

func Padme(l uint64) uint64

Return the number of bytes to pad the given input size according to: https://lbarman.ch/blog/padme

func ParseRecoveryCode 

func ParseRecoveryCode(s string) ([]byte, error)

func PathCompare added in v0.0.4 

func PathCompare(a Path, aIsDir bool, b Path, bIsDir bool) int

Order two paths.

Entries are sorted by path alone. A directory therefore comes before its contents, because its path is a prefix of theirs, and no marker or special case is needed for that.

The only thing the directory bit decides is a file and a directory of the very same path, where the file comes first. That pair is not a filesystem state, it is how one revision expresses a path changing type.

func Rand 

func Rand(n int) ([]byte, error)

Rand returns n cryptographically random bytes from the system CSPRNG.

func RandStr 

func RandStr(n int) (string, error)

RandStr returns a string of n hex characters of entropy (n/2 random bytes hex-encoded).

func ReadFile 

func ReadFile(fs FS, name string) ([]byte, error)

func RevisionEntryPathFilter 

func RevisionEntryPathFilter(pathFilter PathFilter) func(e *RevisionEntry) bool

func RewriteRevisions added in v0.0.4 

func RewriteRevisions(
	ctx context.Context,
	repository *Repository,
	tempFS FS,
	monitor RevisionSnapshotMonitor,
	tempChunkSize int,
) error

Rewrite every revision in the current sort order, keeping the chain intact.

This is temporary, for repositories written before the sort order changed. `RevisionReader` rejects those outright, so the entries are read here from their blocks instead. Everything it needs is copied rather than opened up in the rest of the package, because it goes away with the next release.

Every revision id changes, since an id is the hash of the revision. The old blocks are left behind and `check --orphaned-blocks` will report them. Workspaces have to be pointed at the new head afterwards.

func Stringify 

func Stringify(v any) string

func SyncRepository 

func SyncRepository(
	ctx context.Context, src, dst Storage, tempFS FS, srcRevisionChain RevisionChain, opts RepositorySyncOptions,
) error

Sync new blocks from src to dst, then advance dst's head to src's. Both storages must share the exact same repository config. The dst head revision must be in the in the srcRevisionChain unless `opts.SkipHeadCheck` is true.

func TagLen 

func TagLen(field, wireType int) int

func ValidateControlFileName 

func ValidateControlFileName(name string) error

func ValidateStorageLockName 

func ValidateStorageLockName(name string) error

func VarintLen 

func VarintLen(v int64) int

func WalkDirIgnore 

func WalkDirIgnore(fs FS, dir string, f iofs.WalkDirFunc) error

Same as `fs.WalkDir`, but will respect all `.gitignore` and `.clingignore` files along the way.

func WriteFile 

func WriteFile(fs FS, name string, data []byte) error

func WriteRef 

func WriteRef(ctx context.Context, storage Storage, name string, revisionId RevisionId) error

func WriteToml 

func WriteToml(dst io.Writer, headerComment string, toml Toml) error

Sections and keys within sections are sorted alphabetically.

Types

type Argon2id 

type Argon2id struct {
	Time        uint32
	Memory      uint32
	Parallelism uint8
	Salt        Salt
}

func NewArgon2id 

func NewArgon2id(salt Salt, params Argon2idParams) Argon2id

func UnmarshalArgon2idConfig 

func UnmarshalArgon2idConfig(s string) (Argon2id, error)

Parse the PHC password format but we expect a strict format like this:

$argon2id$v=19$m=<memory>,t=<time>,p=<parallelism>$<salt>

Needs to at least meet OWASP recommendations of 12MB RAM, 3 iterations, 1 thread.

PHC format: https://github.com/P-H-C/phc-string-format/blob/master/phc-sf-spec.md

func (Argon2id) Marshal 

func (a Argon2id) Marshal() string

type Argon2idParams 

type Argon2idParams struct {
	Time        uint32
	Memory      uint32
	Parallelism uint8
}

The cost of deriving the user key, without the salt, which is generated when the repository is created.

func DefaultArgon2idParams 

func DefaultArgon2idParams() Argon2idParams

todo: measure on a phone or raspberry. The default cost: time=4, memory=128MiB, parallelism=2.

func NewArgon2idParams 

func NewArgon2idParams(time uint32, memory uint32, parallelism uint8) (Argon2idParams, error)

Higher is slower to derive and slower to attack, `memory` is in KiB.

The cost has to meet the OWASP recommendation of 12MiB memory, 3 iterations and 1 thread, and has to stay within what a client can realistically derive with.

func ParseArgon2idParams 

func ParseArgon2idParams(s string) (Argon2idParams, error)

Parse `m=<memory>,t=<time>,p=<parallelism>`, the parameter section of the PHC format.

func (Argon2idParams) Marshal 

func (p Argon2idParams) Marshal() string

type Assert 

type Assert struct {
	Any any
	// contains filtered or unexported fields
}

func NewAssert 

func NewAssert(tb testing.TB) Assert

func (Assert) AllFieldsSet added in v0.0.4 

func (a Assert) AllFieldsSet(v any, msg ...any)

Assert that nothing in `v` is left at its zero value, walking into nested structs and through pointers. A slice, map, or array is checked as a whole.

Use it where the test data has to exercise every field, so that a field added later cannot silently go untested.

func (Assert) Call 

func (a Assert) Call(expected MockCall, calls []MockCall, msg ...any)

Make sure at least one to the given function is found.

func (Assert) Calls 

func (a Assert) Calls(expected []MockCall, calls []MockCall, msg ...any)

func (Assert) Contains 

func (a Assert) Contains(haystack any, needle any, msg ...any)

func (Assert) Equal 

func (a Assert) Equal(expected, actual any, msg ...any)

func (Assert) Error 

func (a Assert) Error(err error, contains string, msg ...any)

func (Assert) ErrorIs 

func (a Assert) ErrorIs(err, target error, msg ...any)

func (Assert) Fields added in v0.0.4 

func (a Assert) Fields(expected []string, typ reflect.Type, msg ...any)

Assert the fields of `typ`, each rendered as `Name Type`. Pin the shape of a struct here when a change to it has to be noticed by this test.

func (Assert) Greater 

func (a Assert) Greater(x, y any, msg ...any)

func (Assert) Less 

func (a Assert) Less(x, y any, msg ...any)

func (Assert) Nil 

func (a Assert) Nil(v any, msg ...any)

func (Assert) NoError 

func (a Assert) NoError(err error, msg ...any)

func (Assert) NotEqual 

func (a Assert) NotEqual(expected, actual any, msg ...any)

func (Assert) NotNil 

func (a Assert) NotNil(v any, msg ...any)

type Block 

type Block struct {
	EncryptedHeader []byte
	EncryptedData   []byte
}

func UnmarshallBlock 

func UnmarshallBlock(r *ProtobufReader) (*Block, error)

func (*Block) Marshall 

func (o *Block) Marshall(w ProtobufWriter) error

func (*Block) MarshallSize 

func (o *Block) MarshallSize() int

func (*Block) Validate 

func (o *Block) Validate() error

type BlockBuf 

type BlockBuf struct {
	// contains filtered or unexported fields
}

func NewBlockBuf 

func NewBlockBuf() BlockBuf

func (BlockBuf) Bytes 

func (b BlockBuf) Bytes() []byte

Bytes exposes the underlying fixed-size slice so callers (e.g. transport layers) can read directly into it without allocating.

func (BlockBuf) Read 

func (b BlockBuf) Read(src io.Reader) ([]byte, error)

Read up to `MaxBlockSize` bytes from `src` into the buffer and return the populated sub-slice. If `src` has more than `MaxBlockSize` bytes available, return an error rather than silently truncating.

type BlockHeader 

type BlockHeader struct {
	Version           uint32
	Compression       Compression
	Dek               RawKey
	EncryptedDataSize uint32
}

func UnmarshallBlockHeader 

func UnmarshallBlockHeader(r *ProtobufReader) (*BlockHeader, error)

func (*BlockHeader) Marshall 

func (o *BlockHeader) Marshall(w ProtobufWriter) error

func (*BlockHeader) MarshallSize 

func (o *BlockHeader) MarshallSize() int

func (*BlockHeader) Validate 

func (o *BlockHeader) Validate() error

type BlockId 

type BlockId Sha256Hmac

func NewBlockIdFromString 

func NewBlockIdFromString(s string) (BlockId, error)

Counterpart of `BlockId.String()`. Parse a hex-encoded BlockId.

func (BlockId) String 

func (id BlockId) String() string

type Commit 

type Commit struct {
	BaseRevision RevisionId
	// contains filtered or unexported fields
}

func NewCommit 

func NewCommit(ctx context.Context, repository *Repository, tmpFS FS) (*Commit, error)

func (*Commit) Add 

func (c *Commit) Add(entry *RevisionEntry) error

func (*Commit) Commit 

func (c *Commit) Commit(ctx context.Context, info *CommitInfo) (RevisionId, error)

Return `ErrHeadChanged` if the head has changed during the commit. Return `ErrEmptyCommit` if the commit is empty. A `Commit` is single-use: any call after the first closes it, so further `Add` / `Commit` calls return "commit is closed".

func (*Commit) EnsureDirExists 

func (c *Commit) EnsureDirExists(path Path, exists func(PathKey) (bool, error)) error

Make sure that the directory `path` exists once the commit is written.

`exists` answers whether the base revision holds an entry. Missing parent directories are created with `NewEmptyDirPathMetadata` metadata.

type CommitInfo 

type CommitInfo struct {
	Author  string
	Message string
}

type Compression 

type Compression uint32
const (
	CompressionNone    Compression = 0
	CompressionDeflate Compression = 1
)

type ControlFileSection 

type ControlFileSection string
const (
	ControlFileSectionRefs     ControlFileSection = "refs"
	ControlFileSectionSecurity ControlFileSection = "security"
	ControlFileSectionConf     ControlFileSection = "conf"
)

type DisplayOrderReader added in v0.0.4 

type DisplayOrderReader struct {
	// contains filtered or unexported fields
}

Read entries in the order they should be printed, where a directory comes directly before its contents.

Entries are stored ordered by path alone, so a sibling whose name continues below `/` falls between a directory and its contents: `sub`, `sub.txt`, `sub/a.txt`. Holding a directory back until its contents are reached gives `sub.txt`, `sub`, `sub/a.txt`, which is the same order as comparing a directory as its path with a trailing `/`.

func NewDisplayOrderReader added in v0.0.4 

func NewDisplayOrderReader(source func(BlockBuf) (*RevisionEntry, error)) *DisplayOrderReader

func (*DisplayOrderReader) Read added in v0.0.4 

func (dr *DisplayOrderReader) Read(buf BlockBuf) (*RevisionEntry, error)

type EncryptedKey 

type EncryptedKey [EncryptedKeySize]byte

type EnhancedStat_t 

type EnhancedStat_t struct {
	CTimeSec  int64
	CTimeNSec int32
	Inode     uint64
}

func EnhancedStat 

func EnhancedStat(fileInfo fs.FileInfo) (*EnhancedStat_t, error)

type ExtendedGlobPattern 

type ExtendedGlobPattern struct {
	GlobPattern
	// If a negation pattern is detected by the "!" prefix, this is set to true
	// AND the leading "!" is removed from the pattern.
	IsNegate bool
	BaseDir  string
}

func NewExtendedGlobPattern 

func NewExtendedGlobPattern(pattern string, baseDir string) ExtendedGlobPattern

type ExtendedGlobPatterns 

type ExtendedGlobPatterns []ExtendedGlobPattern

func CollectIgnorePatterns 

func CollectIgnorePatterns(fs FS, dir string) (ExtendedGlobPatterns, error)

Walk `dir` and collect every ignore pattern from the `.gitignore` and `.clingignore` files found along the way, respecting nested ignores (an ignored directory's contents are not visited).

func ParseGlobIgnoreFile 

func ParseGlobIgnoreFile(dir string, patterns []string) ExtendedGlobPatterns

Parse a `.gitignore` or `.clingignore` file.

func (ExtendedGlobPatterns) Match 

func (i ExtendedGlobPatterns) Match(path string, isDir bool) bool

type FS 

type FS interface {
	// The file is always fully overwritten.
	OpenWrite(name string) (io.WriteCloser, error)
	// Return `fs.ErrExist` if the file already exists.
	OpenWriteExcl(name string) (io.WriteCloser, error)
	FSync(file io.WriteCloser) error
	FSyncDir(path string) error
	OpenRead(name string) (io.ReadCloser, error)
	Chmod(name string, mode fs.FileMode) error
	Chmtime(name string, mtime time.Time) error
	Chown(name string, uid int, gid int) error
	Stat(name string) (fs.FileInfo, error)
	Symlink(target string, name string) error
	ReadLink(name string) (string, error)
	ReadDir(name string) ([]fs.DirEntry, error)
	Mkdir(name string) error
	MkdirAll(path string) error
	Remove(name string) error
	RemoveAll(path string) error
	Rename(oldpath, newpath string) error
	// Create a sub directory, including any missing parents, and return a `FS` for it.
	MkSub(path string) (FS, error)
	// Return a `FS` for the sub directory. Return `fs.ErrNotExist` if the directory does not exist.
	Sub(path string) (FS, error)
	// `fn` is called with a path relative to the root of the FS.
	WalkDir(path string, fn fs.WalkDirFunc) error
	String() string
	// Create the lock file if it does not exist and places a lock on it.
	// Use this to synchronize inter-process access to any resource.
	// The file is not deleted when the lock is released.
	Lock(ctx context.Context, path string) (unlock func() error, err error)
}

A file system abstraction that only provides what is actually needed.

About file modes: `OpenWrite*` and `Mkdir*` don't take a `fs.FileMode` argument, because the effective mode is determined by the umask. So we are better off setting the mode explicitly. `RealFS` uses sensible defaults (0o600 and 0o700) in `OpenWrite*` and `Mkdir*`.

type FileLock 

type FileLock struct {
	// contains filtered or unexported fields
}

func NewLock 

func NewLock(path string) *FileLock

func (*FileLock) Lock 

func (l *FileLock) Lock(ctx context.Context) error

Block until the lock can be acquired or the context is cancelled.

func (*FileLock) TryLock 

func (l *FileLock) TryLock() (bool, error)

func (*FileLock) Unlock 

func (l *FileLock) Unlock() error

type FileMode 

type FileMode uint32
const (
	FileModeOtherExec  FileMode = 0x001
	FileModeOtherWrite FileMode = 0x002
	FileModeOtherRead  FileMode = 0x004
	FileModeGroupExec  FileMode = 0x008
	FileModeGroupWrite FileMode = 0x010
	FileModeGroupRead  FileMode = 0x020
	FileModeOwnerExec  FileMode = 0x040
	FileModeOwnerWrite FileMode = 0x080
	FileModeOwnerRead  FileMode = 0x100
	FileModePerm       FileMode = 0x1FF
	FileModeDir        FileMode = 0x400
	FileModeSymlink    FileMode = 0x800
	FileModeSetUid     FileMode = 0x1000
	FileModeSetGid     FileMode = 0x2000
	FileModeSticky     FileMode = 0x4000
)
const FileModeType FileMode = FileModeDir | FileModeSymlink

FileModeType is the set of bits that indicate a non-regular file (directory or symlink).

func NewFileMode 

func NewFileMode(fm fs.FileMode) FileMode

func (FileMode) AsFsFileMode 

func (m FileMode) AsFsFileMode() fs.FileMode

func (FileMode) IsDir 

func (m FileMode) IsDir() bool

func (FileMode) IsRegular 

func (m FileMode) IsRegular() bool

func (FileMode) IsSetGID 

func (m FileMode) IsSetGID() bool

func (FileMode) IsSetUID 

func (m FileMode) IsSetUID() bool

func (FileMode) IsSticky 

func (m FileMode) IsSticky() bool
func (m FileMode) IsSymlink() bool

func (FileMode) Perm 

func (m FileMode) Perm() FileMode

func (FileMode) ShortString 

func (m FileMode) ShortString() string

Return a string in the style of `ls -l`.

func (FileMode) String 

func (m FileMode) String() string

type FileStorage 

type FileStorage struct {
	FS      FS
	Purpose StoragePurpose
}

func NewFileStorage 

func NewFileStorage(fs FS, purpose StoragePurpose) (*FileStorage, error)

func (*FileStorage) DeleteControlFile 

func (s *FileStorage) DeleteControlFile(_ context.Context, section ControlFileSection, name string) error

func (*FileStorage) ForceUnlock 

func (s *FileStorage) ForceUnlock(_ context.Context, name string) error

ForceUnlock removes the lock file. Any process still holding the orphaned flock keeps the kernel-level lock on its open fd until it exits, but a new acquirer opens a fresh file (different inode) and gets its own flock cleanly.

func (*FileStorage) HasBlock 

func (s *FileStorage) HasBlock(_ context.Context, blockId BlockId) (bool, error)

func (*FileStorage) HasControlFile 

func (s *FileStorage) HasControlFile(_ context.Context, section ControlFileSection, name string) (bool, error)

func (*FileStorage) Init 

func (s *FileStorage) Init(_ context.Context, config Toml, headerComment string) error

func (*FileStorage) Lock 

func (s *FileStorage) Lock(ctx context.Context, name string) (func() error, error)

func (*FileStorage) Open 

func (s *FileStorage) Open(_ context.Context) (Toml, error)

func (*FileStorage) ReadBlock 

func (s *FileStorage) ReadBlock(_ context.Context, blockId BlockId, buf BlockBuf) ([]byte, error)

Return `ErrBlockNotFound` if the block does not exist.

func (*FileStorage) ReadBlockIds 

func (s *FileStorage) ReadBlockIds(ctx context.Context, yield func(BlockId) bool) error

func (*FileStorage) ReadControlFile 

func (s *FileStorage) ReadControlFile(_ context.Context, section ControlFileSection, name string) ([]byte, error)

func (*FileStorage) WriteBlock 

func (s *FileStorage) WriteBlock(_ context.Context, blockId BlockId, data []byte) (bool, error)

func (*FileStorage) WriteControlFile 

func (s *FileStorage) WriteControlFile(_ context.Context, section ControlFileSection, name string, data []byte) error

type GearCDC 

type GearCDC struct {
	// contains filtered or unexported fields
}

func NewGearCDC 

func NewGearCDC(r io.Reader, mask uint64, minSize, maxSize int, table GearCDCTable) *GearCDC

Initialize the GearCDC.

func NewGearCDCWithDefaults 

func NewGearCDCWithDefaults(r io.Reader, table GearCDCTable) *GearCDC

func (*GearCDC) Read 

func (g *GearCDC) Read() ([]byte, error)

Read from the underlying reader until we reach a block boundary. If we reach the end of the underlying reader, return `io.EOF`.

Return a slice of `dst` with the read bytes.

type GearCDCTable 

type GearCDCTable [256]uint64

func NewGearCDCTable 

func NewGearCDCTable(key RawKey) (GearCDCTable, error)

type GlobPattern 

type GlobPattern []byte

func PrepareGlobPattern 

func PrepareGlobPattern(pattern string) GlobPattern

Take the given pattern and trim trailing spaces.

type HealthCheckMonitor 

type HealthCheckMonitor interface {
	OnRevisionStart(revisionId RevisionId)
	OnRevisionEntry(entry *RevisionEntry)
	OnBlockVerified(blockId BlockId, length int)
	OnOrphanedBlock(blockId BlockId)
}

type HealthCheckOptions 

type HealthCheckOptions struct {
	Monitor HealthCheckMonitor
	// Read and decrypt every block referenced by any revision and check that no
	// two block headers were encrypted with the same nonce.
	CheckBlocks bool
	// Report every block in storage that is not referenced by any revision.
	CheckOrphanedBlocks bool
}

type Heap added in v0.0.4 

type Heap[T any] struct {
	// contains filtered or unexported fields
}

A binary min-heap ordered by `compare`, which returns a negative number when `a` comes before `b`, like `slices.SortFunc`.

Go's `container/heap` would box every element into `any` and needs a five-method interface implementation. It also sifts down with two comparisons per level, which is the wrong trade when comparing is expensive.

func NewHeap added in v0.0.4 

func NewHeap[T any](compare func(a, b T) int, capacity int) *Heap[T]

func (*Heap[T]) Len added in v0.0.4 

func (h *Heap[T]) Len() int

func (*Heap[T]) Peek added in v0.0.4 

func (h *Heap[T]) Peek() T

The smallest element. The heap must not be empty.

func (*Heap[T]) Pop added in v0.0.4 

func (h *Heap[T]) Pop() T

Remove and return the smallest element. The heap must not be empty.

The root is refilled by moving the gap all the way down to a leaf and only then bubbling the last element back up (Floyd's bounce). That costs one comparison per level instead of two, and the bubble-up nearly always stops at once because the element came from the bottom to begin with. It pays for itself whenever comparing costs more than moving.

func (*Heap[T]) Push added in v0.0.4 

func (h *Heap[T]) Push(v T)

type LockExistsError 

type LockExistsError struct {
	Name      string
	Owner     string
	Host      string
	Pid       int
	CreatedAt time.Time
}

LockExistsError is returned by `Storage.Lock` when the lock is already held. Fields describe the current holder so a user can decide whether to wait or force-release.

func (*LockExistsError) Error 

func (e *LockExistsError) Error() string

type Marshallable 

type Marshallable interface {
	Marshall(ProtobufWriter) error
	MarshallSize() int
}

Marshallable is the proto-message contract: serialize to a writer and report the size of what was written.

type MemoryFS 

type MemoryFS struct {
	// contains filtered or unexported fields
}

func NewMemoryFS 

func NewMemoryFS(maxMemory int64) *MemoryFS

func (*MemoryFS) Chmod 

func (f *MemoryFS) Chmod(name string, mode fs.FileMode) error

func (*MemoryFS) Chmtime 

func (f *MemoryFS) Chmtime(name string, mtime time.Time) error

`Chmtime` operates on the path's final component without following. Setting a symlink's own mtime is allowed (matches `lutimes` semantics).

func (*MemoryFS) Chown 

func (f *MemoryFS) Chown(name string, uid int, gid int) error

func (*MemoryFS) FSync 

func (f *MemoryFS) FSync(file io.WriteCloser) error

func (*MemoryFS) FSyncDir 

func (f *MemoryFS) FSyncDir(path string) error

func (*MemoryFS) Lock 

func (f *MemoryFS) Lock(ctx context.Context, path string) (func() error, error)

Lock is cancel-aware: a size-1 channel per path is the mutex (send to acquire, receive to release), so a blocked acquire can lose to ctx.Done().

func (*MemoryFS) MkSub 

func (f *MemoryFS) MkSub(path string) (FS, error)

func (*MemoryFS) Mkdir 

func (f *MemoryFS) Mkdir(name string) error

func (*MemoryFS) MkdirAll 

func (f *MemoryFS) MkdirAll(path string) error

func (*MemoryFS) OpenRead 

func (f *MemoryFS) OpenRead(name string) (io.ReadCloser, error)

func (*MemoryFS) OpenWrite 

func (f *MemoryFS) OpenWrite(name string) (io.WriteCloser, error)

func (*MemoryFS) OpenWriteExcl 

func (f *MemoryFS) OpenWriteExcl(name string) (io.WriteCloser, error)

func (*MemoryFS) ReadDir 

func (f *MemoryFS) ReadDir(name string) ([]fs.DirEntry, error)
func (f *MemoryFS) ReadLink(name string) (string, error)

func (*MemoryFS) Remove 

func (f *MemoryFS) Remove(name string) error

func (*MemoryFS) RemoveAll 

func (f *MemoryFS) RemoveAll(path string) error

func (*MemoryFS) Rename 

func (f *MemoryFS) Rename(oldpath, newpath string) error

func (*MemoryFS) Stat 

func (f *MemoryFS) Stat(name string) (fs.FileInfo, error)

func (*MemoryFS) String 

func (f *MemoryFS) String() string

func (*MemoryFS) Sub 

func (f *MemoryFS) Sub(path string) (FS, error)
func (f *MemoryFS) Symlink(target string, name string) error

func (*MemoryFS) WalkDir 

func (f *MemoryFS) WalkDir(path string, fn fs.WalkDirFunc) error

type MockCall 

type MockCall struct {
	Name string
	Args []any
}

func NewMockCall 

func NewMockCall(name string, args ...any) MockCall

func (MockCall) Equal 

func (m MockCall) Equal(other MockCall) bool

func (MockCall) String 

func (m MockCall) String() string

type Path 

type Path struct {
	// contains filtered or unexported fields
}

func NewPath 

func NewPath(path string) (Path, error)

func NewPathUnchecked added in v0.0.4 

func NewPathUnchecked(path string) Path

Wrap a string that is already known to be a valid path, such as one derived from another `Path`. Use `NewPath` for anything coming from outside.

func (Path) Base 

func (p Path) Base() Path

func (Path) Depth 

func (p Path) Depth() int

The number of path segments. The empty path has depth 0.

func (Path) Dir 

func (p Path) Dir() Path

func (Path) IsEmpty 

func (p Path) IsEmpty() bool

func (Path) IsRelativeTo 

func (p Path) IsRelativeTo(base Path) bool

func (Path) Join 

func (p Path) Join(other Path) Path

func (Path) Len 

func (p Path) Len() int

func (Path) String 

func (p Path) String() string

func (Path) TrimBase 

func (p Path) TrimBase(base Path) (Path, bool)

Trim the base path from the beginning of the path. Return the trimmed path and a boolean indicating whether the path was trimmed.

type PathExclusionFilter 

type PathExclusionFilter struct {
	Excludes ExtendedGlobPatterns
}

A PathFilter that can exclude paths. A path is excluded if it matches any of the exclude patterns and none of the include patterns. So the include patterns are only used to override exclude patterns.

A nil `*PathExclusionFilter` means no filter and is safe to call: it keeps every path. Keep it nil rather than empty, so that the code asking whether a filter is in effect at all can tell the difference.

func NewPathExclusionFilter 

func NewPathExclusionFilter(excludes []string) *PathExclusionFilter

Parse the exclude and include patterns and create a PathFilter.

func (*PathExclusionFilter) Include 

func (pef *PathExclusionFilter) Include(p Path, isDir bool) bool

type PathFilter 

type PathFilter interface {
	Include(p Path, isDir bool) bool
}

type PathInclusionFilter 

type PathInclusionFilter struct {
	Includes ExtendedGlobPatterns
}

A PathFilter that keeps only the paths matching one of the include patterns.

A nil `*PathInclusionFilter` means no filter and is safe to call: it keeps every path. An empty one is not the same thing, it matches nothing and so drops every path.

func NewPathInclusionFilter 

func NewPathInclusionFilter(includes []string) *PathInclusionFilter

func (*PathInclusionFilter) Include 

func (pif *PathInclusionFilter) Include(p Path, isDir bool) bool

type PathKey added in v0.0.4 

type PathKey struct {
	Path  Path
	IsDir bool
}

A path together with its directory bit. The two identify an entry, and the struct is comparable, so it can key a map without building a string.

type PathMetadata 

type PathMetadata struct {
	FileMode      FileMode
	Mtime         Timestamp
	Size          int64
	FileHash      Sha256
	BlockIds      []BlockId
	SymLinkTarget *Path
	Uid           *uint32
	Gid           *uint32
	Birthtime     *Timestamp
}

func NewEmptyDirPathMetadata 

func NewEmptyDirPathMetadata(mtime time.Time) PathMetadata

NewEmptyDirPathMetadata returns a PathMetadata representing a directory created at the given time. UID/GID are left unset; Birthtime is set to mtime.

func NewPathMetadataFromFileInfo 

func NewPathMetadataFromFileInfo(fileInfo fs.FileInfo, fileHash Sha256, blockIds []BlockId) PathMetadata

NewPathMetadataFromFileInfo returns a PathMetadata populated from the given FileInfo. Platform-specific fields (UID/GID/Birthtime) are filled in via EnhanceMetadata.

Symlinks carry only the symlink bit, their own `Mtime`, and a `SymLinkTarget` set by the caller. Perm bits, owner, group, size, and birthtime are blanked: the FS layer refuses to chmod/chown a symlink, and the link's reported size (target string length) is just noise once the target is stored explicitly.

Panics if `fileHash` is non-zero or `blockIds` is non-nil for a directory or symlink: that's a sign the caller mis-typed the entry.

func UnmarshallPathMetadata 

func UnmarshallPathMetadata(r *ProtobufReader) (*PathMetadata, error)

func (*PathMetadata) HasBirthtime 

func (p *PathMetadata) HasBirthtime() bool

func (*PathMetadata) HasGID 

func (p *PathMetadata) HasGID() bool

func (*PathMetadata) HasUID 

func (p *PathMetadata) HasUID() bool

func (*PathMetadata) IsEqualRestorableAttributes 

func (p *PathMetadata) IsEqualRestorableAttributes(other PathMetadata, flags RestorableMetadataFlag) bool

Compare all attributes that can be restored like `FileMode`, `Size`, `FileHash` etc. `Birthtime` is not compared because it cannot be restored. `BlockIds` are not compared because they should be the same if the `FileHash` is the same.

func (*PathMetadata) MTime 

func (p *PathMetadata) MTime() time.Time

func (*PathMetadata) Marshall 

func (o *PathMetadata) Marshall(w ProtobufWriter) error

func (*PathMetadata) MarshallSize 

func (o *PathMetadata) MarshallSize() int
func (p *PathMetadata) SymLink() (Path, bool)

The target of a symlink, and whether this is one.

Panic if the mode says this is a symlink but `SymLinkTarget` is empty.

func (*PathMetadata) Validate 

func (o *PathMetadata) Validate() error

type ProtobufBytesWriter 

type ProtobufBytesWriter struct {
	// contains filtered or unexported fields
}

func NewProtobufWriter 

func NewProtobufWriter(out []byte) *ProtobufBytesWriter

`out` must be at least `MarshallSize()` bytes for what's being written.

The Write* methods return an error when the buffer would overflow. Errors are not recoverable: by the time a write fails, the writer may have advanced past the tag (and possibly the length prefix) of the failing field. Callers must discard `Bytes()` on any Write* error.

func (*ProtobufBytesWriter) Bytes 

func (w *ProtobufBytesWriter) Bytes() []byte

func (*ProtobufBytesWriter) WriteBytes 

func (w *ProtobufBytesWriter) WriteBytes(field int, v []byte) error

func (*ProtobufBytesWriter) WriteMessage 

func (w *ProtobufBytesWriter) WriteMessage(field int, marshall func(ProtobufWriter) error) error

func (*ProtobufBytesWriter) WriteTag 

func (w *ProtobufBytesWriter) WriteTag(field, wireType int) error

func (*ProtobufBytesWriter) WriteUint64 

func (w *ProtobufBytesWriter) WriteUint64(field int, v uint64) error

func (*ProtobufBytesWriter) WriteVarint 

func (w *ProtobufBytesWriter) WriteVarint(v_ int64) error

type ProtobufReader 

type ProtobufReader struct {
	// contains filtered or unexported fields
}

ProtobufReader decodes a protobuf wire-format byte stream WITHOUT copying. `ReadBytes` returns a sub-slice that aliases the underlying buffer, and any `bytes` field assigned directly from `ReadBytes` (i.e. a generated field whose Go type is `[]byte`, not a length-checked array conversion like `RawKey` / `BlockId` / `Sha256`) is also a sub-slice.

Callers must not mutate or reuse the input buffer while any reference returned by the reader, or any object unmarshalled from it, is still live.

func NewProtobufReader 

func NewProtobufReader(in []byte) *ProtobufReader

func (*ProtobufReader) AtEnd 

func (r *ProtobufReader) AtEnd() bool

func (*ProtobufReader) ReadBytes 

func (r *ProtobufReader) ReadBytes() ([]byte, error)

ReadBytes returns a sub-slice of the reader's input buffer (no copy). See the doc on `ProtobufReader` for the aliasing contract.

func (*ProtobufReader) ReadTag 

func (r *ProtobufReader) ReadTag() (int, int, error)

func (*ProtobufReader) ReadUint32 

func (r *ProtobufReader) ReadUint32() (uint32, error)

func (*ProtobufReader) ReadUint64 

func (r *ProtobufReader) ReadUint64() (uint64, error)

func (*ProtobufReader) ReadVarint 

func (r *ProtobufReader) ReadVarint() (int64, error)

func (*ProtobufReader) Skip 

func (r *ProtobufReader) Skip(wireType int) error

Skip consumes the value of an unknown field given its wire type. Only the wire types format.proto uses (0 = varint, 2 = length-delimited) are supported; anything else returns an error.

type ProtobufSizeWriter 

type ProtobufSizeWriter struct {
	// contains filtered or unexported fields
}

func NewProtobufSizeWriter 

func NewProtobufSizeWriter() *ProtobufSizeWriter

func (*ProtobufSizeWriter) Size 

func (w *ProtobufSizeWriter) Size() int

func (*ProtobufSizeWriter) WriteBytes 

func (w *ProtobufSizeWriter) WriteBytes(field int, v []byte) error

func (*ProtobufSizeWriter) WriteMessage 

func (w *ProtobufSizeWriter) WriteMessage(field int, marshall func(ProtobufWriter) error) error

func (*ProtobufSizeWriter) WriteTag 

func (w *ProtobufSizeWriter) WriteTag(field, wireType int) error

func (*ProtobufSizeWriter) WriteUint64 

func (w *ProtobufSizeWriter) WriteUint64(field int, v uint64) error

func (*ProtobufSizeWriter) WriteVarint 

func (w *ProtobufSizeWriter) WriteVarint(v int64) error

type ProtobufWriter 

type ProtobufWriter interface {
	WriteTag(field, wireType int) error
	WriteVarint(v int64) error
	WriteUint64(field int, v uint64) error
	WriteBytes(field int, v []byte) error
	WriteMessage(field int, marshall func(ProtobufWriter) error) error
}

type RawKey 

type RawKey [RawKeySize]byte

func DeriveUserKey 

func DeriveUserKey(passphrase []byte, argon2id Argon2id) (RawKey, error)

Derive the user's UserKey from the given passphrase using Argon2id.

func NewRawKey 

func NewRawKey() (RawKey, error)

type RealFS 

type RealFS struct {
	BasePath string
}

func NewRealFS 

func NewRealFS(basePath string) *RealFS

func (*RealFS) Chmod 

func (f *RealFS) Chmod(name string, mode fs.FileMode) error

func (*RealFS) Chmtime 

func (f *RealFS) Chmtime(name string, mtime time.Time) error

func (*RealFS) Chown 

func (f *RealFS) Chown(name string, uid int, gid int) error

func (*RealFS) FSync 

func (f *RealFS) FSync(file io.WriteCloser) error

func (*RealFS) FSyncDir 

func (f *RealFS) FSyncDir(path string) error

func (*RealFS) Lock 

func (f *RealFS) Lock(ctx context.Context, path string) (unlock func() error, err error)

func (*RealFS) MkSub 

func (f *RealFS) MkSub(path string) (FS, error)

func (*RealFS) Mkdir 

func (f *RealFS) Mkdir(name string) error

func (*RealFS) MkdirAll 

func (f *RealFS) MkdirAll(path string) error

func (*RealFS) OpenRead 

func (f *RealFS) OpenRead(name string) (io.ReadCloser, error)

func (*RealFS) OpenWrite 

func (f *RealFS) OpenWrite(name string) (io.WriteCloser, error)

func (*RealFS) OpenWriteExcl 

func (f *RealFS) OpenWriteExcl(name string) (io.WriteCloser, error)

func (*RealFS) ReadDir 

func (f *RealFS) ReadDir(name string) ([]fs.DirEntry, error)
func (f *RealFS) ReadLink(name string) (string, error)

func (*RealFS) Remove 

func (f *RealFS) Remove(name string) error

func (*RealFS) RemoveAll 

func (f *RealFS) RemoveAll(path string) error

func (*RealFS) Rename 

func (f *RealFS) Rename(oldpath, newpath string) error

func (*RealFS) Stat 

func (f *RealFS) Stat(name string) (fs.FileInfo, error)

func (*RealFS) String 

func (f *RealFS) String() string

func (*RealFS) Sub 

func (f *RealFS) Sub(path string) (FS, error)
func (f *RealFS) Symlink(target string, name string) error

func (*RealFS) WalkDir 

func (f *RealFS) WalkDir(path string, fn fs.WalkDirFunc) error

type Repository 

type Repository struct {
	// contains filtered or unexported fields
}

func InitNewRepository 

func InitNewRepository(
	ctx context.Context,
	storage Storage,
	passphrase []byte,
	kdf Argon2idParams,
) (*Repository, error)

func OpenRepository 

func OpenRepository(ctx context.Context, storage Storage, passphrase []byte) (*Repository, error)

func (*Repository) Close 

func (r *Repository) Close() error

Close wipes the repository's key material. The instance must not be used afterwards.

func (*Repository) GearCDCTable 

func (r *Repository) GearCDCTable() GearCDCTable

func (*Repository) Head 

func (r *Repository) Head(ctx context.Context) (RevisionId, error)

func (*Repository) ReadBlock 

func (r *Repository) ReadBlock(ctx context.Context, blockId BlockId, buf BlockBuf) ([]byte, error)

func (*Repository) ReadRevision 

func (r *Repository) ReadRevision(ctx context.Context, revisionId RevisionId, buf BlockBuf) (Revision, error)

Return `ErrRootRevision` if revisionId is the root revisionId.

func (*Repository) WriteBlock 

func (r *Repository) WriteBlock(
	ctx context.Context,
	data []byte,
	buf BlockBuf,
) (blockId BlockId, dataBytesWritten *int, err error)

WriteBlock stores `data` as an encrypted, padded, optionally-compressed block and returns its id. If `dataBytesWritten` is nil the block already existed. Otherwise, it is the payload size after compression (if any). Padding obfuscates the block size (Padmé: https://lbarman.ch/blog/padme).

func (*Repository) WriteRevision 

func (r *Repository) WriteRevision(ctx context.Context, revision *Revision) (RevisionId, error)

Write a revision and set it as the current HEAD. A revision can only reference the current head as their parent. Return `ErrHeadChanged` if the head has changed during the commit.

type RepositorySyncMonitor 

type RepositorySyncMonitor interface {
	OnSrcBlockIdsRead(blocksTotal int)
	OnDstBlockIdsRead(blocksTotal int)
	OnBeforeCopy(srcBlocks, dstBlocks int)
	OnCopyBlock(blockId BlockId, existed bool, length int)
	OnBeforeUpdateDstHead(newHead RevisionId)
}

type RepositorySyncOptions 

type RepositorySyncOptions struct {
	Monitor       RepositorySyncMonitor
	Workers       int
	SkipHeadCheck bool
}

type RepositoryView added in v0.0.5 

type RepositoryView struct {
	Repository *Repository
	// contains filtered or unexported fields
}

A repository seen from one of its directories.

Every path that crosses the view, entry paths and symlink targets alike, is relative to that directory. Entries outside of it, and symlinks whose target points outside of it, do not exist as far as callers are concerned ("hidden link"). What is hidden is kept privately so that a commit through the view leaves the repository consistent. An empty prefix is a view of the whole repository.

func NewRepositoryView added in v0.0.5 

func NewRepositoryView(repository *Repository, prefix Path) *RepositoryView

func (*RepositoryView) NewCommit added in v0.0.5 

func (v *RepositoryView) NewCommit(ctx context.Context, tmpFS FS, base *ViewSnapshot) (*ViewCommit, error)

Start a commit on top of `base`, which must be a snapshot of the repository head.

The commit makes sure the view's prefix exists in the repository once it is committed.

func (*RepositoryView) NewRevisionReader added in v0.0.5 

func (v *RepositoryView) NewRevisionReader(revision *Revision) *ViewRevisionReader

func (*RepositoryView) NewSnapshot added in v0.0.5 

func (v *RepositoryView) NewSnapshot(
	ctx context.Context,
	revisionId RevisionId,
	tmpFS FS,
	mon RevisionSnapshotMonitor,
) (*ViewSnapshot, error)

func (*RepositoryView) Sub added in v0.0.5 

func (v *RepositoryView) Sub(path Path) *RepositoryView

The view of the directory `path` inside this view.

type RestorableMetadataFlag 

type RestorableMetadataFlag uint8
const (
	// This includes `FileModePerm`, `FileModeSetUid`, `FileModeSetGid`, `FileModeSticky` but
	// not `FileModeDir` or `FileModeSymlink`, because the latter indicates a fundamental change.
	RestorableMetadataMode      RestorableMetadataFlag = 1
	RestorableMetadataMTime     RestorableMetadataFlag = 2
	RestorableMetadataOwnership RestorableMetadataFlag = 4
	RestorableMetadataAll       RestorableMetadataFlag = RestorableMetadataMode | RestorableMetadataMTime | RestorableMetadataOwnership
)

type Revision 

type Revision struct {
	Magic            string
	Timestamp        Timestamp
	ParentRevisionId RevisionId
	Message          *string
	Author           *string
	BlockIds         []BlockId
}

func UnmarshallRevision 

func UnmarshallRevision(r *ProtobufReader) (*Revision, error)

func (*Revision) Marshall 

func (o *Revision) Marshall(w ProtobufWriter) error

func (*Revision) MarshallSize 

func (o *Revision) MarshallSize() int

func (*Revision) Validate 

func (o *Revision) Validate() error

type RevisionChain 

type RevisionChain []RevisionId

RevisionChain is a list of revision ids, head first, ending at the revision whose parent is the root (so, the root revision is excluded).

func ReadRevisionChain 

func ReadRevisionChain(ctx context.Context, repository *Repository) (RevisionChain, error)

ReadRevisionChain returns the repository's revision chain, head first.

func (RevisionChain) ParseRevisionId 

func (chain RevisionChain) ParseRevisionId(spec string) (RevisionId, error)

ParseRevisionId resolves a revision spec against the chain. A spec is a hex revision id or `head`, optionally suffixed with `~<n>` to walk n revisions back toward the root, like git's `HEAD~2`. `head` and `head~0` are the head revision (the root revision on an empty repository).

func (RevisionChain) ParseRevisionRange 

func (chain RevisionChain) ParseRevisionRange(spec string) (RevisionRange, error)

ParseRevisionRange parses a revision range, resolving each bound against the chain. Formats: 

<rev>             only <rev>
<since>..<until>  excludes <since>, like git's `since..until`
<since>..         after <since> up to the head
..<until>         the root up to <until>
(empty)           the whole chain, the same as `..head`

Each bound is a spec accepted by ParseRevisionId (an id or `head`, with an optional `~<n>`).

type RevisionEntry 

type RevisionEntry struct {
	Kind     RevisionEntryKind
	Path     Path
	Metadata PathMetadata
}

func UnmarshallRevisionEntry 

func UnmarshallRevisionEntry(r *ProtobufReader) (*RevisionEntry, error)

func (*RevisionEntry) Marshall 

func (o *RevisionEntry) Marshall(w ProtobufWriter) error

func (*RevisionEntry) MarshallSize 

func (o *RevisionEntry) MarshallSize() int

func (*RevisionEntry) PathCompare added in v0.0.4 

func (e *RevisionEntry) PathCompare(other *RevisionEntry) int

Compare two revision entries by their full path.

func (*RevisionEntry) PathDesc added in v0.0.4 

func (e *RevisionEntry) PathDesc() string

The path, with a trailing separator for a directory, so that a file and a directory of one path can be told apart.

func (*RevisionEntry) PathKey added in v0.0.4 

func (e *RevisionEntry) PathKey() PathKey

func (*RevisionEntry) Validate 

func (o *RevisionEntry) Validate() error

type RevisionEntryCache added in v0.0.4 

type RevisionEntryCache = TempCache[*RevisionEntry, PathKey]

func NewRevisionEntryTempCache 

func NewRevisionEntryTempCache(
	temp *Temp[*RevisionEntry],
	maxChunksInCache int,
) (*RevisionEntryCache, error)

type RevisionEntryChunk 

type RevisionEntryChunk struct {
	Entries []*RevisionEntry
}

func UnmarshallRevisionEntryChunk 

func UnmarshallRevisionEntryChunk(r *ProtobufReader) (*RevisionEntryChunk, error)

func (*RevisionEntryChunk) Marshall 

func (o *RevisionEntryChunk) Marshall(w ProtobufWriter) error

func (*RevisionEntryChunk) MarshallSize 

func (o *RevisionEntryChunk) MarshallSize() int

func (*RevisionEntryChunk) Validate 

func (o *RevisionEntryChunk) Validate() error

type RevisionEntryKind 

type RevisionEntryKind uint32
const (
	RevisionEntryKindAdd    RevisionEntryKind = 0
	RevisionEntryKindUpdate RevisionEntryKind = 1
	RevisionEntryKindDelete RevisionEntryKind = 2
)

func (RevisionEntryKind) String 

func (k RevisionEntryKind) String() string

type RevisionId 

type RevisionId BlockId

func ReadRef 

func ReadRef(ctx context.Context, storage Storage, name string) (RevisionId, error)

func (RevisionId) IsInChain 

func (id RevisionId) IsInChain(chain RevisionChain) bool

func (RevisionId) IsRoot 

func (id RevisionId) IsRoot() bool

func (RevisionId) String 

func (id RevisionId) String() string

type RevisionRange 

type RevisionRange struct {
	Since *RevisionId
	Until *RevisionId
}

RevisionRange is a span of the revision chain. Until is the included revision and Since is the excluded one, like git's `Since..Until`. A nil Until means the head. A nil Since means the root.

func (RevisionRange) IsInChain 

func (r RevisionRange) IsInChain(chain RevisionChain) bool

IsInChain reports whether every bound of the range is part of the chain. A nil bound (the head or the root) is always considered valid.

func (RevisionRange) String 

func (r RevisionRange) String() string

type RevisionReader 

type RevisionReader struct {
	// contains filtered or unexported fields
}

func NewRevisionReader 

func NewRevisionReader(repository *Repository, revision *Revision) *RevisionReader

func (*RevisionReader) Read 

func (rr *RevisionReader) Read(ctx context.Context, buf BlockBuf) (*RevisionEntry, error)

Return the next entry of the revision, or `io.EOF`.

Entries come out strictly increasing by `RevisionEntry.PathCompare`. A revision with entries not in order will fail loudly.

type RevisionSnapshotMonitor 

type RevisionSnapshotMonitor interface {
	OnRevisionStart(revisionId RevisionId)
	OnRevisionEntry(entry *RevisionEntry)
}

Building a snapshot reads every revision down to the root, so it is the longest silent phase of most commands.

type Salt 

type Salt [32]byte

func NewSalt 

func NewSalt() (Salt, error)

type Sha256 

type Sha256 [32]byte

func CalculateSha256 

func CalculateSha256(data []byte) Sha256

func (Sha256) String 

func (s Sha256) String() string

type Sha256Hmac 

type Sha256Hmac Sha256

func CalculateHmac 

func CalculateHmac(data []byte, key RawKey) Sha256Hmac

type Storage 

type Storage interface {
	Init(ctx context.Context, config Toml, headerComment string) error
	Open(ctx context.Context) (Toml, error)
	HasBlock(ctx context.Context, blockId BlockId) (bool, error)

	// Stream all block ids present in storage. `yield` returns false to stop early.
	ReadBlockIds(ctx context.Context, yield func(BlockId) bool) error

	// Return `ErrBlockNotFound` if the block does not exist.
	ReadBlock(ctx context.Context, blockId BlockId, buf BlockBuf) ([]byte, error)

	// Write a block and return whether it was written.
	//
	// Returns `true` if the block was already present.
	WriteBlock(ctx context.Context, blockId BlockId, data []byte) (bool, error)

	// Return `ErrControlFileNotFound` if the control file does not exist.
	ReadControlFile(ctx context.Context, section ControlFileSection, name string) ([]byte, error)
	WriteControlFile(ctx context.Context, section ControlFileSection, name string, data []byte) error
	HasControlFile(ctx context.Context, section ControlFileSection, name string) (bool, error)

	// Return `ErrControlFileNotFound` if the control file does not exist.
	DeleteControlFile(ctx context.Context, section ControlFileSection, name string) error

	// Create a lock file in `.cling/<purpose>/locks/<name>`. Returns
	// `*LockExistsError` if the lock is already held by another acquirer.
	Lock(ctx context.Context, name string) (func() error, error)

	// Forcefully drop a lock regardless of ownership. The caller is responsible
	// for being sure the previous holder is dead. Returns `ErrLockNotFound` if
	// there is nothing to release.
	ForceUnlock(ctx context.Context, name string) error
}

Implementations are all-or-nothing: every read returns the complete object or an error, and every write stores all bytes or fails.

type StoragePurpose 

type StoragePurpose string
const (
	StoragePurposeRepository StoragePurpose = "repository"
	StoragePurposeWorkspace  StoragePurpose = "workspace"
)

type Temp 

type Temp[T any] struct {
	// contains filtered or unexported fields
}

func NewRevisionSnapshot 

func NewRevisionSnapshot(
	ctx context.Context,
	repository *Repository,
	revisionId RevisionId,
	tmpFS FS,
	mon RevisionSnapshotMonitor,
) (*Temp[*RevisionEntry], error)

func OpenTemp 

func OpenTemp[T any](fs FS, marshaller chunkMarshaller[T]) (*Temp[T], error)

func ReadSortedBlockIds 

func ReadSortedBlockIds(ctx context.Context, storage Storage, fs FS, inspect func(BlockId)) (*Temp[BlockId], error)

ReadSortedBlockIds drains `storage.ReadBlockIds` into a sorted Temp. `inspect`, if non-nil, sees every id before it is added to the writer.

func (*Temp[T]) Chunks 

func (t *Temp[T]) Chunks() int

func (*Temp[T]) Reader 

func (t *Temp[T]) Reader(filter func(T) bool) *TempReader[T]

func (*Temp[T]) Remove 

func (t *Temp[T]) Remove() error

type TempCache 

type TempCache[T any, K comparable] struct {
	Source *Temp[T]

	CacheMisses int
	// contains filtered or unexported fields
}

`K` keys the per-chunk maps and orders the chunks, so it must be comparable and cheap to build. A string key would allocate on every lookup.

func NewTempCache 

func NewTempCache[T any, K comparable](
	temp *Temp[T],
	cacheKey func(T) K,
	compareKey func(a, b K) int,
	maxChunksInCache int,
) (*TempCache[T, K], error)

func (*TempCache[T, K]) Get 

func (tc *TempCache[T, K]) Get(key K) (T, bool, error)

type TempFrame 

type TempFrame struct {
	Data []byte
}

func UnmarshallTempFrame 

func UnmarshallTempFrame(r *ProtobufReader) (*TempFrame, error)

func (*TempFrame) Marshall 

func (o *TempFrame) Marshall(w ProtobufWriter) error

func (*TempFrame) MarshallSize 

func (o *TempFrame) MarshallSize() int

func (*TempFrame) Validate 

func (o *TempFrame) Validate() error

type TempReader 

type TempReader[T any] struct {
	// contains filtered or unexported fields
}

func (*TempReader[T]) Read 

func (tr *TempReader[T]) Read(buf BlockBuf) (T, error)

func (*TempReader[T]) ReadChunk 

func (tr *TempReader[T]) ReadChunk(i int, buf BlockBuf) ([]T, error)

type TempWriter 

type TempWriter[T any] struct {
	// contains filtered or unexported fields
}

func NewBlockIdTempWriter 

func NewBlockIdTempWriter(fs FS) *TempWriter[BlockId]

NewBlockIdTempWriter returns a sorted, de-duplicating TempWriter for BlockIds backed by `fs`.

func NewRevisionEntryTempWriter 

func NewRevisionEntryTempWriter(fs FS, maxChunkSize int) *TempWriter[*RevisionEntry]

func NewTempWriter 

func NewTempWriter[T any](
	compare func(a, b T) int,
	marshaller chunkMarshaller[T],
	fs FS,
	maxChunkSize int,
) *TempWriter[T]

Create a new TempWriter. Parameters:

  • compare: A function that compares two entries. Two entries must never be equal — use NewTempWriterWithIgnoreDuplicates to silently drop duplicates.
  • marshaller: Serializes a sorted batch of entries to a chunk file.

func NewTempWriterWithIgnoreDuplicates 

func NewTempWriterWithIgnoreDuplicates[T any](
	compare func(a, b T) int,
	marshaller chunkMarshaller[T],
	fs FS,
	maxChunkSize int,
) *TempWriter[T]

Like NewTempWriter, but duplicate entries (compare == 0) are silently dropped in rotateChunk and the CloseAndSort k-way merge instead of erroring.

func (*TempWriter[T]) Add 

func (tw *TempWriter[T]) Add(t T) error

func (*TempWriter[T]) CloseAndSort added in v0.0.4 

func (tw *TempWriter[T]) CloseAndSort() (*Temp[T], error)

Close the writer, sorting and merging every chunk into the result.

func (*TempWriter[T]) CloseWithoutSort added in v0.0.4 

func (tw *TempWriter[T]) CloseWithoutSort() (*Temp[T], error)

Close the writer and return the chunks as they are, for a caller that added its entries in order. Nothing is sorted or merged, so the caller has to be sure of that.

type TestData 

type TestData struct{}

func (TestData) Argon2idParams 

func (td TestData) Argon2idParams() Argon2idParams

Choosing the lowest allowed values to speed up the tests.

func (TestData) BlockId 

func (td TestData) BlockId(suffix string) BlockId

func (TestData) Column 

func (td TestData) Column(s string, column int) string

Return the column at `column` for every line in `s`. A bit like the command `cut -f`.

func (TestData) CommitInfo 

func (td TestData) CommitInfo() *CommitInfo

func (TestData) Dedent 

func (td TestData) Dedent(s string) string

func (TestData) EncryptedKey 

func (td TestData) EncryptedKey(suffix string) EncryptedKey

func (TestData) NewFS 

func (td TestData) NewFS(tb testing.TB) FS

Return a new FS that is cleaned up after the test. todo: Make the FS implementation configurable.

func (TestData) NewHealthCheckMonitor 

func (td TestData) NewHealthCheckMonitor() *TestHealthCheckMonitor

func (TestData) NewRealFS 

func (td TestData) NewRealFS(tb testing.TB) *RealFS

Return a new RealFS that is cleaned up after the test.

func (TestData) NewRevisionSnapshotMonitor 

func (td TestData) NewRevisionSnapshotMonitor() *TestRevisionSnapshotMonitor

func (TestData) NewTestFS 

func (td TestData) NewTestFS(tb testing.TB, fs FS) *TestFS

func (TestData) NewTestRepository 

func (td TestData) NewTestRepository(tb testing.TB, fs FS) *TestRepository

func (TestData) OpenRepository 

func (td TestData) OpenRepository(tb testing.TB, fs FS) *TestRepository

func (TestData) Path 

func (td TestData) Path(p string) Path

func (TestData) PathMetadata 

func (td TestData) PathMetadata(mode FileMode) *PathMetadata

func (TestData) RawKey 

func (td TestData) RawKey(suffix string) RawKey

func (TestData) Revision 

func (td TestData) Revision(parent RevisionId) *Revision

func (TestData) RevisionChain 

func (td TestData) RevisionChain(tb testing.TB, r *TestRepository) RevisionChain

func (TestData) RevisionEntry 

func (td TestData) RevisionEntry(path string, entryType RevisionEntryKind) *RevisionEntry

func (TestData) RevisionEntryExt 

func (td TestData) RevisionEntryExt(
	path string,
	entryType RevisionEntryKind,
	mode FileMode,
	content string,
) *RevisionEntry

func (TestData) RevisionId 

func (td TestData) RevisionId(suffix string) RevisionId

func (TestData) SHA256 

func (td TestData) SHA256(content string) Sha256

func (TestData) Sort 

func (td TestData) Sort(s string, column int) string

Sort like the `sort` command.

func (TestData) Wc 

func (td TestData) Wc(option string, content string) int

`option` can only be `-l` at the moment.

type TestFS 

type TestFS struct {
	FS
	// contains filtered or unexported fields
}

func (*TestFS) Cat 

func (f *TestFS) Cat(path string) string

func (*TestFS) Chmod 

func (f *TestFS) Chmod(path string, mode fs.FileMode)

func (*TestFS) Chown 

func (f *TestFS) Chown(path string, uid int, gid int)

func (*TestFS) Ls 

func (f *TestFS) Ls(path string) []TestFileInfo

func (*TestFS) Mkdir 

func (f *TestFS) Mkdir(path string)

func (*TestFS) MkdirAll 

func (f *TestFS) MkdirAll(path string)

func (*TestFS) PathMetadata 

func (f *TestFS) PathMetadata(path string) *PathMetadata
func (f *TestFS) ReadLink(name string) string

func (*TestFS) Rm 

func (f *TestFS) Rm(path string)

func (*TestFS) RmAll 

func (f *TestFS) RmAll(path string)

func (*TestFS) Sha256 

func (f *TestFS) Sha256(path string) Sha256

func (*TestFS) Stat 

func (f *TestFS) Stat(path string) fs.FileInfo
func (f *TestFS) Symlink(target string, name string)

func (*TestFS) Touch 

func (f *TestFS) Touch(path string, mtime time.Time)

func (*TestFS) Write 

func (f *TestFS) Write(path string, content string)

type TestFileInfo 

type TestFileInfo struct {
	Path    string
	Mode    fs.FileMode
	Size    int
	Content string
}

type TestHealthCheckMonitor 

type TestHealthCheckMonitor struct {
	Calls []MockCall
}

func (*TestHealthCheckMonitor) OnBlockVerified 

func (m *TestHealthCheckMonitor) OnBlockVerified(blockId BlockId, length int)

func (*TestHealthCheckMonitor) OnOrphanedBlock 

func (m *TestHealthCheckMonitor) OnOrphanedBlock(blockId BlockId)

func (*TestHealthCheckMonitor) OnRevisionEntry 

func (m *TestHealthCheckMonitor) OnRevisionEntry(entry *RevisionEntry)

func (*TestHealthCheckMonitor) OnRevisionStart 

func (m *TestHealthCheckMonitor) OnRevisionStart(revisionId RevisionId)

type TestRepository 

type TestRepository struct {
	*Repository
	*TestFS
	Passphrase string
	Storage    *FileStorage
	// contains filtered or unexported fields
}

func (*TestRepository) AddRevision added in v0.0.4 

func (r *TestRepository) AddRevision(parent RevisionId, blocks ...[]*RevisionEntry) RevisionId

Write a revision with one entry block per argument, so a test can control the block layout that `Commit` derives from `DefaultTempChunkSize`. Blocks must be given in path order; entries within a block are sorted.

func (*TestRepository) Head 

func (r *TestRepository) Head() RevisionId

func (*TestRepository) RevisionEntryReaderInfos 

func (r *TestRepository) RevisionEntryReaderInfos(
	read func(buf BlockBuf) (*RevisionEntry, error),
) []TestRevisionEntryInfo

func (*TestRepository) RevisionInfos 

func (r *TestRepository) RevisionInfos(revisionId RevisionId) []TestRevisionEntryInfo

func (*TestRepository) RevisionSnapshot 

func (r *TestRepository) RevisionSnapshot(revisionId RevisionId, pathFilter PathFilter) []*RevisionEntry

func (*TestRepository) RevisionSnapshotFileInfos 

func (r *TestRepository) RevisionSnapshotFileInfos(revisionId RevisionId, pathFilter PathFilter) []TestFileInfo

func (*TestRepository) RevisionTempInfos 

func (r *TestRepository) RevisionTempInfos(temp *Temp[*RevisionEntry]) []TestRevisionEntryInfo

func (*TestRepository) View added in v0.0.5 

func (r *TestRepository) View(prefix string) *RepositoryView

type TestRevisionEntryInfo 

type TestRevisionEntryInfo struct {
	Path string
	Type RevisionEntryKind
	Mode fs.FileMode
	Hash Sha256
}

type TestRevisionSnapshotMonitor 

type TestRevisionSnapshotMonitor struct {
	Calls []MockCall
}

func (*TestRevisionSnapshotMonitor) OnRevisionEntry 

func (m *TestRevisionSnapshotMonitor) OnRevisionEntry(entry *RevisionEntry)

func (*TestRevisionSnapshotMonitor) OnRevisionStart 

func (m *TestRevisionSnapshotMonitor) OnRevisionStart(revisionId RevisionId)

type Timestamp 

type Timestamp struct {
	Sec  int64
	Nsec uint32
}

func NewTimestampFromTime 

func NewTimestampFromTime(t time.Time) Timestamp

func NewTimestampNow 

func NewTimestampNow() Timestamp

func UnmarshallTimestamp 

func UnmarshallTimestamp(r *ProtobufReader) (*Timestamp, error)

func (*Timestamp) Marshall 

func (o *Timestamp) Marshall(w ProtobufWriter) error

func (*Timestamp) MarshallSize 

func (o *Timestamp) MarshallSize() int

func (*Timestamp) Time 

func (t *Timestamp) Time() time.Time

func (*Timestamp) Validate 

func (o *Timestamp) Validate() error

type Toml 

type Toml map[string]map[string]string

This is a very rudimentary TOML reader/writer. It only supports what is really needed.

func ReadToml 

func ReadToml(src io.Reader) (Toml, error)

func (Toml) Eq 

func (t Toml) Eq(other Toml) bool

func (Toml) GetIntValue 

func (t Toml) GetIntValue(section string, key string) (int, bool)

func (Toml) GetValue 

func (t Toml) GetValue(section string, key string) (string, bool)

Return `value, true` if the key exists, `"", false` otherwise.

type UserKey 

type UserKey RawKey

This is the key derived from the user's passphrase that is used to encrypt the KEK.

type ViewCommit added in v0.0.5 

type ViewCommit struct {
	// contains filtered or unexported fields
}

A commit whose entries are given as seen from a view.

func (*ViewCommit) Add added in v0.0.5 

func (c *ViewCommit) Add(entry *RevisionEntry) error

Add an entry to the commit.

A directory added or updated where the view hides a symlink is recorded next to the symlink's deletion, because a file and a directory of one path are different entries to the repository. A file-like entry simply supersedes the symlink and is recorded as given. An update can meet a hidden symlink because local changes are computed against the workspace head, while the commit lands on the repository head.

func (*ViewCommit) Commit added in v0.0.5 

func (c *ViewCommit) Commit(ctx context.Context, info *CommitInfo) (RevisionId, error)

type ViewRevisionReader added in v0.0.5 

type ViewRevisionReader struct {
	// contains filtered or unexported fields
}

Reads the entries of one revision as seen from a view.

func (*ViewRevisionReader) Hidden added in v0.0.5 

func (r *ViewRevisionReader) Hidden() int

The number of entries read so far that the view does not show.

func (*ViewRevisionReader) Read added in v0.0.5 

func (r *ViewRevisionReader) Read(ctx context.Context, buf BlockBuf) (*RevisionEntry, error)

Return the next visible entry of the revision, or `io.EOF`.

type ViewSnapshot added in v0.0.5 

type ViewSnapshot struct {
	*Temp[*RevisionEntry]
	RevisionId RevisionId
	// contains filtered or unexported fields
}

A revision snapshot as seen from a view.

func (*ViewSnapshot) Cache added in v0.0.5 

func (s *ViewSnapshot) Cache() (*RevisionEntryCache, error)

Snapshot cache by path, built on first use.

func (*ViewSnapshot) Remove added in v0.0.5 

func (s *ViewSnapshot) Remove() error

type WrappedError 

type WrappedError struct {
	Msg string
	// contains filtered or unexported fields
}

func Errorf 

func Errorf(msg string, msgArgs ...any) *WrappedError

func WrapErrorf 

func WrapErrorf(err error, msg string, msgArgs ...any) *WrappedError

func (*WrappedError) Error 

func (w *WrappedError) Error() string

func (*WrappedError) Is 

func (w *WrappedError) Is(target error) bool

func (*WrappedError) Unwrap 

func (w *WrappedError) Unwrap() error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.