cache

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 27, 2025 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package cache provides efficient caching of Git repositories with two-tier storage.

Overview

The repository cache avoids repeated network operations by maintaining:

  1. Bare repositories (Tier 1): Single source of truth for Git objects
  2. Working tree checkouts (Tier 2): Multiple isolated checkouts per repository
  3. Metadata index: Tracks checkout lifecycle with TTL-based expiration

Architecture

The cache uses a two-tier structure with git alternates for efficient storage: 

~/.cache/git/
├── index.json               # Metadata index
├── bare/                    # Tier 1: Bare repositories (object storage)
│   └── github.com/
│       └── my/
│           └── repo.git/    # All git objects stored here
└── checkouts/               # Tier 2: Working trees
    └── github.com/my/repo/
        ├── main/
        │   ├── team-docs/       # Persistent checkout
        │   └── build-abc123/    # Ephemeral checkout
        └── v1.0.0/
            └── prod-ref/

Checkouts use git alternates (.git/objects/info/alternates) to reference the bare repository's object database, avoiding duplication of git objects. This provides ~90% disk savings compared to copying objects to each checkout.

Usage

Create a cache and get a checkout: 

cache, err := cache.NewRepositoryCache("~/.cache/git")
if err != nil {
    return err
}

// Persistent checkout with stable key
path, err := cache.GetCheckout(ctx, "https://github.com/my/repo", "team-docs",
    cache.WithRef("main"))

// Ephemeral checkout with unique key and TTL
cacheKey := uuid.New().String()
path, err = cache.GetCheckout(ctx, "https://github.com/my/repo", cacheKey,
    cache.WithRef("main"),
    cache.WithTTL(1*time.Hour))

Start background garbage collection: 

stop := cache.StartGC(5*time.Minute, cache.PruneExpired())
defer stop()

Cache Keys

Checkouts are identified by a composite key: (URL + ref + cacheKey)

  • Same composite key = reuses existing checkout
  • Different cache key = creates isolated checkout
  • Different ref = creates separate checkout for that ref

Use stable keys (e.g., "team-docs") for persistent checkouts that should be reused across calls. Use unique keys (e.g., UUID) for ephemeral checkouts that should be isolated and cleaned up after use.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CacheOption 

type CacheOption func(*cacheOptions)

CacheOption configures cache operations.

func WithAuth 

func WithAuth(auth git.Auth) CacheOption

WithAuth provides authentication for network operations.

Example: 

auth, _ := git.SSHKeyFile("git", "~/.ssh/id_rsa")
path, _ := cache.GetCheckout(ctx, url, key, WithAuth(auth))

func WithDepth 

func WithDepth(depth int) CacheOption

WithDepth sets shallow clone depth (0 = full clone).

Example: 

path, _ := cache.GetCheckout(ctx, url, key, WithDepth(1))

func WithRef 

func WithRef(ref string) CacheOption

WithRef specifies which Git reference to checkout (branch, tag, or commit). If not specified, uses the repository's default branch. The ref becomes part of the composite cache key.

Example: 

path, _ := cache.GetCheckout(ctx, url, key, WithRef("v1.0.0"))

func WithTTL 

func WithTTL(ttl time.Duration) CacheOption

WithTTL sets time-to-live for automatic cleanup via Prune(). Checkouts with expired TTL are removed during pruning. Use for ephemeral checkouts (e.g., CI/CD builds).

Example: 

// Auto-cleanup after 1 hour
path, _ := cache.GetCheckout(ctx, url, uuid.New().String(),
    WithTTL(1*time.Hour))

func WithUpdate 

func WithUpdate() CacheOption

WithUpdate checks the remote and updates the cache if needed. Without this option, cached data is returned as-is (may be stale).

Example: 

path, _ := cache.GetCheckout(ctx, url, key, WithUpdate())

type CacheStats 

type CacheStats struct {
	BareRepos      int   // Number of bare repositories
	Checkouts      int   // Number of checkouts
	TotalSize      int64 // Total disk usage in bytes
	BareSize       int64 // Disk usage of bare repositories
	CheckoutsSize  int64 // Disk usage of checkouts
	OldestCheckout *time.Time
	NewestCheckout *time.Time
}

CacheStats provides statistics about the cache.

type CheckoutMetadata 

type CheckoutMetadata struct {
	URL        string         // Original repository URL
	Ref        string         // Git reference (branch/tag/commit)
	CacheKey   string         // User-provided cache key
	CreatedAt  time.Time      // When checkout was created
	LastAccess time.Time      // Last time checkout was accessed
	TTL        *time.Duration // Time-to-live (nil = persistent)
	ExpiresAt  *time.Time     // Computed expiration time
}

CheckoutMetadata tracks metadata for a single checkout.

type PruneStrategy 

type PruneStrategy interface {
	ShouldPrune(metadata *CheckoutMetadata) bool
}

PruneStrategy determines which checkouts should be removed during pruning.

func PruneExpired 

func PruneExpired() PruneStrategy

PruneExpired removes checkouts with expired TTL. This is the default strategy if no strategies are provided.

func PruneOlderThan 

func PruneOlderThan(maxAge time.Duration) PruneStrategy

PruneOlderThan removes checkouts not accessed within the specified duration.

Example: 

cache.Prune(PruneOlderThan(7*24*time.Hour)) // Remove if unused for 7 days

func PruneToSize 

func PruneToSize(maxBytes int64) PruneStrategy

PruneToSize removes oldest checkouts until total cache size is under limit. Removes least-recently-accessed checkouts first. Never removes checkouts without TTL (persistent checkouts).

Example: 

cache.Prune(PruneToSize(10*1024*1024*1024)) // Keep under 10GB

type RepositoryCache 

type RepositoryCache struct {
	// contains filtered or unexported fields
}

RepositoryCache manages a two-tier cache of Git repositories.

Tier 1 (Bare Cache): Stores bare repositories (Git database only) for efficient storage and network operations.

Tier 2 (Checkout Cache): Stores working trees for actual file access. Each checkout is identified by a composite key (URL + ref + cacheKey).

The cache maintains a metadata index for lifecycle management, supporting TTL-based expiration and automatic garbage collection.

func NewRepositoryCache 

func NewRepositoryCache(basePath string, opts ...RepositoryCacheOption) (*RepositoryCache, error)

NewRepositoryCache creates a new repository cache at the specified base path. The cache manages bare repositories and checkouts with metadata tracking.

The basePath typically points to ~/.cache/git or a similar cache directory. The cache creates two subdirectories (bare/, checkouts/) and an index.json file.

By default, NewRepositoryCache uses the local filesystem (osfs). A custom filesystem can be provided via WithFilesystem for testing purposes.

Example: 

cache, err := cache.NewRepositoryCache("~/.cache/git")

// With custom filesystem (for testing)
cache, err := cache.NewRepositoryCache("/cache/path",
    cache.WithFilesystem(memfs.New()))

func (*RepositoryCache) Clear 

func (c *RepositoryCache) Clear(url string) error

Clear removes all cached data for a specific URL. This includes the bare cache, all checkouts for that URL, and index entries.

Example: 

cache.Clear("https://github.com/my/repo")

func (*RepositoryCache) ClearAll 

func (c *RepositoryCache) ClearAll() error

ClearAll removes all cached repositories and resets the cache. This removes all bare repositories, checkouts, and the index.

func (*RepositoryCache) GetCheckout 

func (c *RepositoryCache) GetCheckout(ctx context.Context, url, cacheKey string, opts ...CacheOption) (string, error)

GetCheckout returns a path to a working tree suitable for symlinking or temporary use.

The checkout is identified by a composite key: (url + ref + cacheKey). - Same composite key = reuses existing checkout (if it exists) - Different cache key = creates isolated checkout - Different ref = creates separate checkout for that ref

If the repository doesn't exist in cache, it's cloned. If WithUpdate() is specified, the cache is refreshed from the remote before returning.

The cacheKey controls checkout lifecycle: - Stable key (e.g., "team-docs") = persistent, reused across calls - Unique key (e.g., UUID) = ephemeral, intended for single use

Ephemeral checkouts should specify WithTTL() to enable automatic cleanup.

The returned path remains valid across calls and can be safely symlinked. Symlinks are preserved during updates via in-place refresh.

Examples: 

// Persistent checkout with stable key
path, _ := cache.GetCheckout(ctx, "https://github.com/my/docs", "team-docs",
    WithRef("main"))
os.Symlink(filepath.Join(path, "docs"), ".sow/refs/team-docs")

// Ephemeral checkout with unique key and TTL
cacheKey := uuid.New().String()
path, _ := cache.GetCheckout(ctx, "https://github.com/my/repo", cacheKey,
    WithRef("main"),
    WithTTL(1*time.Hour))
defer cache.RemoveCheckout(url, cacheKey)

// Force fresh update
path, _ := cache.GetCheckout(ctx, url, "build",
    WithUpdate(),
    WithAuth(auth))

func (*RepositoryCache) Prune 

func (c *RepositoryCache) Prune(strategies ...PruneStrategy) error

Prune removes checkouts based on the provided strategies. Common strategies include expired TTL, last access time, and cache size limits.

If no strategies are provided, defaults to removing expired checkouts.

Examples: 

// Remove expired checkouts (TTL-based)
cache.Prune()

// Remove checkouts not accessed in 7 days
cache.Prune(PruneOlderThan(7*24*time.Hour))

// Multiple strategies (OR logic)
cache.Prune(PruneExpired(), PruneOlderThan(30*24*time.Hour))

func (*RepositoryCache) RemoveCheckout 

func (c *RepositoryCache) RemoveCheckout(url, cacheKey string) error

RemoveCheckout removes a specific checkout by URL and cache key. This removes the checkout directory, updates the index, and removes it from the in-memory cache.

Example: 

cache.RemoveCheckout("https://github.com/my/repo", "build-abc123")

func (*RepositoryCache) StartGC 

func (c *RepositoryCache) StartGC(interval time.Duration, strategies ...PruneStrategy) (stop func())

StartGC starts a background garbage collector that periodically prunes the cache. The GC runs at the specified interval using the provided pruning strategies.

Returns a function to stop the garbage collector. This function is safe to call multiple times and will block until the GC goroutine has fully stopped.

The stop function should be called on shutdown or deferred to ensure clean shutdown.

Examples: 

// Start GC that runs every 5 minutes, removing expired checkouts
stop := cache.StartGC(5*time.Minute, PruneExpired())
defer stop()

// Worker service with multiple strategies
stop := cache.StartGC(10*time.Minute,
    PruneExpired(),
    PruneOlderThan(24*time.Hour))
defer stop()

func (*RepositoryCache) Stats 

func (c *RepositoryCache) Stats() (*CacheStats, error)

Stats returns statistics about the cache (entries, disk usage, etc.).

type RepositoryCacheOption 

type RepositoryCacheOption func(*repositoryCacheOptions)

RepositoryCacheOption configures RepositoryCache creation.

func WithFilesystem 

func WithFilesystem(fs billy.Filesystem) RepositoryCacheOption

WithFilesystem sets the billy filesystem to use for cache operations. If not provided, defaults to osfs.New(basePath) rooted at the cache base path.

This option is primarily useful for testing, allowing use of memfs or other virtual filesystems.

Example: 

cache, err := cache.NewRepositoryCache("/cache/path",
    cache.WithFilesystem(memfs.New()))

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.