hydrator

Documentation

Overview

Package hydrator turns a priority-queued stream of HydrationTasks into materialised blobs in a content-addressed cache. Multiple concurrent readers requesting the same OID are coalesced onto a single fetch.

Inflight deduplicates multiple callers waiting on the same key. Used to coalesce FUSE readers that all want the same OID hydrated.

Index

• Constants
• func ClassifyPriority(path string) int
• type BlobFetcher
• type OnHydratedFunc
• type Service
  • func New(fetcher BlobFetcher) *Service
  • func (s *Service) Enqueue(task model.HydrationTask)
  • func (s *Service) EnsureHydrated(ctx context.Context, repo model.RepoConfig, node model.BaseNode) (cachePath string, size int64, err error)
  • func (s *Service) QueueDepth(repoID model.RepoID) int
  • func (s *Service) SetOnHydrated(fn OnHydratedFunc)
  • func (s *Service) Start(workers int, repo model.RepoConfig)
  • func (s *Service) Stop()

Constants

const (
	PriorityExplicitRead = 1000
	PrioritySibling      = 800
	PriorityBootstrap    = 700
	PriorityLikelyText   = 500
	PriorityNearbyCode   = 400
	PriorityBinary       = 100
)

Priority levels — higher wins. An explicit user read beats any speculative pre-fetch.

Functions

func ClassifyPriority

func ClassifyPriority(path string) int

ClassifyPriority picks a priority for speculative pre-fetching based on the file's path. (ghfs v1 doesn't pre-fetch, but this is retained for parity and future use.)

Types

type BlobFetcher

type BlobFetcher interface {
	BlobToCache(ctx context.Context, repo model.RepoConfig, objectOID string, dstPath string) (size int64, err error)
	VerifyBlob(ctx context.Context, repo model.RepoConfig, objectOID string, cachePath string) (ok bool, err error)
}

BlobFetcher is the interface the hydrator calls to materialise a blob. gitstore.Store implements it.

type OnHydratedFunc

type OnHydratedFunc func(repoID model.RepoID, objectOID string, size int64)

OnHydratedFunc is invoked after every successful blob fetch. ghfs uses it to account bytes added to the shared blob cache for eviction.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service owns the hydrator's priority queue, worker pool, and inflight-dedup maps for one repo's clone.

func New

func New(fetcher BlobFetcher) *Service

New constructs a Service. Call Start before Enqueue / EnsureHydrated.

func (*Service) Enqueue

func (s *Service) Enqueue(task model.HydrationTask)

Enqueue adds a task without waiting for it to complete. Useful for speculative pre-fetching (not used in ghfs v1).

func (*Service) EnsureHydrated

func (s *Service) EnsureHydrated(ctx context.Context, repo model.RepoConfig, node model.BaseNode) (cachePath string, size int64, err error)

EnsureHydrated blocks until node's blob is present in the cache. If the cache already has a valid file (size matches, or hash verifies), returns immediately. Otherwise enqueues an explicit-read priority task and waits for the worker.

func (*Service) QueueDepth

func (s *Service) QueueDepth(repoID model.RepoID) int

QueueDepth returns the number of queued tasks for a given repo.

func (*Service) SetOnHydrated

func (s *Service) SetOnHydrated(fn OnHydratedFunc)

SetOnHydrated registers a callback invoked after each successful blob fetch.

func (*Service) Start

func (s *Service) Start(workers int, repo model.RepoConfig)

Start spins up `workers` goroutines pinned to repo.

func (*Service) Stop

func (s *Service) Stop()

Stop terminates workers and fails any waiters that were mid-flight.