projection

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 10, 2026 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package projection builds virtual human-friendly views over the store without copying data. A read-only alternative to searching by namespace/SessionID for cases when the natural mental model is a hierarchy (by path, by session, by type).

A View is a snapshot of the state at the moment it was created. Live updates are on the backlog. The data source can be either a single core.DataStore or a curator.Curator: in the first case StorageFacet remains nil; in the second it is filled from MultistoreIndex and HostStorage.

Mounting (FUSE, WebDAV) ships as separate integrations gated by the build tags `fuse` and `webdav`. Without the tags the API is available (so development stays transparent) but returns ErrFUSENotSupported / ErrWebDAVNotSupported.

DAG: projection imports core, event. It does not import curator (the dependency is inverted via ProjectionSource), agent, or maintenance.

Implementation lands in M6. In M0 — the type contracts.

Package projection builds virtual filesystem-like views over the flat content-addressed store. The package is the seam at which transport-specific daemons (cmd/scrinium-fuse, cmd/scrinium-webdav) plug in: projection itself does no syscalls and no networking.

Architecture: View is the in-memory tree (read side) populated by backfill from a ProjectionSource. FSOps adds the write side — create/unlink/rename/setattr — and is the place where scratch buffering, path-level locks and editing policies live. Together they cover ~80% of what FUSE and WebDAV daemons need; the transport layer is a thin dispatcher.

Schemas describing how artifacts map to filesystem paths live in subpackages (projection/fsmeta is the standard one). They are pluggable through the PathResolver function passed to NewView.

Specification: docs/3 §5 Projection API, docs/4 §13 Projection, docs/4 §14 FUSE Mount.

Index

Constants

View Source 
const (
	// EventPathCollision is emitted when two artifacts compete for
	// the same by-path entry; the loser stays accessible through
	// by-artifact.
	EventPathCollision = "projection.path_collision"

	// EventViewRebuilt is emitted after a successful backfill.
	EventViewRebuilt = "projection.view_rebuilt"
)

Variables

View Source 
var ErrRouteRejected = errors.New("scrinium-fuse: path rejected by routing")

ErrRouteRejected is returned by Route when the path falls into RouteRejected. The dispatcher translates it to ENOENT or EACCES depending on the call site.

Functions

func IsServicePath 

func IsServicePath(path string, cfg RoutingConfig) bool

IsServicePath reports whether the path's first segment equals the configured service prefix. Useful when validating new-file creation: writes to <servicePrefix>/* are forbidden because the service trees are read-only.

func RenderStats 

func RenderStats(view *View, info DaemonInfo) []byte

RenderStats produces the canonical text rendering used by FUSE's _scrinium/stats virtual file and WebDAV's same-named endpoint. Sections are grouped and labelled; empty groups are omitted; numbers that aren't available are rendered "n/a" rather than "-1" so the output stays readable.

Format is plain text with one "key: value" per line. Stable across versions: tooling that scrapes it stays valid as long as the field names don't move.

Types

type ArtifactFacet 

type ArtifactFacet struct {
	ArtifactID  domain.ArtifactID
	ContentHash domain.ContentHash
	BlobRef     domain.BlobRef
	Namespace   string
	SessionID   string
	CreatedAt   time.Time
	Type        domain.ManifestType
	Metadata    json.RawMessage
}

ArtifactFacet carries the CAS metadata of a concrete artifact. Populated for file nodes; nil for virtual directories.

type Attrs 

type Attrs struct {
	Mode    *uint32
	UID     *uint32
	GID     *uint32
	ModTime *time.Time
}

Attrs is the set of attribute updates passed to Setattr. nil fields mean "leave unchanged".

type DaemonInfo 

type DaemonInfo struct {
	// StartedAt is the daemon's startup timestamp. Used for
	// the Started/Uptime lines in the rendered output.
	StartedAt time.Time

	// MountSession is the per-process session id assigned to
	// every artifact this daemon writes. Useful when
	// inspecting "what did this mount produce so far".
	MountSession string

	// StorePath is the on-disk root the daemon was launched
	// against. Helps when multiple daemons run on one host.
	StorePath string

	// ReadOnly reflects the daemon's write policy. WebDAV and
	// FUSE both have a read-only mode — surface it so the
	// inspector knows whether writes were even possible.
	ReadOnly bool

	// Editing is the editing-mode label ("off" / "on" /
	// "custom") for daemons that have one. Empty hides the row.
	Editing string

	// Namespace is the default namespace stamped on artifacts
	// created through this daemon. Empty hides the row.
	Namespace string

	// Capacity is the optional storage snapshot from
	// core.Store.Capacity. nil hides the [storage] section
	// entirely. -1 inside fields means "Driver did not report".
	Capacity *domain.StorageInfo

	// Extensions lists the host-side index extensions
	// registered against the StoreIndex. nil hides the
	// [extensions] section. Order doesn't matter — RenderStats
	// sorts by Name for stable output.
	Extensions []ExtensionInfo
}

DaemonInfo carries the per-process state RenderStats can't derive from the View alone. Daemons (FUSE, WebDAV) construct it once at startup and pass it on every render — RenderStats is stateless beyond what the View itself holds.

Fields with no meaningful value should be left zero; RenderStats hides empty groups and "n/a"-renders absent numbers (Capacity == nil, Extensions == nil).

type EditingPolicy 

type EditingPolicy struct {
	AllowRename   bool
	AllowSetattr  bool
	AllowTruncate bool
	AllowAppend   bool
}

EditingPolicy is the per-operation switchboard for the editing surface (rename, setattr, truncate, append). Each bit is independent; helpers EditingOff / EditingOn are sugar.

func EditingOff 

func EditingOff() EditingPolicy

EditingOff is the conservative default: no editing of existing artifacts. Create and Unlink still work.

func EditingOn 

func EditingOn() EditingPolicy

EditingOn enables every editing capability. Use only when callers understand the CAS implications (every mutation produces a new artifact).

type ExtensionInfo 

type ExtensionInfo struct {
	Name          string
	SchemaVersion int
}

ExtensionInfo is the projection-layer DTO for rendering information about a registered index extension. It is a pure render-time type — no behaviour, no dependencies on engine/index — so the projection package stays a leaf in the import graph.

Callers that hold an index.ExtensionInfo (the contract type) translate field-for-field at the call site: the shapes are intentionally identical.

type FSOps 

type FSOps struct {
	// contains filtered or unexported fields
}

FSOps is the filesystem-shaped operations layer over a View. It serves transports (FUSE, WebDAV) by translating path-keyed POSIX-like calls into View lookups (read side) and, in stage 4b, into Store mutations (write side).

The two reasons FSOps exists rather than the transport calling View directly:

  1. Editing policy, scratch buffering, path-level locks live in one place — FUSE and WebDAV inherit them by construction.
  2. The transport works in terms of the configured root tree (RootView). FSOps hides that routing from the caller.

Stage 4a: read-side (Stat, Listdir, Open). Mutations land in 4b.

func NewFSOps 

func NewFSOps(v *View, opts ...FSOpsOption) (*FSOps, error)

NewFSOps wraps a View with filesystem-shaped operations.

The View must already exist (the caller is responsible for the View's lifecycle); NewFSOps does not call NewView itself because the View may be shared with other transports.

Defaults:

  • DefaultMode 0644, DefaultUID/GID 0
  • EditingOff (no rename, setattr, truncate, append)
  • ScratchQuota 0 (unlimited; the OS still imposes its own)

Returns an error only if v is nil. Configuration sanity is otherwise the caller's responsibility (e.g. an invalid scratch dir surfaces only at the first Create call).

func (*FSOps) Create 

func (o *FSOps) Create(ctx context.Context, path string, mode uint32) (File, error)

Create makes a new file at path and returns a writable File handle. The handle buffers writes in a scratch file; on Close the scratch is consumed by Store.Put and the resulting manifest is added to the View.

Errors:

  • ErrInvalidPath if path fails fsmeta validation.
  • ErrEditingDisabled if FSOps was constructed with WithReadOnly.
  • "WithNamespace not configured" if Namespace is empty.
  • "WithStore not configured" if no StoreClient was supplied.
  • ErrPathExists wrapping the existing-path detail when the target is already taken.

Stage 4b only supports Create for new paths; opening an existing path for write lands in 4c.

func (*FSOps) Listdir 

func (o *FSOps) Listdir(path string) FileInfoSeq

Listdir streams the immediate children of path. Returns ErrNotADirectory on a file, ErrPathNotFound on a missing path.

Like Stat, Listdir surfaces Mkdir-created virtual directories even when they have no real children yet. The streamed children include both real (View-known) entries and pending directories whose parent matches path.

func (*FSOps) Mkdir 

func (o *FSOps) Mkdir(path string, mode uint32) error

Mkdir creates a virtual directory at path. The directory is "pending" until a real artifact is created inside it; until then it is visible only through Stat/Listdir on this FSOps (it does not exist in any tree of the View).

Errors:

  • ErrEditingDisabled if FSOps is read-only.
  • ErrInvalidPath if path fails validation.
  • ErrPathExists if path is already taken (real or pending).

func (*FSOps) Open 

func (o *FSOps) Open(ctx context.Context, path string, mode OpenMode) (File, error)

Open returns a File handle. The mode bits select the access pattern:

  • OpenReadOnly — read existing artifact via View.
  • OpenWriteOnly / OpenReadWrite — open scratch buffer for editing the existing artifact at path. Editing of an existing artifact requires AllowSetattr or AllowTruncate (4c); 4b only supports OpenReadWrite for newly-created files (use Create for new files).
  • OpenAppend — requires AllowAppend (4c).

In stage 4b, Open with a write mode on an existing file returns ErrEditingDisabled — Create is the documented entry point for new files; Setattr/Truncate (4c) covers editing.

func (*FSOps) Rename 

func (o *FSOps) Rename(ctx context.Context, oldPath, newPath string) error

Rename moves an artifact from oldPath to newPath. In CAS terms the operation is a Put-with-new-fsmeta-Path followed by a Delete of the old artifact, atomically reflected in the View via View.Move.

Errors:

  • ErrEditingDisabled if AllowRename is off or FSOps is read-only.
  • ErrInvalidPath if newPath fails validation.
  • ErrPathNotFound if oldPath does not exist.
  • ErrIsADirectory if oldPath points at a virtual directory.
  • ErrPathExists if newPath is already taken.
  • Any error from Store.Put / Store.Delete.

func (*FSOps) Rmdir 

func (o *FSOps) Rmdir(path string) error

Rmdir removes a directory.

Behaviour:

  • For a pending directory (Mkdir-created, no real children) — drop it from pendingDirs.
  • For a virtual directory in the View — succeed if empty (no children in the tree), otherwise ErrNotEmpty.
  • On a file path — ErrNotADirectory.
  • On an unknown path — ErrPathNotFound.

Removing a virtual directory from the View has no persistent effect: the directory exists by virtue of having children, and a successful Rmdir on an already-empty view-dir is a no-op outside the FSOps's own state. Future Adds re-create it automatically through ensureDirs.

func (*FSOps) Setattr 

func (o *FSOps) Setattr(ctx context.Context, path string, attrs Attrs) error

Setattr changes POSIX attributes (mode, uid, gid, mtime) of an existing artifact. Each non-nil field of attrs is applied; other fsmeta fields (Path, MIME) are preserved. The operation produces a new artifact with the same content (the underlying blob is deduplicated by the Store) and removes the old.

Errors mirror Rename, plus ErrEditingDisabled when AllowSetattr is off.

func (*FSOps) Stat 

func (o *FSOps) Stat(path string) (FileInfo, error)

Stat returns the FileInfo for a virtual path interpreted in the configured root tree.

Stat also surfaces virtual directories created via Mkdir that have no children yet — without them, sequences like `mkdir foo && stat foo` would yield ENOENT for paths that the caller just created.

func (*FSOps) Truncate 

func (o *FSOps) Truncate(ctx context.Context, path string, size int64) error

Truncate adjusts the size of an existing artifact. The new file is materialised by reading the existing content, capping at size (or extending with zeros if size > current), and writing a new artifact. The old is removed.

Errors mirror Rename, plus ErrEditingDisabled when AllowTruncate is off, plus ErrScratchQuota if the scratch pre-allocation would exceed the quota. 

func (o *FSOps) Unlink(ctx context.Context, path string) error

Unlink deletes the artifact at path. The View entry is removed after a successful Store.Delete.

Errors:

  • ErrEditingDisabled if FSOps is read-only.
  • ErrPathNotFound if path is unknown to the View.
  • ErrIsADirectory if path is a virtual directory; use Rmdir.
  • Any error from Store.Delete (e.g. ErrLocked, ErrRetentionActive) is propagated.

type FSOpsOption 

type FSOpsOption func(*fsOpsOptions)

FSOpsOption configures NewFSOps.

func WithDefaultGID 

func WithDefaultGID(gid uint32) FSOpsOption

WithDefaultGID applies to fsmeta.GID == 0.

func WithDefaultMode 

func WithDefaultMode(mode uint32) FSOpsOption

WithDefaultMode is the POSIX mode applied to artifacts whose fsmeta.Mode is zero. Default 0644.

func WithDefaultUID 

func WithDefaultUID(uid uint32) FSOpsOption

WithDefaultUID applies to fsmeta.UID == 0.

func WithEditingPolicy 

func WithEditingPolicy(p EditingPolicy) FSOpsOption

WithEditingPolicy gates editing operations. Default: EditingOff.

func WithMountSession 

func WithMountSession(sid string) FSOpsOption

WithMountSession sets the SessionID stamped onto every Put performed through FSOps in this mount. Empty means "no session" — artifacts will not appear in by-session.

func WithNamespace 

func WithNamespace(ns string) FSOpsOption

WithNamespace sets the Namespace stamped onto every Put. Required when Create is called; Mkdir/Rmdir/Unlink/Rename do not depend on it (they operate on existing artifacts).

func WithReadOnly 

func WithReadOnly() FSOpsOption

WithReadOnly forces every mutation to return ErrEditingDisabled regardless of EditingPolicy or the existence of a StoreClient.

func WithScratchDir 

func WithScratchDir(path string) FSOpsOption

WithScratchDir sets the directory for scratch files. Must exist and be writable. Scratch is created lazily on the first Create/Open(write).

func WithScratchQuota 

func WithScratchQuota(bytes int64) FSOpsOption

WithScratchQuota caps the total bytes held by active scratch files. 0 means unlimited.

func WithStore 

func WithStore(s StoreClient) FSOpsOption

type File 

type File interface {
	io.ReaderAt
	io.WriterAt
	io.Closer
	Sync() error
	Truncate(size int64) error
}

File is the handle returned by Open/Create. It bundles random I/O, sync, in-place truncate, and Close — together they cover what FUSE write paths need.

Stage 4a: only read-only handles are produced (via Open with OpenReadOnly). Write methods (WriteAt, Truncate, Sync) on a read-only handle return ErrEditingDisabled.

type FileInfo 

type FileInfo struct {
	Name    string
	Path    string
	Size    int64
	Mode    uint32
	UID     uint32
	GID     uint32
	ModTime time.Time
	IsDir   bool

	// ArtifactID is the underlying artifact's id when the
	// FileInfo describes a file backed by a real artifact.
	// Empty for virtual directories and synthetic entries
	// that have no artifact identity. Surfaced by the web
	// browser to build info-links into the artifact details
	// page; ignored by FUSE/WebDAV which don't need it.
	ArtifactID domain.ArtifactID

	// MIME carries the MIME type recorded in fsmeta.MIME, if
	// any. Empty when the artifact has no fsmeta payload or
	// the payload didn't set a MIME. Surfaced by the web
	// browser to decide whether a file is safe to advertise
	// via an inline [view] link.
	MIME string
}

FileInfo is the POSIX-shaped descriptor that Stat/Listdir returns. Built from FilesystemFacet plus FSOps defaults.

type FileInfoSeq 

type FileInfoSeq = iter.Seq2[FileInfo, error]

FileInfoSeq is a stream of FileInfo with optional error per position; mirrors NodeSeq.

type FilesystemFacet 

type FilesystemFacet struct {
	Name    string
	Path    string
	IsDir   bool
	Size    int64
	ModTime time.Time
}

FilesystemFacet is the minimal POSIX-shaped view of a Node: what every consumer of the View needs regardless of schema. Always populated, including for virtual directories synthesised from grouping.

POSIX attributes (mode, uid, gid) are NOT in this facet: they belong to the filesystem schema (fsmeta.FileSystem) and are materialised by FSOps when the consumer crosses the transport boundary (FUSE/WebDAV). Storing them here would commit View to a single schema and pre-empt the consumer's policy. Email- or other non-POSIX projections reuse Node without paying for unused POSIX fields.

type Locations 

type Locations struct {
	ByArtifact  string
	BySession   string
	ByNamespace string
	ByDate      string
	ByPath      string // "" if orphaned
	ByOrphaned  string // "" if placed under byPath
}

Locations bundles every tree-placement of one artifact — what the web "Locations" panel shows. Empty fields (e.g. PathByPath="" for orphaned, PathByOrphaned="" for placed) signal "this tree doesn't carry this artifact".

type MetadataSource 

type MetadataSource interface {
	// Metadata returns the metadata bytes for the given
	// artifact id. (raw, true, nil) — found; (nil, false, nil)
	// — not present; the third return is reserved for
	// infrastructure errors (DB I/O failure).
	Metadata(id domain.ArtifactID) (json.RawMessage, bool, error)
}

MetadataSource is the contract View backfill uses to fetch manifest metadata in bulk. Source.Walk normally returns stripped manifests (the index is the routing layer, not the content store) — without a MetadataSource, View has to round- trip Source.Get for every manifest just to recover Metadata, which is N+1.

A MetadataSource — typically an index extension that persisted metadata at write time — answers Metadata(id) in O(log N) from local storage. View.backfill skips the per-manifest Get when one is configured.

The interface is schema-agnostic: the source returns the raw json.RawMessage that Manifest.Metadata held at write time. View consumers (FSOps and friends) decode into whatever schema they care about (fsmeta, email, archive).

type Node 

type Node struct {
	FS       FilesystemFacet
	Artifact *ArtifactFacet
	Storage  *StorageFacet
}

Node is one entry in the View. FS is always populated; Artifact for files; Storage only on a Curator source.

type NodeSeq 

type NodeSeq = iter.Seq2[Node, error]

NodeSeq is a sequence of nodes with an optional error per position (the standard iter.Seq2 pattern for fallible streams).

type OpenMode 

type OpenMode int

OpenMode is the access mode for Open. Bit-flags: combine with OR (e.g. OpenReadWrite | OpenAppend). 

const (
	OpenReadOnly  OpenMode = 0
	OpenWriteOnly OpenMode = 1
	OpenReadWrite OpenMode = 2
	OpenAppend    OpenMode = 4
)

type PathCollisionPayload 

type PathCollisionPayload struct {
	Path   string
	Winner domain.ArtifactID
	Loser  domain.ArtifactID
}

PathCollisionPayload carries the resolution data of a path collision. Winner is the artifact now holding the path; Loser is the artifact that lost it (still reachable through by-artifact).

type PathFallback 

type PathFallback string

PathFallback governs how artifacts without a resolver path are surfaced. 

const (
	// FallbackOrphaned (default) — no by-path entry. Artifacts
	// remain reachable through by-artifact and the orphaned/
	// service tree.
	FallbackOrphaned PathFallback = "orphaned"

	// FallbackSynthetic — artifacts get a synthetic path derived
	// from namespace + session + short ArtifactID. Mixes real and
	// synthetic paths in by-path; for debugging on noisy stores.
	FallbackSynthetic PathFallback = "synthetic"
)

type PathResolver 

type PathResolver func(m domain.Manifest) (path string, ok bool)

PathResolver extracts a virtual path from a manifest. Implementing it is how a host plugs a metadata schema into the projection.

Returns:

  • (path, true) when the artifact carries a recognised schema and a valid path.
  • ("", false) when the artifact is opaque to this resolver.

Pure: the same Manifest must always produce the same result — the View caches the decision and any non-determinism shows up as stale-tree bugs.

Standard implementation for the filesystem schema is projection/fsmeta.Resolver.

type ProjectionSource 

type ProjectionSource interface {
	Walk(ctx context.Context, namespace string, cb func(domain.Manifest) error) error
	Get(ctx context.Context, id domain.ArtifactID, opts domain.GetOptions) (core.ReadHandle, error)
}

ProjectionSource is the minimal contract for an artifact source supplying a View. Satisfied by core.DataStore and curator.Curator without additional code. Extended abilities (StorageFacet population) are detected on the View side via a type assertion when needed — keeps curator out of projection's import graph.

type RelatedArtifact 

type RelatedArtifact struct {
	ArtifactID domain.ArtifactID
	Path       string // by-path placement; empty if orphaned
	Namespace  string
	SessionID  string
	CreatedAt  time.Time
}

RelatedArtifact is the small descriptor returned by RelatedByBlobRef. Carries enough fields for a UI to render "where else this blob lives" without forcing the caller to follow up with manifest lookups.

type RootView 

type RootView string

RootView selects which logical tree appears at the root of the View. The chosen tree does not duplicate into the service directory of a FUSE mount. 

const (
	RootByPath      RootView = "by-path" // default
	RootBySession   RootView = "by-session"
	RootByNamespace RootView = "by-namespace"
	RootByDate      RootView = "by-date"
	RootByArtifact  RootView = "by-artifact"
	RootByOrphaned  RootView = "by-orphaned"
)

type RouteKind 

type RouteKind int

RouteKind tags the destination of a routed path. The FUSE dispatcher branches on this value to choose between FSOps (for the root view) and direct View access (for the service trees), plus virtual files (stats) and the raw mirror. 

const (
	// RouteRoot — the path lives in the configured root tree and
	// goes through FSOps (read-write per editing policy).
	RouteRoot RouteKind = iota

	// RouteServiceTree — the path lives in a _scrinium/<treeName>
	// subtree, served read-only directly from the View.
	RouteServiceTree

	// RouteServiceRoot — the path is exactly the service-prefix
	// directory (e.g. "_scrinium"). The dispatcher exposes a
	// synthesised directory listing of the enabled service trees.
	RouteServiceRoot

	// RouteStatsFile — the path is _scrinium/stats; the dispatcher
	// returns a virtual file whose contents are generated on
	// each read.
	RouteStatsFile

	// RouteRawMirror — the path is under _scrinium/raw/. The
	// dispatcher serves it directly from the store directory on
	// disk (read-only).
	RouteRawMirror

	// RouteRejected — the path is reserved or otherwise refused
	// (e.g. the root component is a duplicate of the service
	// prefix and cannot be created).
	RouteRejected
)

type RouteTarget 

type RouteTarget struct {
	Kind RouteKind

	// Tree is the View tree to query when Kind == RouteRoot or
	// RouteServiceTree. Unused otherwise.
	Tree RootView

	// SubPath is the path *inside* Tree. For RouteRoot it's the
	// input path verbatim; for RouteServiceTree it's the input
	// path with the "_scrinium/<treeName>/" prefix stripped.
	// Empty string means "tree root".
	SubPath string

	// RawSubPath, when Kind == RouteRawMirror, is the path inside
	// the store directory (the part after "_scrinium/raw/").
	RawSubPath string
}

RouteTarget is the result of Route. The fields meaningful for a given Kind are listed in the Kind doc above.

func Route 

func Route(path string, cfg RoutingConfig) (RouteTarget, error)

Route classifies an incoming filesystem path. The path is slash-separated, no leading slash (consistent with the projection package's convention). An empty path is the mount root.

Routing rules:

  • "" → RouteRoot at the configured RootView, SubPath="".
  • "<servicePrefix>" → RouteServiceRoot.
  • "<servicePrefix>/<treeName>[/...]" → RouteServiceTree at the corresponding RootView (or RouteStatsFile, RouteRawMirror for the special leaves), provided the tree is enabled in cfg. Disabled tree → RouteRejected.
  • everything else → RouteRoot, SubPath = path.

Service prefix in non-root positions is allowed: "photos/_scrinium" is a regular path component. Only the first segment matters.

The function does no I/O and does not consult the View; it is pure with respect to its inputs.

type RoutingConfig 

type RoutingConfig struct {
	// ServicePrefix is the root component reserved for service
	// trees. Empty disables the service surface entirely; every
	// path then routes to RouteRoot.
	ServicePrefix string

	// RootView selects the tree that backs RouteRoot.
	RootView RootView

	// Show* mirror Config.Show* flags. A path under a hidden
	// service tree returns RouteRejected (the dispatcher then
	// surfaces ENOENT).
	ShowStats       bool
	ShowByArtifact  bool
	ShowOrphaned    bool
	ShowByDate      bool
	ShowBySession   bool
	ShowByNamespace bool
	ShowRaw         bool

	// UnprefixedServiceTrees, when true, exposes service tree
	// names (by-path, by-date, by-session, by-namespace,
	// by-artifact, orphaned, stats, raw) at the root of the
	// path namespace — without the ServicePrefix wrapper.
	//
	// Only honoured when ServicePrefix is empty: the
	// configurations are mutually exclusive. Surfaces that
	// dedicate the entire URL space to diagnostics (webview)
	// turn this on; surfaces sharing root with user content
	// (webdav, fuse) keep ServicePrefix non-empty and leave
	// this off.
	//
	// Caveat: with UnprefixedServiceTrees on, names like
	// "by-date" cannot exist as ordinary path components
	// (they always route to the service tree). The webview
	// surface accepts this trade-off because it never
	// surfaces user content under the root anyway.
	UnprefixedServiceTrees bool
}

RoutingConfig captures the routing-relevant subset of Config. Decoupled so routing.go does not depend on the full Config definition (and tests can construct it cheaply).

type SearchResult 

type SearchResult struct {
	ArtifactID  domain.ArtifactID
	Path        string // by-path placement; empty if orphaned
	Namespace   string
	SessionID   string
	CreatedAt   time.Time
	MIME        string // from fsmeta when present
	MatchReason string // "path" | "namespace" | "id"
}

SearchResult is one hit returned by View.Search. Carries enough fields to render a result row without forcing the caller to follow up with manifest lookups.

type SourceKind 

type SourceKind string

SourceKind labels the type of source backing a View. Governs whether StorageFacet is meaningful for a Node. 

const (
	// SourceKindStore — a single core.DataStore. StorageFacet is
	// always nil.
	SourceKindStore SourceKind = "store"

	// SourceKindCurator — a Curator with MultistoreIndex.
	// StorageFacet is populated.
	SourceKindCurator SourceKind = "curator"
)

type StorageFacet 

type StorageFacet struct {
	StoreID   domain.StoreID
	Workspace domain.Workspace
	IsTransit bool
	RefCount  int
}

StorageFacet carries placement data within a Curator stack. Populated only when SourceKind == Curator.

type StoreClient 

type StoreClient interface {
	Put(ctx context.Context, a domain.Artifact, opts domain.PutOptions) (domain.ArtifactID, error)
	Delete(ctx context.Context, id domain.ArtifactID) error
	Get(ctx context.Context, id domain.ArtifactID, opts domain.GetOptions) (core.ReadHandle, error)
}

StoreClient is the write-side surface FSOps depends on. Defined here rather than reusing core.Store so that:

  • the dependency is minimal — FSOps does not need namespace enumeration, lifecycle, crypto admin, or any of core's other surface;
  • tests can supply a fake without implementing every method of core.Store.

core.Store satisfies this interface naturally (subset typing in Go).

type View 

type View struct {
	// Public, read-only after NewView returns.
	Source    SourceKind
	CreatedAt time.Time
	Stats     ViewStats
	// contains filtered or unexported fields
}

View is the read side of the projection. It holds five parallel in-memory trees (by-path, by-session, by-namespace, by-date, by-artifact) populated by backfill at NewView time.

Concurrency: every public method takes the View's RWMutex. Add/Remove/Move take the write lock; readers take a read lock. All read methods build a private copy of any state they iterate over, so callers can do their own work without holding the projection lock.

Tree access: trees are exposed through symmetric methods — GetByPath/ListByPath/WalkByPath/OpenByPath, GetBySession/..., etc. The View has no notion of a single "current root"; the transport layer (FUSE, WebDAV) decides which tree to surface in the mount root and which to hide under a service prefix.

func NewView 

func NewView(ctx context.Context, source ProjectionSource, opts ...ViewOption) (*View, error)

NewView constructs a View by walking source and populating every tree. Backfill is synchronous: NewView returns only after the source has been fully traversed.

Default options:

  • root view: RootByPath (informational only)
  • fallback: FallbackOrphaned
  • filter: empty

EventBus is optional via WithEventBus; without it the View silently produces no events.

func (*View) Add 

func (v *View) Add(m domain.Manifest) error

Add registers a new manifest, mirroring backfill's per-manifest path. Used by FSOps after Store.Put. Concurrent with reads; holds the write lock.

Returns ErrViewClosed if the View is closed. Otherwise nil — classification cannot fail for a valid manifest (the input itself is what the source produced).

func (*View) Close 

func (v *View) Close() error

Close marks the View closed. Idempotent. Subsequent reads return ErrViewClosed.

func (*View) GetByArtifact 

func (v *View) GetByArtifact(path string) (Node, error)

func (*View) GetByDate 

func (v *View) GetByDate(path string) (Node, error)

func (*View) GetByNamespace 

func (v *View) GetByNamespace(path string) (Node, error)

func (*View) GetByOrphaned 

func (v *View) GetByOrphaned(path string) (Node, error)

func (*View) GetByPath 

func (v *View) GetByPath(path string) (Node, error)

func (*View) GetBySession 

func (v *View) GetBySession(path string) (Node, error)

func (*View) GetIn 

func (v *View) GetIn(rv RootView, path string) (Node, error)

GetIn dispatches GetByX based on rv.

func (*View) ListByArtifact 

func (v *View) ListByArtifact(path string) NodeSeq

func (*View) ListByDate 

func (v *View) ListByDate(path string) NodeSeq

func (*View) ListByNamespace 

func (v *View) ListByNamespace(path string) NodeSeq

func (*View) ListByOrphaned 

func (v *View) ListByOrphaned(path string) NodeSeq

func (*View) ListByPath 

func (v *View) ListByPath(path string) NodeSeq

func (*View) ListBySession 

func (v *View) ListBySession(path string) NodeSeq

func (*View) ListIn 

func (v *View) ListIn(rv RootView, path string) NodeSeq

ListIn dispatches ListByX based on rv.

func (*View) LookupLocations 

func (v *View) LookupLocations(id domain.ArtifactID) (Locations, bool)

LookupLocations returns the per-tree paths of an artifact. Used by the web artifact details page to surface "show me where this lives" links into each tree's listing.

(zero, false) if the artifact isn't tracked.

func (*View) Move 

func (v *View) Move(oldPath, newPath string, m domain.Manifest) error

Move atomically replaces an old artifact with a new one — used by FSOps to emulate rename. The old artifact's by-path entry is dropped (with collision re-election), and the new manifest is added through the standard Add path.

oldPath/newPath are passed for documentation and future use (FSOps wants to log the user-level rename); the actual location in by-path comes from the new manifest's resolver result.

func (*View) OpenByArtifact 

func (v *View) OpenByArtifact(ctx context.Context, path string, opts domain.GetOptions) (core.ReadHandle, error)

func (*View) OpenByDate 

func (v *View) OpenByDate(ctx context.Context, path string, opts domain.GetOptions) (core.ReadHandle, error)

func (*View) OpenByNamespace 

func (v *View) OpenByNamespace(ctx context.Context, path string, opts domain.GetOptions) (core.ReadHandle, error)

func (*View) OpenByOrphaned 

func (v *View) OpenByOrphaned(ctx context.Context, path string, opts domain.GetOptions) (core.ReadHandle, error)

func (*View) OpenByPath 

func (v *View) OpenByPath(ctx context.Context, path string, opts domain.GetOptions) (core.ReadHandle, error)

func (*View) OpenBySession 

func (v *View) OpenBySession(ctx context.Context, path string, opts domain.GetOptions) (core.ReadHandle, error)

func (*View) OpenIn 

func (v *View) OpenIn(ctx context.Context, rv RootView, path string, opts domain.GetOptions) (core.ReadHandle, error)

OpenIn dispatches OpenByX based on rv.

func (*View) RelatedByBlobRef 

func (v *View) RelatedByBlobRef(blobRef domain.BlobRef, exclude domain.ArtifactID) []RelatedArtifact

RelatedByBlobRef returns every artifact that shares the given BlobRef, excluding the artifact identified by `exclude`. Useful for the "this blob is also used here" web view — one of the few introspections specific to a CAS store.

Implementation is a linear scan of the artifacts map. That scales to roughly 100K artifacts inside a single web request without blocking; bigger stores will want an index by blob_ref. We accept the linearity now because the alternative (push the query into core.Store/index) costs more wiring than the value justifies at this scale.

Concurrency: holds RLock for the scan duration. A long-running scan would block writers; the 100K-artifact budget keeps it under ~10ms in practice.

func (*View) Remove 

func (v *View) Remove(id domain.ArtifactID) error

Remove drops every entry of the artifact from every tree. Handles by-path collision re-election when the removed artifact was the current owner of a path.

Idempotent: Remove for an unknown ArtifactID is a no-op.

func (*View) RootView 

func (v *View) RootView() RootView

RootView returns the configured root tree. It is informational metadata: the View itself does not hide other trees, but transports (FUSE, FSOps) read this to decide which tree to surface in the mount root and which to relegate to the service directory.

Stable for the lifetime of the View — the option is set at NewView time and never mutated.

func (*View) Search 

func (v *View) Search(query string, limit int) []SearchResult

Search scans the View for artifacts matching the query. Substring matching, case-insensitive, against:

  • the artifact's by-path placement (covers fsmeta names);
  • the namespace field;
  • an exact ArtifactID match (so users can paste an id and jump straight to it).

limit caps the result count; passing 0 disables the cap (use only for diagnostic flows). Order matches the scan order over the artifacts map — random-but-stable within a single View state. Callers sort if they need a specific order.

Implementation is the same linear scan as RelatedByBlobRef: O(N) under RLock, fast for stores up to ~100K artifacts. Beyond that, we'd want an actual search index — see backlog.

func (*View) WalkByArtifact 

func (v *View) WalkByArtifact(prefix string) NodeSeq

func (*View) WalkByDate 

func (v *View) WalkByDate(prefix string) NodeSeq

func (*View) WalkByNamespace 

func (v *View) WalkByNamespace(prefix string) NodeSeq

func (*View) WalkByOrphaned 

func (v *View) WalkByOrphaned(prefix string) NodeSeq

func (*View) WalkByPath 

func (v *View) WalkByPath(prefix string) NodeSeq

func (*View) WalkBySession 

func (v *View) WalkBySession(prefix string) NodeSeq

type ViewFilter 

type ViewFilter struct {
	Namespace string
	SessionID string
	Prefix    string
}

ViewFilter restricts which manifests are admitted into the View during backfill. All non-zero conditions combine by AND.

type ViewOption 

type ViewOption func(*viewOptions)

ViewOption is the option type passed to NewView.

func WithEventBus 

func WithEventBus(bus event.EventBus) ViewOption

WithEventBus wires an event bus that receives EventViewRebuilt after backfill and EventPathCollision on each by-path collision. nil bus (the default when this option is not used) silently drops events.

func WithFSIndex 

func WithFSIndex(fsidx MetadataSource) ViewOption

WithFSIndex is a typed convenience for the projection/fsindex case: pass the registered *fsindex.Extension and it doubles as a MetadataSource. Equivalent to WithMetadataSource(fsidx).

Implemented at the package level via an interface to avoid taking a hard dependency on projection/fsindex from projection — fsindex imports projection's fsmeta, so a back- edge would cycle.

func WithFallback 

func WithFallback(fb PathFallback) ViewOption

WithFallback governs how artifacts without a resolver path are represented. Default: FallbackOrphaned.

func WithFilter 

func WithFilter(f ViewFilter) ViewOption

WithFilter restricts the View to a subset of the source. Use for namespace-scoped or session-scoped views.

func WithMetadataSource 

func WithMetadataSource(ms MetadataSource) ViewOption

WithMetadataSource installs a bulk metadata source for backfill. When set, View.backfill consults the source instead of round-tripping Source.Get for each manifest. A miss (artifact not indexed by the source) falls back to Source.Get transparently — the option is a performance hint, not a correctness requirement.

func WithPathResolver 

func WithPathResolver(r PathResolver) ViewOption

WithPathResolver registers the path-extraction function. Without it the by-path tree contains only artifacts produced by the fallback (when FallbackSynthetic) or is empty.

func WithRootView 

func WithRootView(rv RootView) ViewOption

WithRootView selects the tree that occupies the View root. The default is RootByPath. The choice is informational for the View itself; transports (FUSE) react to it by hiding the same tree from the service directory.

type ViewRebuiltPayload 

type ViewRebuiltPayload struct {
	Duration  time.Duration
	NodeCount int64
}

ViewRebuiltPayload carries timing and counts of a backfill completion. NodeCount is the total number of nodes across every tree (one file artifact may appear under several trees).

type ViewStats 

type ViewStats struct {
	TotalNodes     int64
	TotalBytes     int64
	SessionCount   int64
	NamespaceCount int64
	OrphanedCount  int64
	CollisionCount int64
	ByStore        map[string]int64
	TransitCount   int64
}

ViewStats holds the aggregate counters of a View. Populated during backfill and updated by Add/Remove/Move calls.

Source Files

View all Source files

Directories

Path Synopsis
Package fsindex is an index extension that persists the fsmeta payload of every artifact whose Manifest.Metadata uses the filesystem schema.
 Package fsindex is an index extension that persists the fsmeta payload of every artifact whose Manifest.Metadata uses the filesystem schema.
Package fsmeta is the standard filesystem schema for Manifest.Metadata.
 Package fsmeta is the standard filesystem schema for Manifest.Metadata.
Package vfs is the read/write virtual filesystem layer over a projection.View.
 Package vfs is the read/write virtual filesystem layer over a projection.View.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.