driver

Documentation

Overview

Package driver declares the Scrinium transport layer (L1): stateless adapters for accessing a Location of arbitrary nature (local filesystem, S3, network shares).

A Driver translates the unified set of I/O operations into the concrete backend's API, guarantees atomicity of data commit (Rename for POSIX, CompleteMultipartUpload for S3) and reports its abilities through Capabilities.

Concrete implementations (driver/localfs, driver/s3, driver/faulty) live in subpackages. This package contains only the interface contract and shared types.

DAG: driver imports event and the standard library. It does not import core, plugin, index, curator, agent.

Index

• func RegisterDialer(scheme string, d Dialer)
• func RegisteredSchemes() []string
• type CapabilityMask
  • func (m CapabilityMask) Has(c CapabilityMask) bool
• type Dialer
• type Driver
  • func DialDriver(uri string) (Driver, error)
• type FileInfo
• type ObjectMeta

Functions

func RegisterDialer

func RegisterDialer(scheme string, d Dialer)

RegisterDialer attaches a Dialer to a URI scheme. Called from package init() in driver implementations:

// driver/localfs/register.go
func init() { driver.RegisterDialer("file", openFileURI) }

Re-registering the same scheme overwrites the previous dialer and panics — this catches accidental double imports or scheme collisions at startup, before any user URIs are dialled.

func RegisteredSchemes

func RegisteredSchemes() []string

RegisteredSchemes returns the schemes currently registered. Sorted for deterministic output; useful in error messages and in --help output of binaries.

Types

type CapabilityMask

type CapabilityMask uint16

CapabilityMask is a bitmask of optional driver abilities. A driver declares its capabilities statically; the result of Capabilities() is stable for the lifetime of the instance.

const (
	// CapSlowRead indicates high latency or monetary cost of reads
	// (S3, Glacier, network shares over WAN). Curator uses this flag
	// to choose WriteStrategy: Auto becomes HostBuffered.
	CapSlowRead CapabilityMask = 1 << iota

	// CapBlockAlign512 indicates an optimal alignment of 512 bytes
	// (HDDs, classic SSDs).
	CapBlockAlign512

	// CapBlockAlign4096 indicates an optimal alignment of 4096 bytes
	// (NVMe, 4K-sector drives). Declaring this together with
	// CapBlockAlign512 is incorrect.
	CapBlockAlign4096

	// CapWatch indicates support for native new-file notifications
	// (inotify, FSEvents, S3 Event Notifications). Used by the
	// Continuous Watch Ingester.
	CapWatch

	// CapNativeChecksum indicates that the backend verifies data on
	// every read (S3 ETag, Btrfs/ZFS block checksums). Silent bit rot
	// is impossible; the Scrub Agent reduces the rate of explicit
	// verification accordingly.
	CapNativeChecksum
)

func (CapabilityMask) Has

func (m CapabilityMask) Has(c CapabilityMask) bool

Has reports whether the given flag is set in the mask.

type Dialer

type Dialer func(u *url.URL) (Driver, error)

Dialer opens a Driver from a parsed URI. Implementations live in scheme-specific packages and register themselves via RegisterDialer in their package init().

The URI is already parsed by DialDriver — the dialer receives the *url.URL and is responsible for translating scheme-specific parts (host, path, query) into driver configuration.

type Driver

type Driver interface {
	// I/O.
	Put(ctx context.Context, path string, r io.Reader) error
	Get(ctx context.Context, path string) (io.ReadCloser, error)
	ReadAt(ctx context.Context, path string, offset, size int64) (io.ReadCloser, error)
	Open(ctx context.Context, uri string) (io.ReadCloser, error)
	Remove(ctx context.Context, path string) error
	Rename(ctx context.Context, src, dst string) error
	Clone(ctx context.Context, src, dst string) error

	// Introspection.
	Stat(ctx context.Context, path string) (FileInfo, error)
	List(ctx context.Context, prefix string) ([]string, error)
	ListObjectsWithModTime(ctx context.Context, prefix string, since time.Time, cb func(ObjectMeta) error) error
	CountObjects(ctx context.Context, prefix string) (int64, error)

	// Maintenance.
	PruneEmptyDirs(ctx context.Context, root string) error
	Capabilities() CapabilityMask

	// Tombstone mechanics.
	MarkTombstone(ctx context.Context, path string) error
	IsTombstone(ctx context.Context, path string) (bool, error)
}

Driver is a stateless adapter for a single Location. It translates the unified set of operations into the concrete backend's API. One Location is served by exactly one Driver; attaching the same Location through two different Drivers is architecturally forbidden.

Put atomicity: a Get(path) running in parallel with Put never observes a partially written file — it sees either the previous content (or os.ErrNotExist) or the new content after Put succeeds.

The tombstone methods (MarkTombstone, IsTombstone) are mandatory for supporting Two-Phase Deletion in a multi-host environment.

Sentinel errors (ErrUnsupportedURIScheme for Open; ErrStopWalk for the ListObjectsWithModTime callback) live in the errs package and are matched via errors.Is.

func DialDriver

func DialDriver(uri string) (Driver, error)

DialDriver opens a Driver by URI. The scheme selects the implementation (registered via RegisterDialer); the rest of the URI is forwarded to the dialer.

Bare paths without a scheme — "/abs/path", "./relative", or "~/something" — are treated as file:// for backward compatibility with configs that pre-date URI support and for ergonomics on the CLI. Requires the file:// dialer to be registered (it is, by importing driver/localfs).

Returned errors:

• empty URI → "empty URI"
• URL parse failure → wrapped url.Parse error
• unregistered scheme → "scheme X not registered"
• dialer-specific failures → wrapped from the dialer

type FileInfo

type FileInfo struct {
	Size    int64
	ModTime time.Time
	IsDir   bool
}

FileInfo is the file metadata returned by Stat. The minimal set sufficient for existence, size, and modification-time checks.

type ObjectMeta

type ObjectMeta struct {
	Path         string
	Size         int64
	LastModified time.Time
	ETag         string
}

ObjectMeta is the extended object metadata used during iteration via ListObjectsWithModTime. The ETag field is optional: S3 fills it in, LocalFS leaves it empty.