filesystem

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2026 License: MIT Imports: 11 Imported by: 0

README

filesystem

CI CodeQL Coverage Mutation Documentation Go Reference Release Go License

filesystem is a capability-based, streaming filesystem abstraction for Go. It supports local files, deterministic in-memory storage, Amazon S3, Cloudflare R2, SFTP, and FTP without claiming that those backends provide the same guarantees.

store := memory.New()
path := filesystem.MustParsePath("documents/report.txt")

_, err := store.Write(ctx, path, source, filesystem.WriteOptions{
    ContentType: "text/plain",
})
if err != nil {
    return err
}

stream, err := store.Open(ctx, path)
if err != nil {
    return err
}
defer stream.Close()
_, err = io.Copy(destination, stream)

Consumers depend on the smallest interface they need:

func download(ctx context.Context, reader filesystem.Reader, path filesystem.Path) error

Incremental producers use filesystem.WriteOpener and must check Close, which waits for final publication:

writer, err := store.OpenWriter(ctx, path, filesystem.WriteOptions{})
if err != nil {
    return err
}
if _, err := io.Copy(writer, source); err != nil {
    _ = writer.Close()
    return err
}
if err := writer.Close(); err != nil {
    return err
}

Inspect Capabilities() before offering backend-dependent behavior. Calling an unsupported operation returns a typed *filesystem.CapabilityError that wraps filesystem.ErrUnsupportedCapability.

Design guarantees

  • Logical paths are root-relative, slash-separated, and traversal-safe.
  • Reads and writes stream through io.Reader and io.Writer; whole-object buffering is never part of the root contract.
  • Listings are closeable iterators and every network adapter applies a bound.
  • S3/R2 metadata has configurable entry and byte bounds.
  • Remote adapters validate credentials, host identity, transport support, and root settings before use; unsupported FTPS configurations fail before dial.
  • Retry and atomicity behavior is adapter-specific and documented explicitly.
  • filesystem.NewIOFS exposes read-only capabilities through standard io/fs APIs.

See the capability matrix, adapter guide, decorator guide, operations guide, hardening matrix, and security policy. The module requires Go 1.26 or newer.

Status

The API is pre-1.0. Compatibility commitments and tested service versions are recorded in COMPATIBILITY.md. Google Cloud Storage and Azure Blob Storage are intentionally outside the initial release.

License

Licensed under the MIT License.

Ecosystem

Use the Golib documentation portal to choose companion packages, supported stacks, recipes, and operations guidance.

Documentation

Overview

Package filesystem defines capability-based interfaces and shared types for storage adapters.

Example 
package main

import (
	"context"
	"fmt"
	"io"
	"strings"

	filesystem "github.com/faustbrian/go-filesystem"
	"github.com/faustbrian/go-filesystem/memory"
)

func main() {
	ctx := context.Background()
	store := memory.New()
	logicalPath := filesystem.MustParsePath("documents/report.txt")
	_, err := store.Write(ctx, logicalPath, strings.NewReader("report"), filesystem.WriteOptions{
		ContentType: "text/plain",
	})
	if err != nil {
		panic(err)
	}
	stream, err := store.Open(ctx, logicalPath)
	if err != nil {
		panic(err)
	}
	defer func() { _ = stream.Close() }()
	content, err := io.ReadAll(stream)
	if err != nil {
		panic(err)
	}
	fmt.Println(string(content))
}

Output:
report

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	// ErrUnsupportedCapability means an adapter cannot safely provide an
	// operation with its documented semantics.
	ErrUnsupportedCapability = errors.New("unsupported filesystem capability")
	// ErrNotFound means the requested logical path does not exist.
	ErrNotFound = errors.New("filesystem path not found")
	// ErrAlreadyExists means an operation required an absent destination.
	ErrAlreadyExists = errors.New("filesystem path already exists")
	// ErrInvalidRange means a byte range is malformed or unsatisfiable.
	ErrInvalidRange = errors.New("invalid filesystem byte range")
	// ErrPreconditionFailed means a conditional request was not satisfied.
	ErrPreconditionFailed = errors.New("filesystem precondition failed")
	// ErrPartialWrite means a backend accepted only part of a write.
	ErrPartialWrite = errors.New("partial filesystem write")
	// ErrResourceLimit means remote or caller-controlled data exceeded an
	// adapter's configured allocation or cardinality bound.
	ErrResourceLimit = errors.New("filesystem resource limit exceeded")
)
View Source 
var ErrInvalidPath = errors.New("invalid filesystem path")

ErrInvalidPath identifies a logical path that is empty, escapes its logical root, or has platform-specific or otherwise ambiguous syntax.

Functions

func Unsupported 

func Unsupported(adapter string, capability Capability, operation Operation) error

Unsupported constructs a typed unsupported-capability error.

Types

type ByteRange 

type ByteRange struct {
	// Offset is the zero-based first byte.
	Offset int64
	// Length is the maximum number of bytes to return.
	Length int64
}

ByteRange identifies a half-open byte range [Offset, Offset+Length).

type Capability 

type Capability string

Capability identifies an independently supported filesystem operation. 

const (
	// CapabilityRead opens complete objects for streaming reads.
	CapabilityRead Capability = "read"
	// CapabilityWrite consumes readers and publishes objects.
	CapabilityWrite Capability = "write"
	// CapabilityDelete removes objects.
	CapabilityDelete Capability = "delete"
	// CapabilityList enumerates bounded directory snapshots.
	CapabilityList Capability = "list"
	// CapabilityStat retrieves object metadata.
	CapabilityStat Capability = "stat"
	// CapabilityCopy copies objects within one adapter.
	CapabilityCopy Capability = "copy"
	// CapabilityMove renames or moves objects with documented atomicity.
	CapabilityMove Capability = "move"
	// CapabilityRangeRead opens bounded object byte ranges.
	CapabilityRangeRead Capability = "range-read"
	// CapabilityMetadata reads and updates user-controlled metadata.
	CapabilityMetadata Capability = "metadata"
	// CapabilityChecksum returns explicitly identified digest algorithms.
	CapabilityChecksum Capability = "checksum"
	// CapabilityTemporaryURL signs time-limited read URLs.
	CapabilityTemporaryURL Capability = "temporary-url"
	// CapabilityVisibility reads and changes coarse object visibility.
	CapabilityVisibility Capability = "visibility"
	// CapabilityMultipart uses bounded multipart publication for large writes.
	CapabilityMultipart Capability = "multipart-upload"
	// CapabilityStreamingWrite opens incremental io.Writer uploads.
	CapabilityStreamingWrite Capability = "streaming-write"
)

type CapabilityError 

type CapabilityError struct {
	// Adapter identifies the implementation rejecting the operation.
	Adapter string
	// Capability is the unavailable semantic contract.
	Capability Capability
	// Operation is the attempted public action.
	Operation Operation
}

CapabilityError describes a requested operation that an adapter cannot safely implement.

func (*CapabilityError) Error 

func (e *CapabilityError) Error() string

Error implements error.

func (*CapabilityError) Unwrap 

func (e *CapabilityError) Unwrap() error

Unwrap allows errors.Is to classify all capability errors.

type CapabilityReporter 

type CapabilityReporter interface {
	Capabilities() CapabilitySet
}

CapabilityReporter exposes an adapter's supported operations.

type CapabilitySet 

type CapabilitySet struct {
	// contains filtered or unexported fields
}

CapabilitySet is an immutable collection of supported capabilities.

func NewCapabilitySet 

func NewCapabilitySet(capabilities ...Capability) CapabilitySet

NewCapabilitySet constructs a deterministic capability set. It panics for unknown values because advertising an invented capability is a programming error in an adapter.

func (CapabilitySet) List 

func (s CapabilitySet) List() []Capability

List returns a copy in the order capabilities were declared.

func (CapabilitySet) Supports 

func (s CapabilitySet) Supports(capability Capability) bool

Supports reports whether capability is advertised.

type Checksum 

type Checksum struct {
	// Algorithm identifies how Value was calculated.
	Algorithm ChecksumAlgorithm
	// Value is the lowercase hexadecimal digest.
	Value string
}

Checksum is a backend checksum and its unambiguous algorithm.

type ChecksumAlgorithm 

type ChecksumAlgorithm string

ChecksumAlgorithm identifies an explicitly requested digest algorithm. 

const (
	// ChecksumMD5 requests an MD5 digest.
	ChecksumMD5 ChecksumAlgorithm = "md5"
	// ChecksumSHA256 requests a SHA-256 digest.
	ChecksumSHA256 ChecksumAlgorithm = "sha256"
	// ChecksumCRC32C requests a CRC-32C Castagnoli digest.
	ChecksumCRC32C ChecksumAlgorithm = "crc32c"
)

type Checksummer 

type Checksummer interface {
	Checksum(context.Context, Path, ChecksumAlgorithm) (Checksum, error)
}

Checksummer retrieves a backend checksum without assuming algorithms match.

type Copier 

type Copier interface {
	Copy(context.Context, Path, Path, CopyOptions) error
}

Copier copies an object without implying atomicity.

type CopyOptions 

type CopyOptions struct {
	// Overwrite permits replacement where the adapter can guarantee it safely.
	Overwrite bool
}

CopyOptions controls destination replacement for copies.

type Deleter 

type Deleter interface {
	Delete(context.Context, Path) error
}

Deleter removes an object.

type Entry 

type Entry struct {
	// Path is the normalized logical entry path.
	Path Path
	// Kind distinguishes files from directories.
	Kind EntryKind
	// Size is the byte length for files.
	Size int64
	// Modified is the backend-reported modification time.
	Modified time.Time
}

Entry is one item produced by a listing.

type EntryIterator 

type EntryIterator interface {
	Next() bool
	Entry() Entry
	Err() error
	Close() error
}

EntryIterator permits adapters to page listings without unbounded buffering.

type EntryKind 

type EntryKind string

EntryKind distinguishes objects from logical directories. 

const (
	// EntryKindFile identifies an object containing bytes.
	EntryKindFile EntryKind = "file"
	// EntryKindDirectory identifies a real or synthesized directory.
	EntryKindDirectory EntryKind = "directory"
)

type IOFS 

type IOFS struct {
	// contains filtered or unexported fields
}

IOFS adapts read, stat, and list capabilities to Go's read-only io/fs contracts. Logical directories may be synthesized from object prefixes.

func NewIOFS 

func NewIOFS(reader Reader, statter Statter, lister Lister) *IOFS

NewIOFS constructs a read-only io/fs bridge. All three capabilities are required because io/fs directories must support Stat and ReadDir.

func (*IOFS) Open 

func (f *IOFS) Open(name string) (fs.File, error)

Open implements fs.FS.

func (*IOFS) ReadDir 

func (f *IOFS) ReadDir(name string) ([]fs.DirEntry, error)

ReadDir implements fs.ReadDirFS.

func (*IOFS) Stat 

func (f *IOFS) Stat(name string) (fs.FileInfo, error)

Stat implements fs.StatFS.

type ListOptions 

type ListOptions struct {
	// Recursive includes descendants rather than only direct children.
	Recursive bool
	// Limit bounds returned entries; zero selects the adapter maximum.
	Limit int
}

ListOptions bounds and shapes a listing.

type Lister 

type Lister interface {
	List(context.Context, Path, ListOptions) (EntryIterator, error)
}

Lister returns a bounded, closeable iterator over logical entries.

type Metadata 

type Metadata struct {
	// Path is the normalized logical object path.
	Path Path
	// Kind distinguishes files from directories.
	Kind EntryKind
	// Size is the byte length for files.
	Size int64
	// Modified is the backend-reported modification time.
	Modified time.Time
	// ETag is the opaque backend entity tag and is not assumed to be a digest.
	ETag string
	// ContentType is the backend-reported media type.
	ContentType string
	// UserMetadata is a defensive copy of supported custom metadata.
	UserMetadata map[string]string
	// Visibility is populated only by adapters that support it.
	Visibility Visibility
}

Metadata describes an object without assuming every backend populates every field. Optional values are represented by their zero values.

type MetadataSetter 

type MetadataSetter interface {
	SetMetadata(context.Context, Path, map[string]string) error
}

MetadataSetter replaces user-controlled metadata where supported.

type MoveOptions 

type MoveOptions struct {
	// Overwrite permits replacement where the adapter can guarantee it safely.
	Overwrite bool
}

MoveOptions controls destination replacement for moves.

type Mover 

type Mover interface {
	Move(context.Context, Path, Path, MoveOptions) error
}

Mover moves an object according to adapter-specific documented guarantees.

type Operation 

type Operation string

Operation names a public filesystem action for errors and instrumentation. 

const (
	// OperationRead opens a complete object.
	OperationRead Operation = "read"
	// OperationWrite publishes an object.
	OperationWrite Operation = "write"
	// OperationDelete removes an object.
	OperationDelete Operation = "delete"
	// OperationList enumerates a logical directory.
	OperationList Operation = "list"
	// OperationStat retrieves object metadata.
	OperationStat Operation = "stat"
	// OperationCopy copies an object.
	OperationCopy Operation = "copy"
	// OperationMove moves or renames an object.
	OperationMove Operation = "move"
	// OperationRangeRead opens a byte range.
	OperationRangeRead Operation = "range-read"
	// OperationSetMetadata replaces user metadata.
	OperationSetMetadata Operation = "set-metadata"
	// OperationChecksum calculates or retrieves a digest.
	OperationChecksum Operation = "checksum"
	// OperationTemporaryURL signs a temporary read URL.
	OperationTemporaryURL Operation = "temporary-url"
	// OperationVisibility reads object visibility.
	OperationVisibility Operation = "visibility"
	// OperationSetVisibility changes object visibility.
	OperationSetVisibility Operation = "set-visibility"
)

type Path 

type Path struct {
	// contains filtered or unexported fields
}

Path is a normalized, root-relative logical filesystem path.

Paths always use forward slashes and never contain empty, current-directory, or parent-directory segments.

func MustParsePath 

func MustParsePath(value string) Path

MustParsePath is ParsePath that panics when value is invalid.

func ParsePath 

func ParsePath(value string) (Path, error)

ParsePath validates and normalizes a logical filesystem path.

func Root 

func Root() Path

Root returns the logical root. ParsePath intentionally rejects an empty string so accidental empty object names cannot be confused with this value.

func (Path) Base 

func (p Path) Base() string

Base returns the final path segment.

func (Path) Dir 

func (p Path) Dir() Path

Dir returns the path containing p. For a top-level path, Dir returns the zero Path, which represents the logical root for relationship operations.

func (Path) IsRoot 

func (p Path) IsRoot() bool

IsRoot reports whether p represents the logical root.

func (Path) Join 

func (p Path) Join(relative string) (Path, error)

Join appends a relative logical path to p.

func (Path) String 

func (p Path) String() string

String returns the normalized logical path.

type RangeReader 

type RangeReader interface {
	OpenRange(context.Context, Path, ByteRange) (io.ReadCloser, error)
}

RangeReader opens a bounded region of an object as a stream.

type Reader 

type Reader interface {
	Open(context.Context, Path) (io.ReadCloser, error)
}

Reader opens objects as streams. The caller must close successful results.

type Statter 

type Statter interface {
	Stat(context.Context, Path) (Metadata, error)
}

Statter retrieves object metadata.

type TemporaryURLOptions 

type TemporaryURLOptions struct {
	// DownloadName requests a response content-disposition filename.
	DownloadName string
	// ContentType requests a response content type.
	ContentType string
}

TemporaryURLOptions carries response properties understood by signing backends.

type TemporaryURLer 

type TemporaryURLer interface {
	TemporaryURL(context.Context, Path, time.Duration, TemporaryURLOptions) (string, error)
}

TemporaryURLer creates a time-limited URL where the backend supports it.

type Visibility 

type Visibility string

Visibility is a deliberately small cross-adapter visibility vocabulary. 

const (
	// VisibilityPrivate restricts access according to backend policy.
	VisibilityPrivate Visibility = "private"
	// VisibilityPublic permits public access according to backend policy.
	VisibilityPublic Visibility = "public"
)

type VisibilityManager 

type VisibilityManager interface {
	Visibility(context.Context, Path) (Visibility, error)
	SetVisibility(context.Context, Path, Visibility) error
}

VisibilityManager reads and changes coarse object visibility.

type WriteOpener 

type WriteOpener interface {
	OpenWriter(context.Context, Path, WriteOptions) (io.WriteCloser, error)
}

WriteOpener exposes a streaming writer when a backend can support one without weakening its cleanup or error guarantees.

type WriteOptions 

type WriteOptions struct {
	// ContentType records the media type where metadata is supported.
	ContentType string
	// Metadata is defensively copied user-controlled metadata.
	Metadata map[string]string
	// Visibility requests private or public publication where supported.
	Visibility Visibility
	// IfNoneMatch requires the destination to be absent where supported.
	IfNoneMatch bool
}

WriteOptions controls object creation without assigning unsupported semantics to all adapters.

type Writer 

type Writer interface {
	Write(context.Context, Path, io.Reader, WriteOptions) (Metadata, error)
}

Writer consumes a stream and does not require the whole object in memory.

Source Files

View all Source files

Directories

Path Synopsis
Package decorator provides composable policy wrappers for filesystem adapters without changing their underlying guarantees.
 Package decorator provides composable policy wrappers for filesystem adapters without changing their underlying guarantees.
Package fstest provides reusable adapter conformance tests.
 Package fstest provides reusable adapter conformance tests.
Package ftp provides a capability-based FTP adapter.
 Package ftp provides a capability-based FTP adapter.
internal
redact
Package redact removes credential-shaped values from errors while retaining their original cause for errors.Is and errors.As.
 Package redact removes credential-shaped values from errors while retaining their original cause for errors.Is and errors.As.
streamwriter
Package streamwriter adapts a reader-consuming upload into a closeable writer without buffering the whole object.
 Package streamwriter adapts a reader-consuming upload into a closeable writer without buffering the whole object.
Package local provides a filesystem adapter contained within a local directory root.
 Package local provides a filesystem adapter contained within a local directory root.
Package memory provides a deterministic, concurrency-safe in-memory filesystem adapter intended for tests and ephemeral workloads.
 Package memory provides a deterministic, concurrency-safe in-memory filesystem adapter intended for tests and ephemeral workloads.
Package r2 provides a first-class Cloudflare R2 adapter profile over the S3 transport.
 Package r2 provides a first-class Cloudflare R2 adapter profile over the S3 transport.
Package s3 provides an Amazon S3 adapter backed by AWS SDK for Go v2.
 Package s3 provides an Amazon S3 adapter backed by AWS SDK for Go v2.
Package sftp provides a capability-based SFTP adapter using pkg/sftp and golang.org/x/crypto/ssh.
 Package sftp provides a capability-based SFTP adapter using pkg/sftp and golang.org/x/crypto/ssh.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.