filesystem

package
v0.14.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 23, 2026 License: MIT Imports: 28 Imported by: 0

Documentation

Overview

Package filesystem is the file contract: what a tenant uploaded, and where it went.

The shape, in one paragraph. A Disk is what an application calls, and every one of its methods takes an auth.Grant; an Adapter is what a driver implements, and it never hears of a tenant. Between the two sits Key, which turns a Grant and a key into the one stored path that Grant may reach. That split is the whole design: tenant isolation is a property of this package, not of each driver remembering to ask.

The contract lives in the collection and so does the driver that needs nothing installed: LocalFilesystemAdapter is a directory, and it is the one to develop against and to run a single machine on.

Object storage is github.com/arandu-io/hesape/filesystem/s3 -- the same contract over the S3 protocol, with Cloudflare R2 as the default, and a module of its own so that a project storing on disk does not carry it. In Go there is no optional dependency, which is the whole reason that line exists. Register it with Disks.Extend: there is no built-in S3 driver name, because the root module cannot import its own submodule.

That module speaks the protocol over net/http and signs its own requests, so importing it brings no cloud SDK with it, and it hands back no client object for a caller to reach past the adapter with.

A file is customer data, and a path without a tenant is a leak with a directory name -- which is why the Grant is on the collection's side of the split and not on the driver's, the same way it is for database.Repository and queue.Queue.

There is no symlink into a document root. Publishing a storage directory that way makes every stored file world-readable by URL and turns authorization into "hope nobody guesses the name". Here a file is served by a route, and the route runs a Policy like any other -- see Serve for the serving half and URLSigner for the link that stands in for a session.

The way in from a form is Upload, checked against UploadRules. It is the "own contract" that validation refuses uploads in favour of: a file is not in url.Values, and validating one inside the form pipeline would be a second way to validate.

Two file APIs, and the line between them

Disk is customer data: every method takes an auth.Grant and every path it builds starts with a tenant. Filesystem is the application's own files -- stubs, compiled views, session files, cache entries -- and takes plain paths, exactly as os.ReadFile does. The failure worth naming is putting an upload through the second one, where there is no prefix and therefore no isolation.

LockableFile is what makes the second one safe to share between processes, and Filesystem.SharedGet and Filesystem.Put with lock are the pattern.

Testing

A test builds the disk it wants and passes it, the same way it passes every other collaborator: 

adapter, err := filesystem.NewLocalFilesystemAdapter(t.TempDir())
if err != nil {
    t.Fatal(err)
}
disk := filesystem.NewDisk("local", adapter)

archiveInvoice(ctx, g, disk)

disk.AssertExists(ctx, t, g, "invoices/2026-114.pdf")

t.TempDir() is what keeps it isolated: the directory is made for this test and removed when it ends, so two tests running in parallel cannot see each other's files. A directory of your own in place of it is the version that outlives the run. The assertions are already on Disk: AssertExists, AssertMissing, AssertCount and AssertDirectoryEmpty.

URL, GetVisibility and MakeDirectory

All three are here, and each of them means less than its name suggests:

  • Disk.URL is the permanent public address of a file. It carries no authorization at all, so it answers ErrNoURL unless the disk was configured with one. The answer for a tenant's file is Disk.TemporaryURL or URLSigner.TemporaryURL, which expire.
  • Disk.GetVisibility and Disk.SetVisibility report and set what the STORE will hand out to somebody who never came through the application. They do not replace a Policy: every read here still takes a Grant.
  • Disk.MakeDirectory makes a directory on a driver that has them, and succeeds without doing anything on one that does not -- an object store has no directories, and a marker object standing in for one would put a key in every listing that Disk.Get then answers ErrNotFound for.

Index

Constants

View Source 
const (
	// VisibilityPublic is a file the store itself will hand to anybody who has
	// its address.
	VisibilityPublic = "public"
	// VisibilityPrivate is a file the store will not, which is the only correct
	// setting for anything a tenant uploaded.
	VisibilityPrivate = "private"
)

The two visibilities, as the strings they are stored and compared as.

Variables

View Source 
var ErrBadKey = errors.New("filesystem: the key escapes the tenant prefix")

ErrBadKey is returned for a key that would escape its tenant's prefix.

View Source 
var ErrBadURL = errors.New("filesystem: the link is not valid")

ErrBadURL is returned by URLSigner.Redeem for a link that was not issued here, was tampered with, or has run out. It unwraps encryption.ErrSignature, so a caller that wants to tell "expired" apart can still ask for encryption.ErrExpired.

View Source 
var ErrLockTimeout = errors.New("filesystem: the file is locked by another process")

ErrLockTimeout is returned when a lock was asked for without blocking and somebody else holds it.

View Source 
var ErrNoDisk = errors.New("filesystem: no such disk")

ErrNoDisk is returned by Disks.Disk for a name nobody registered.

View Source 
var ErrNoFileLocking = errors.New("filesystem: file locking is not available on this platform")

ErrNoFileLocking is returned on a platform whose kernel this package has no advisory lock for. Every unix does; it is here so the type still compiles where one does not, instead of the whole collection failing to build.

View Source 
var ErrNoPath = errors.New("filesystem: this driver's files have no path on this machine")

ErrNoPath is returned by Disk.Path on a driver whose files are not on this machine.

View Source 
var ErrNoTenant = errors.New("filesystem: the Grant carries no tenant, and a file without one belongs to everybody")

ErrNoTenant is returned when the Grant carries no tenant, or carries one that cannot be a path segment.

View Source 
var ErrNoURL = errors.New("filesystem: this disk cannot produce that address")

ErrNoURL is returned by Disk.URL, Disk.TemporaryURL and Disk.TemporaryUploadURL on a disk that cannot produce the address asked for.

View Source 
var ErrNoVisibility = errors.New("filesystem: this driver has no visibility to read or set")

ErrNoVisibility is returned by Disk.GetVisibility and Disk.SetVisibility on a driver with no such concept.

View Source 
var ErrNotFound = errors.New("filesystem: not found")

ErrNotFound is returned when a key does not exist for this tenant.

It is the same error whether the file is absent or belongs to somebody else, and that is deliberate: distinguishing them would tell a caller which keys exist in other tenants.

View Source 
var ErrRefusedUpload = errors.New("filesystem: the upload was refused")

ErrRefusedUpload is what every UploadRules failure unwraps to, so a handler answers "we cannot take this file" once instead of switching on four reasons.

Functions

func CleanKey 

func CleanKey(key string) (string, error)

CleanKey normalizes a key and refuses one that would escape.

This is the check that matters. A key is often a filename that came from an upload, and "../../../etc/passwd" is what an upload form eventually receives. Rejecting rather than sanitizing: a key that had to be rewritten to be safe is a key the caller did not mean, and silently storing it somewhere else is worse than an error.

func JoinPaths 

func JoinPaths(base string, paths ...string) string

JoinPaths joins path segments with a separator, dropping the empty ones.

The empty segments are dropped, so JoinPaths(base, "", "views") is base/views and not base//views -- which is a different string that names the same file, and therefore two cache keys.

func Key 

func Key(g auth.Grant, key string) (string, error)

Key builds the stored path for a key: <tenant>/<key>.

Every Disk method calls it, which is what makes tenant isolation a property of this package rather than of each Adapter implementation remembering. An Adapter is handed the result and never sees the Grant, so there is no driver in which the prefix can be forgotten.

The zero Grant carries no tenant, so it reaches no file. That is the same answer auth.Grant.Check gives, arrived at by a different route: a caller who authorized nothing has nothing to build a path out of.

func Serve 

func Serve(w http.ResponseWriter, r *http.Request, g auth.Grant, d *Disk, key string, opt ServeOptions) error

Serve writes a stored file to an HTTP response.

The range-aware read, the attachment and the inline render are one function, and which of the three it is comes from ServeOptions.

This is the half that makes "no symlink into a document root" a real answer rather than a refusal. A file is served by a route, the route runs a Policy like any other, and the Grant that Policy produced is what reaches the disk.

It writes nothing on error, so the caller decides the status -- ErrNotFound is a 404 and auth.ErrForbidden is a 403, and both are decisions the exception handler already knows how to make.

Three headers are not negotiable. The content type comes from the stored metadata, which came from the key and never from what an upload announced; X-Content-Type-Options is nosniff, so a browser cannot decide a .txt is HTML; and the disposition filename is escaped, because a filename with a quote and a semicolon in it is a header injection with a friendly name.

func TypeOf 

func TypeOf(key string) string

TypeOf infers a content type from a key's extension, falling back to a type that browsers download rather than render.

The fallback matters: serving an unknown file as text/html is how an upload becomes stored XSS.

Types

type Adapter 

type Adapter interface {
	// Put writes body at path, creating whatever it needs to.
	Put(ctx context.Context, path string, body io.Reader, contentType string) error
	// Get reads it back. It returns ErrNotFound when the path is not there.
	Get(ctx context.Context, path string) (File, error)
	// Stat answers the same metadata Get carries, without the body.
	Stat(ctx context.Context, path string) (Info, error)
	// Exists reports whether the path is there.
	Exists(ctx context.Context, path string) (bool, error)
	// Delete removes it. Removing what is not there is not an error.
	Delete(ctx context.Context, path string) error
	// List returns the stored paths under a prefix, in no promised order.
	List(ctx context.Context, prefix string) ([]string, error)
}

Adapter is what a driver implements.

It takes stored paths, already resolved by Key, and knows nothing about tenants or Grants. That is the point: the isolation is enforced once, in this package, instead of in each driver -- and a driver that forgets it cannot, because it is never told which tenant it is serving.

Six methods, because they are the six an object store and a directory both have. Copy, Move, DeleteDirectory and every listing narrower than this one are composed out of them by Disk, so a new driver is six methods and not twenty.

type Config 

type Config struct {
	// Driver names which creator builds the adapter: "local", "scoped", or a
	// name registered with [Disks.Extend].
	Driver string

	// Root is the directory the local driver writes under.
	Root string

	// URL is the public address files on this disk are reachable at, without a
	// trailing slash, and empty when they are not reachable without a session.
	//
	// Empty is the right answer for almost every disk here: a public address
	// carries no authorization, so a disk that has one is one whose contents are
	// world-readable to anybody holding the address. See [Disk.URL].
	URL string

	// Visibility is what a file written to this disk gets when the caller does
	// not say: [VisibilityPublic] or [VisibilityPrivate]. Empty means private,
	// which is the only default a tenant's file can have.
	Visibility string

	// Disk and Prefix configure the "scoped" driver: every key on the built disk
	// is stored under Prefix on the disk named by Disk.
	Disk   string
	Prefix string

	// ServeSigned makes [ServeFile] require a valid signature. See
	// [LocalFilesystemAdapter.ShouldServeSignedUrls].
	ServeSigned bool
}

Config is one disk's configuration.

It is typed rather than a map because a disk configured with a misspelled key is a disk that boots and then behaves like a different one -- the local driver silently rooted at the working directory, the public URL silently absent. A struct makes the misspelling a compile error, which is the whole argument for build configuration being typed here.

type DirectoryAware 

type DirectoryAware interface {
	// MakeDirectory creates the directory and its parents.
	MakeDirectory(ctx context.Context, storedPath string) error
	// DirectoryExists reports whether the directory is there.
	DirectoryExists(ctx context.Context, storedPath string) (bool, error)
}

DirectoryAware is the optional half of an Adapter whose store has real directories.

A directory on disk exists whether or not anything is in it. In an object store there are no directories at all: a key with slashes in it is one string, and "the invoices folder" is a prefix that exists exactly as long as a key under it does. Both readings are correct for their store, and this interface is what lets Disk give the same answer on either -- see Disk.MakeDirectory for what "the same answer" means when there is nothing to make.

type Disk 

type Disk struct {
	// contains filtered or unexported fields
}

Disk is what an application calls.

Every method takes an auth.Grant. That is not ceremony: it is the only thing standing between "the application stores files" and "any handler can read any customer's files by building a string". The Grant decides the tenant, the tenant decides the prefix, and the prefix is not reachable from the key.

Reads are not exempt. AllFiles, Get, Stat and Exists take a Grant for the same reason Put does, and a listing is the one call where forgetting it hands over the names of every file in the system.

func NewDisk 

func NewDisk(name string, a Adapter, cfg ...Config) *Disk

NewDisk names an adapter.

The name is what appears in an error and in `aru` output; it is not part of any path, so renaming a disk does not move a file.

The optional Config is what Disk.URL reads a public base address out of, and what Disk.GetConfig answers with. Passing none leaves it zero, which is a disk that has no public address -- and saying so is the correct answer for one.

func (*Disk) AllDirectories 

func (d *Disk) AllDirectories(ctx context.Context, g auth.Grant, directory string) ([]string, error)

AllDirectories returns every directory under a directory, at any depth.

func (*Disk) AllFiles 

func (d *Disk) AllFiles(ctx context.Context, g auth.Grant, directory string) ([]string, error)

AllFiles returns every key under a directory, recursively, without the tenant part, sorted.

The empty directory means everything this tenant has. Sorted because a listing that changes order between two identical calls turns a paginated screen into a bug report nobody can reproduce.

The argument is matched as a prefix, so "invoices/" is the directory and "invoices" also catches "invoices-2025.pdf". Only the caller knows which was meant -- Disk.Files and Disk.Directories are the ones that read it as a directory.

func (*Disk) Append 

func (d *Disk) Append(ctx context.Context, g auth.Grant, key, data string) error

Append adds a line to the end of a file, creating it when it is not there.

It is read-modify-write and not an append syscall, because an object store has no such thing: two concurrent Appends to the same key end with one of the two, not both. The answer to a file several writers extend at once is the queue, not this.

func (*Disk) AssertCount 

func (d *Disk) AssertCount(ctx context.Context, t TB, g auth.Grant, directory string, count int, recursive bool)

AssertCount fails the test when a directory does not hold exactly count files.

recursive counts everything underneath rather than the directory's own files, which is the difference between Disk.AllFiles and Disk.Files.

func (*Disk) AssertDirectoryEmpty 

func (d *Disk) AssertDirectoryEmpty(ctx context.Context, t TB, g auth.Grant, directory string)

AssertDirectoryEmpty fails the test when a directory holds anything.

func (*Disk) AssertExists 

func (d *Disk) AssertExists(ctx context.Context, t TB, g auth.Grant, key string, content ...[]byte)

AssertExists fails the test when the key is not there for this tenant.

content is optional: pass one value to also assert what the file holds.

func (*Disk) AssertMissing 

func (d *Disk) AssertMissing(ctx context.Context, t TB, g auth.Grant, key string)

AssertMissing fails the test when the key is there.

func (*Disk) BuildTemporaryURLsUsing 

func (d *Disk) BuildTemporaryURLsUsing(callback TemporaryURLCallback)

BuildTemporaryURLsUsing installs the callback that answers Disk.TemporaryURL on a disk whose driver cannot presign.

It is how a local disk gets temporary URLs: the application hands it a closure that calls URLSigner.TemporaryURL, and every caller of Disk.TemporaryURL then works the same on the directory and on the bucket. Wire it once, at boot, next to where the disk is registered.

func (*Disk) BuildTemporaryUploadURLsUsing 

func (d *Disk) BuildTemporaryUploadURLsUsing(callback TemporaryUploadURLCallback)

BuildTemporaryUploadURLsUsing installs the callback that answers Disk.TemporaryUploadURL.

func (*Disk) Checksum 

func (d *Disk) Checksum(ctx context.Context, g auth.Grant, key string) (string, error)

Checksum returns the SHA-256 of the file's contents, in lowercase hex.

It is SHA-256 and not MD5: the reason to hash a stored file is to answer "is this the same file", and answering it with a hash that can be made to collide on purpose is answering a different question. There is no algorithm option, for the same reason there is no second anything here.

It reads the whole file, because a checksum of part of one is not a checksum.

func (*Disk) Copy 

func (d *Disk) Copy(ctx context.Context, g auth.Grant, src, dst string) error

Copy duplicates a file inside one tenant.

Both keys are resolved against the same Grant, so a copy cannot cross into another tenant in either direction -- which is the mistake worth preventing, because "copy this into the shared folder" is how it gets asked for.

func (*Disk) Delete 

func (d *Disk) Delete(ctx context.Context, g auth.Grant, key string) error

Delete removes a file. Removing what is not there is not an error.

func (*Disk) DeleteDirectory 

func (d *Disk) DeleteDirectory(ctx context.Context, g auth.Grant, directory string) error

DeleteDirectory removes everything under a directory, for this tenant only.

It is the one operation whose blast radius is worth stating out loud: an empty directory deletes everything the tenant has. It still cannot reach past the tenant, because the prefix is resolved the same way a key is.

func (*Disk) Directories 

func (d *Disk) Directories(ctx context.Context, g auth.Grant, directory string) ([]string, error)

Directories returns the directories directly inside a directory.

A directory exists because a key underneath it does; there is nothing else for one to be. Every name comes back with a trailing slash, so what it is cannot be mistaken and it can be passed straight back in.

func (*Disk) DirectoryExists 

func (d *Disk) DirectoryExists(ctx context.Context, g auth.Grant, directory string) (bool, error)

DirectoryExists reports whether a directory is there for this tenant.

On a driver with real directories it is the driver's answer. On one without, it is whether the tenant has any key under that prefix -- which is what a directory is in an object store, and the only reading under which the same module behaves the same way on both.

func (*Disk) DirectoryMissing 

func (d *Disk) DirectoryMissing(ctx context.Context, g auth.Grant, directory string) (bool, error)

DirectoryMissing is Disk.DirectoryExists inverted, and it exists for the reason Disk.Missing does: `if !ok` after a call that also returns an error is where a failed lookup silently becomes "it is not there".

func (*Disk) Download 

func (d *Disk) Download(w http.ResponseWriter, r *http.Request, g auth.Grant, key, name string, headers http.Header) error

Download writes a stored file into the response as an attachment, so the browser saves it instead of rendering it.

func (*Disk) Exists 

func (d *Disk) Exists(ctx context.Context, g auth.Grant, key string) (bool, error)

Exists reports whether the key is there for this tenant.

func (*Disk) FileExists 

func (d *Disk) FileExists(ctx context.Context, g auth.Grant, key string) (bool, error)

FileExists reports whether the key is there for this tenant.

It is Disk.Exists under a second name. Both are here because the pair with Disk.DirectoryExists is what makes the question unambiguous at a call site that deals in both.

func (*Disk) FileMissing 

func (d *Disk) FileMissing(ctx context.Context, g auth.Grant, key string) (bool, error)

FileMissing reports whether the key is not there for this tenant.

func (*Disk) Files 

func (d *Disk) Files(ctx context.Context, g auth.Grant, directory string) ([]string, error)

Files returns the keys directly inside a directory, without descending.

The empty directory is the tenant's root. What "directly inside" means is decided here and not by the driver: an object store has no directories at all, and a listing that differed between local disk and S3 would be a module that works until it is deployed.

func (*Disk) Get 

func (d *Disk) Get(ctx context.Context, g auth.Grant, key string) (File, error)

Get reads one back. The caller closes File.Body.

func (*Disk) GetAdapter 

func (d *Disk) GetAdapter() Adapter

GetAdapter returns the driver underneath.

It exists so URLSigner can ask whether the driver is a Presigner, and so a test can assert against the fake it installed. It is not a way around the Grant: an Adapter takes stored paths, and the only thing that produces one is Key, which needs a Grant.

func (*Disk) GetConfig 

func (d *Disk) GetConfig() Config

GetConfig returns the configuration this disk was built with.

func (*Disk) GetDriver 

func (d *Disk) GetDriver() Adapter

GetDriver returns the same driver Disk.GetAdapter does.

There is one object here rather than a driver wrapping an adapter, so both names answer with it rather than one of them being missing for a reason nobody could act on.

func (*Disk) GetVisibility 

func (d *Disk) GetVisibility(ctx context.Context, g auth.Grant, key string) (string, error)

GetVisibility returns whether the store itself would hand this file to somebody who has not been through a Policy.

It answers ErrNoVisibility on a driver with no such concept, which is not a failure of the call: a store that has no public mode has no visibility to report, and the honest answer is that rather than "private", which would read as a guarantee this package did not make.

func (*Disk) Json 

func (d *Disk) Json(ctx context.Context, g auth.Grant, key string) (map[string]any, error)

Json reads a file and decodes it as JSON.

It answers with map[string]any because the files this is for -- a manifest, an export, a stored payload -- are objects. A caller holding a struct should Disk.Get and unmarshal into it, which is one line more and type-checked.

func (*Disk) LastModified 

func (d *Disk) LastModified(ctx context.Context, g auth.Grant, key string) (time.Time, error)

LastModified returns when the file was last written.

func (*Disk) MakeDirectory 

func (d *Disk) MakeDirectory(ctx context.Context, g auth.Grant, directory string) error

MakeDirectory creates a directory for this tenant.

On a driver with real directories it makes one. On an object store it does nothing and reports success, because there is nothing to make: the directory exists as soon as a key under it does, and it is not there before that no matter what this call did. The alternative -- writing an empty marker object to stand for a folder -- puts a key in every listing that Disk.Get then answers ErrNotFound for, which is a file that exists and cannot be read.

func (*Disk) MimeType 

func (d *Disk) MimeType(ctx context.Context, g auth.Grant, key string) (string, error)

MimeType returns the stored content type.

It is the type [Put] inferred from the key, never the one an upload announced, so it is the same string Serve will send -- which is what makes it safe to show to a person or to switch on.

func (*Disk) Missing 

func (d *Disk) Missing(ctx context.Context, g auth.Grant, key string) (bool, error)

Missing is Exists inverted, and it exists because the two readings are not equally easy to get right: `if !ok` after a call that also returns an error is where a failed lookup silently becomes "the file is not there".

func (*Disk) Move 

func (d *Disk) Move(ctx context.Context, g auth.Grant, src, dst string) error

Move copies and then deletes the source.

In that order. The other order loses the file when the write fails, and the write is the half that fails.

func (*Disk) Name 

func (d *Disk) Name() string

Name returns the name this disk was registered under.

func (*Disk) Path 

func (d *Disk) Path(g auth.Grant, key string) (string, error)

Path returns where a key is stored on this machine.

It carries the tenant prefix, because that is what the path IS -- this is the one method that hands one out, and it does so only to a caller that already holds a Grant for that key. Do not build another key out of it: the way to reach a second file is a second Key call, which checks the Grant again.

func (*Disk) Prepend 

func (d *Disk) Prepend(ctx context.Context, g auth.Grant, key, data string) error

Prepend adds a line to the front of a file, creating it when it is not there.

It reads the whole file into memory to do it. That is what prepending to a file is, on a directory and on a bucket alike.

func (*Disk) ProvidesTemporaryURLs 

func (d *Disk) ProvidesTemporaryURLs() bool

ProvidesTemporaryURLs reports whether Disk.TemporaryURL will answer with one rather than an error.

func (*Disk) ProvidesTemporaryUploadURLs 

func (d *Disk) ProvidesTemporaryUploadURLs() bool

ProvidesTemporaryUploadURLs reports whether Disk.TemporaryUploadURL will.

func (*Disk) Put 

func (d *Disk) Put(ctx context.Context, g auth.Grant, key string, body io.Reader, contentType string) error

Put writes a file under the tenant of the Grant.

An empty contentType is inferred from the key. Inferring from the key and not from what an upload announced is deliberate: the client-supplied type is a string an attacker chose, and storing it is how an upload becomes stored XSS three months later when something serves it back.

func (*Disk) PutFile 

func (d *Disk) PutFile(ctx context.Context, g auth.Grant, directory string, u Upload) (string, error)

PutFile stores an upload under a directory and returns the key it landed on.

The name is drawn at random and keeps only the extension, which is the point of the call: the announced filename is a string the client chose, and storing under it means two people uploading "scan.pdf" overwrite each other -- and that somebody who guesses a filename guesses a key.

It does not check UploadRules. Checking is Upload.Check, called where the answer can be shown to the person who picked the file; folding it in here would make an unchecked Put impossible to write and a checked one impossible to see.

func (*Disk) PutFileAs 

func (d *Disk) PutFileAs(ctx context.Context, g auth.Grant, directory string, u Upload, name string) (string, error)

PutFileAs stores an upload under a directory with the name you give it.

The name is a name: one with a separator in it is refused rather than cleaned, because a caller that meant a subdirectory should say so in directory, and a caller that did not mean one has been handed something by a client.

func (*Disk) ReadStream 

func (d *Disk) ReadStream(ctx context.Context, g auth.Grant, key string) (io.ReadCloser, error)

ReadStream opens a file for reading and returns the stream. The caller closes it.

It is Disk.Get without the metadata: the same bytes, for a caller that is going to io.Copy them somewhere and has nothing to do with the size or the content type. Ask for Get when either of those matters -- serving the file, for one, needs both.

func (*Disk) Response 

func (d *Disk) Response(w http.ResponseWriter, r *http.Request, g auth.Grant, key, name string, headers http.Header) error

Response writes a stored file into the response for the browser to render.

name is what to call the file, and empty means the last segment of the key; headers are added to the response before the file's own are set, so the three headers Serve insists on -- the stored content type, nosniff, and the escaped disposition -- cannot be overridden by a caller passing them in. That is deliberate: they are what stops an upload becoming stored XSS, and a header map is exactly where somebody would turn them off by accident.

func (*Disk) ServeUsing 

func (d *Disk) ServeUsing(callback ServeCallback)

ServeUsing installs the callback that Disk.Response and Disk.Download hand the work to. Passing nil puts Serve back.

func (*Disk) SetVisibility 

func (d *Disk) SetVisibility(ctx context.Context, g auth.Grant, key, visibility string) error

SetVisibility sets it. See VisibilityAware for why this is not a way to authorize anything.

func (*Disk) Size 

func (d *Disk) Size(ctx context.Context, g auth.Grant, key string) (int64, error)

Size returns how many bytes the file holds.

func (*Disk) Stat 

func (d *Disk) Stat(ctx context.Context, g auth.Grant, key string) (Info, error)

Stat answers what Get would carry, without moving the bytes.

func (*Disk) TemporaryURL 

func (d *Disk) TemporaryURL(ctx context.Context, g auth.Grant, key string, ttl time.Duration) (string, error)

TemporaryURL returns a URL that serves one file for ttl and then stops.

The callback installed by Disk.BuildTemporaryURLsUsing wins, so an application can route every temporary link through its own signer. Otherwise the driver presigns, and the bytes never pass through the application.

The link is a bearer credential for one file. It names the file, it expires, and it is proof that a Policy said yes at the moment it was made -- which is why it takes a Grant to mint and none to redeem.

func (*Disk) TemporaryUploadURL 

func (d *Disk) TemporaryUploadURL(ctx context.Context, g auth.Grant, key string, ttl time.Duration) (string, http.Header, error)

TemporaryUploadURL returns a URL a client may upload one file to for ttl, and the headers it has to repeat.

It is the answer to a file too large to pass through the application: the browser sends the bytes straight to the store. The Grant is what says the caller may write that key, and the signature carries that decision -- the store will not check it again, so the ttl is the whole of the exposure and should be minutes.

func (*Disk) URL 

func (d *Disk) URL(ctx context.Context, g auth.Grant, key string) (string, error)

URL returns the permanent public address of a file.

Read this before using it

The address carries no authorization. Anybody who has it has the file, for as long as the file exists -- no session, no Policy, no expiry. That is what a public disk IS, and it is the correct shape for exactly one kind of content: something every visitor is meant to see anyway, like a logo. For anything a tenant uploaded, the answer is Disk.TemporaryURL, which expires, or a route that runs a Policy and calls Serve.

It takes a Grant anyway: handing out the address of a file is a read, and the caller has to have been allowed to reach the file to be allowed to publish where it lives.

It answers ErrNoURL when the disk has no public address, which is the default -- Config.URL is empty and the driver generates none.

func (*Disk) WriteStream 

func (d *Disk) WriteStream(ctx context.Context, g auth.Grant, key string, body io.Reader, contentType string) error

WriteStream writes a stream to a key.

It is Disk.Put under a second name. There is one implementation underneath, because Put already takes an io.Reader.

type Disks 

type Disks struct {
	// contains filtered or unexported fields
}

Disks is the set of configured disks, and the one place a name becomes a disk.

Disks are registered at boot from configuration, and a handler asks for one by name. The zero value is not usable; call NewDisks.

func NewDisks 

func NewDisks(defaultName string, cloud ...string) *Disks

NewDisks returns an empty set whose default disk will be defaultName.

The default may be registered after this call -- configuration names it before it is wired -- and asking for it before it exists is an error, not a panic at boot.

An optional second name is the cloud disk, which is what Disks.Cloud answers with. A project with one disk does not have to name it.

func (*Disks) Add 

func (ds *Disks) Add(name string, a Adapter) *Disk

Add registers an adapter under a name and returns the disk it became.

Registering the same name twice replaces it, because the alternative -- an error at boot from a config file that listed a disk twice -- is a crash for a mistake with an obvious reading.

func (*Disks) Build 

func (ds *Disks) Build(name string, cfg Config) (*Disk, error)

Build makes a disk out of a configuration without registering it.

It is the one-off disk, for a job that needs a directory nobody named in configuration. Register it with Disks.Set or Disks.Add if it should be reachable by name.

func (*Disks) Cloud 

func (ds *Disks) Cloud() (*Disk, error)

Cloud returns the disk named as the cloud disk when the set was built.

func (*Disks) CreateLocalDriver 

func (ds *Disks) CreateLocalDriver(name string, cfg Config) (*Disk, error)

CreateLocalDriver builds a disk over a directory.

func (*Disks) CreateScopedDriver 

func (ds *Disks) CreateScopedDriver(name string, cfg Config) (*Disk, error)

CreateScopedDriver builds a disk that is a subtree of another disk.

Everything it stores lands at <prefix>/<tenant>/<key> on the disk named by Config.Disk: the scope narrows which part of that disk this one reaches, and the tenant still separates two customers inside the scope. Scoping only ever narrows, so a scoped disk cannot be used to step out of a tenant or out of the subtree it was given.

func (*Disks) Disk 

func (ds *Disks) Disk(name string) (*Disk, error)

Disk returns a disk by name. The empty name is the default disk.

There is no second accessor for the default. A caller that has no opinion passes "", and a caller that does passes the name; one function, so nothing has to decide which of two to reach for.

func (*Disks) Drive 

func (ds *Disks) Drive(name string) (*Disk, error)

Drive returns a disk by name. The empty name is the default disk.

It is Disks.Disk under a second name, so a project moving over finds the call it wrote either way.

func (*Disks) Extend 

func (ds *Disks) Extend(driver string, creator DriverCreator) *Disks

Extend registers a driver creator under a name.

It is what an adapter living in its own module hooks into: the S3 driver is github.com/arandu-io/hesape/filesystem/s3, a separate module because in Go there is no optional dependency, and the application that imports it registers it here in one line. That is why there is no CreateS3Driver on this type -- the root module cannot import its own submodule, and a creator that lied about being able to build one would fail at boot instead of at compile time.

func (*Disks) ForgetDisk 

func (ds *Disks) ForgetDisk(names ...string) *Disks

ForgetDisk drops disks from the set, so the next lookup rebuilds or fails.

func (*Disks) GetDefaultCloudDriver 

func (ds *Disks) GetDefaultCloudDriver() string

GetDefaultCloudDriver returns the name of the disk Disks.Cloud answers with.

func (*Disks) GetDefaultDriver 

func (ds *Disks) GetDefaultDriver() string

GetDefaultDriver returns the name of the disk that "" resolves to.

func (*Disks) Names 

func (ds *Disks) Names() []string

Names returns the registered names, sorted.

func (*Disks) Purge 

func (ds *Disks) Purge(name string)

Purge drops one disk. The empty name purges the default disk.

func (*Disks) Set 

func (ds *Disks) Set(name string, disk *Disk) *Disks

Set registers an already-built disk under a name and returns the set, so wiring chains.

type DriverCreator 

type DriverCreator func(name string, cfg Config) (Adapter, error)

DriverCreator builds an adapter out of a configuration.

It is what Disks.Extend registers. It receives the disk's name because a driver often wants it in an error message, and because the local driver puts it in the signed-URL route.

type File 

type File struct {
	Info
	// Body is the content. The caller closes it.
	Body io.ReadCloser
}

File is what a Get returns: the Info plus the bytes.

type Filesystem 

type Filesystem struct{}

Filesystem is the local file API the framework itself runs on.

It is NOT a Disk, and the difference is the whole reason both exist. A Disk holds customer data, so every one of its methods takes an auth.Grant and every path it builds starts with a tenant. A Filesystem holds the application's own files -- a stub, a compiled view, a session file, a cache entry -- which belong to the process and to nobody else, and it takes absolute or working-directory-relative paths exactly as os.ReadFile does.

Storing a tenant's upload through this type is the bug it is worth naming here: there is no prefix, so there is no isolation. The way in for anything a customer sent is Disk.Put.

The zero value is usable, and so is the pointer NewFilesystem returns; this type holds no state.

func NewFilesystem 

func NewFilesystem() *Filesystem

NewFilesystem returns a Filesystem. It exists so wiring reads the same as the rest of the collection; the zero value works too.

func (*Filesystem) AllDirectories 

func (f *Filesystem) AllDirectories(directory string) ([]string, error)

AllDirectories returns every directory under a directory, at any depth, sorted.

func (*Filesystem) AllFiles 

func (f *Filesystem) AllFiles(directory string, hidden bool) ([]string, error)

AllFiles returns every file under a directory, at any depth, sorted.

func (*Filesystem) Append 

func (f *Filesystem) Append(path string, data []byte, lock bool) error

Append writes data to the end of a file, creating it when it is not there.

func (*Filesystem) Basename 

func (f *Filesystem) Basename(path string) string

Basename returns the trailing name component of a path.

func (*Filesystem) Chmod 

func (f *Filesystem) Chmod(path string, mode fs.FileMode) (fs.FileMode, error)

Chmod reads or sets the permissions of a path.

A mode of 0 reads and does not write; anything else is set. The returned mode is the one in force after the call.

func (*Filesystem) CleanDirectory 

func (f *Filesystem) CleanDirectory(directory string) error

CleanDirectory empties a directory and keeps the directory itself.

func (*Filesystem) Copy 

func (f *Filesystem) Copy(path, target string) error

Copy duplicates a file.

func (*Filesystem) CopyDirectory 

func (f *Filesystem) CopyDirectory(directory, destination string) error

CopyDirectory copies a directory, recursively, to a destination.

func (*Filesystem) Delete 

func (f *Filesystem) Delete(paths ...string) error

Delete removes the given files. Removing what is not there is not an error.

It is variadic, so one path and a list of them are the same call.

func (*Filesystem) DeleteDirectories 

func (f *Filesystem) DeleteDirectories(directory string) (bool, error)

DeleteDirectories removes every directory directly inside a directory, leaving the files alone. It reports whether it removed anything.

func (*Filesystem) DeleteDirectory 

func (f *Filesystem) DeleteDirectory(directory string, preserve bool) error

DeleteDirectory removes a directory and everything under it.

preserve keeps the directory itself and removes only what is inside, which is what Filesystem.CleanDirectory is named for.

func (*Filesystem) Directories 

func (f *Filesystem) Directories(directory string, depth int) ([]string, error)

Directories returns the directories in a directory, sorted.

func (*Filesystem) Dirname 

func (f *Filesystem) Dirname(path string) string

Dirname returns the parent directory of a path.

func (*Filesystem) EnsureDirectoryExists 

func (f *Filesystem) EnsureDirectoryExists(path string, mode fs.FileMode, recursive bool) error

EnsureDirectoryExists creates a directory when it is not already there.

A mode of 0 means 0755.

func (*Filesystem) Exists 

func (f *Filesystem) Exists(path string) bool

Exists reports whether a file or directory exists at the path.

func (*Filesystem) Extension 

func (f *Filesystem) Extension(path string) string

Extension returns the extension of a path, without the dot.

filepath.Ext returns ".pdf" and this returns "pdf", so a caller comparing against a configured list of extensions gets the comparison it wrote.

func (*Filesystem) Files 

func (f *Filesystem) Files(directory string, hidden bool, depth int) ([]string, error)

Files returns the files in a directory, sorted.

hidden includes names starting with a dot. depth is how many levels below the directory to descend: 0 is the directory itself, and Filesystem.AllFiles is this with no limit.

func (*Filesystem) Get 

func (f *Filesystem) Get(path string, lock bool) ([]byte, error)

Get returns the contents of a file.

lock takes a shared lock for the read, which is what makes it safe to read a file another process is replacing with Filesystem.Put under an exclusive lock. There is no default: Go has none, and a bool at the call site says which of the two this read is.

It returns ErrNotFound for a file that is not there.

func (*Filesystem) Glob 

func (f *Filesystem) Glob(pattern string) ([]string, error)

Glob returns the paths matching a shell pattern, sorted.

func (*Filesystem) GuessExtension 

func (f *Filesystem) GuessExtension(path string) (string, error)

GuessExtension returns the extension a file's content type implies, without the dot, or "" when nothing is known about it.

func (*Filesystem) HasSameHash 

func (f *Filesystem) HasSameHash(firstFile, secondFile string) bool

HasSameHash reports whether two files hold the same contents.

func (*Filesystem) Hash 

func (f *Filesystem) Hash(path, algorithm string) (string, error)

Hash returns the hash of a file's contents, in lowercase hex.

The empty algorithm is "md5", which is what Filesystem.HasSameHash compares with. It is a change-detection hash and not a security one: two files that hash the same here were not proven to be the same file by an attacker who wanted them to collide. The security answer is Disk.Checksum, which is SHA-256 and has no algorithm option.

Recognised: "md5", "sha1", "sha256".

func (*Filesystem) IsDirectory 

func (f *Filesystem) IsDirectory(directory string) bool

IsDirectory reports whether the path is a directory.

func (*Filesystem) IsEmptyDirectory 

func (f *Filesystem) IsEmptyDirectory(directory string, ignoreDotFiles bool) bool

IsEmptyDirectory reports whether a directory holds nothing.

ignoreDotFiles treats a directory holding only dot files as empty, which is what a check for "did anything get published here" wants: a .gitkeep is not content.

func (*Filesystem) IsFile 

func (f *Filesystem) IsFile(file string) bool

IsFile reports whether the path is a regular file.

func (*Filesystem) IsReadable 

func (f *Filesystem) IsReadable(path string) bool

IsReadable reports whether the process can read the path.

func (*Filesystem) IsWritable 

func (f *Filesystem) IsWritable(path string) bool

IsWritable reports whether the process can write the path.

A directory is writable when a file can be created in it, which is the question a caller is really asking and the only one an access bit cannot answer on its own.

func (*Filesystem) Json 

func (f *Filesystem) Json(path string, lock bool) (map[string]any, error)

Json returns the decoded contents of a JSON file.

It answers with map[string]any because the files this is for -- configuration and manifest files -- are objects. A caller that has a struct should read with Filesystem.Get and unmarshal into it, which is one call more and type-checked.

func (*Filesystem) LastModified 

func (f *Filesystem) LastModified(path string) (time.Time, error)

LastModified returns when a file was last written.

func (*Filesystem) Lines 

func (f *Filesystem) Lines(path string) ([]string, error)

Lines returns the file split on newlines, with the trailing newline of the last line dropped.

It reads the whole file and splits it rather than streaming one line at a time: a caller that needs the lazy shape has bufio.Scanner, and a second, lazier Lines beside this one would be a second way to read a file. 

func (f *Filesystem) Link(target, link string) error

Link creates a symbolic link to the target.

It is a symbolic link on every platform, including the ones where that needs a privilege. A hard link is a different object with different semantics -- deleting the target leaves the link working, which is the opposite of what a link into a build directory is for.

func (*Filesystem) MakeDirectory 

func (f *Filesystem) MakeDirectory(path string, mode fs.FileMode, recursive, force bool) error

MakeDirectory creates a directory.

force removes whatever is in the way first. recursive creates the parents. A mode of 0 means 0755.

func (*Filesystem) MimeType 

func (f *Filesystem) MimeType(path string) (string, error)

MimeType returns the content type implied by the path's extension.

From the extension and never from the bytes, which is the same rule [Put] follows: content sniffing is what turns an uploaded file into stored XSS the day something serves it back. Unknown extensions answer application/octet-stream, which browsers download rather than render.

func (*Filesystem) Missing 

func (f *Filesystem) Missing(path string) bool

Missing reports whether nothing exists at the path.

func (*Filesystem) Move 

func (f *Filesystem) Move(path, target string) error

Move renames a file.

func (*Filesystem) MoveDirectory 

func (f *Filesystem) MoveDirectory(from, to string, overwrite bool) error

MoveDirectory renames a directory.

overwrite removes the destination first. Without it, a destination that is already there is an error rather than a merge: merging two trees silently is how a deploy ends up with half of the previous release in it.

func (*Filesystem) Name 

func (f *Filesystem) Name(path string) string

Name returns the file name without its extension.

func (*Filesystem) Prepend 

func (f *Filesystem) Prepend(path string, data []byte) error

Prepend writes data to the front of a file, creating it when it is not there.

func (*Filesystem) Put 

func (f *Filesystem) Put(path string, contents []byte, lock bool) error

Put writes the contents to a file, creating it if it is not there.

lock takes an exclusive lock for the write, which is what a reader calling Filesystem.SharedGet waits on. 

func (f *Filesystem) RelativeLink(target, link string) error

RelativeLink creates a symbolic link whose target is written relative to the directory holding the link.

That is what survives the tree being moved or mounted somewhere else, which is the whole reason it exists beside Link.

func (*Filesystem) Replace 

func (f *Filesystem) Replace(path string, content []byte, mode fs.FileMode) error

Replace writes the contents atomically, so a reader never sees half of them.

It writes a temporary file beside the target and renames it into place, which is what makes the swap atomic on a POSIX filesystem. A mode of 0 keeps the mode of the file that was already there, or 0644 when there was none: a rename would otherwise hand the file the temporary's private permissions.

func (*Filesystem) ReplaceInFile 

func (f *Filesystem) ReplaceInFile(search, replace, path string) error

ReplaceInFile substitutes every occurrence of search with replace inside a file.

func (*Filesystem) SharedGet 

func (f *Filesystem) SharedGet(path string) ([]byte, error)

SharedGet reads a file with a shared lock held for the whole read.

func (*Filesystem) Size 

func (f *Filesystem) Size(path string) (int64, error)

Size returns how many bytes a file holds.

func (*Filesystem) Type 

func (f *Filesystem) Type(path string) (string, error)

Type returns "dir" for a directory and "file" for anything else.

type Info 

type Info struct {
	// Key is the key the caller asked for, never the stored path. A caller that
	// saw the tenant prefix here would start building paths out of it.
	Key         string
	Size        int64
	ContentType string
	ModifiedAt  time.Time
}

Info is what a file is, without its content.

Stat answers with it, Get carries it inside a File, and Serve writes it into the response headers. One type, because a size that means one thing in a listing and another in a download is how a Content-Length ends up wrong.

type LocalFilesystemAdapter 

type LocalFilesystemAdapter struct {
	// contains filtered or unexported fields
}

LocalFilesystemAdapter stores files in a directory.

It is the Adapter that needs nothing installed, which makes it the right one for development and for a single machine. For anything with more than one replica, github.com/arandu-io/hesape/filesystem/s3 is the same contract over the S3 protocol -- and Cloudflare R2 is the default there.

Like every Adapter it is handed stored paths and never a Grant, so the tenant prefix is not something this file could forget: it was applied by Key before the call.

func NewLocalFilesystemAdapter 

func NewLocalFilesystemAdapter(root string) (*LocalFilesystemAdapter, error)

NewLocalFilesystemAdapter returns an adapter rooted at a directory.

The root is created if it does not exist, because the alternative is an application that boots fine and fails on the first upload.

func (*LocalFilesystemAdapter) Delete 

func (a *LocalFilesystemAdapter) Delete(_ context.Context, storedPath string) error

Delete removes a file. Removing what is not there is not an error: the caller wanted it gone, and it is.

func (*LocalFilesystemAdapter) DirectoryExists 

func (a *LocalFilesystemAdapter) DirectoryExists(_ context.Context, storedPath string) (bool, error)

DirectoryExists reports whether the path is a directory under the root.

func (*LocalFilesystemAdapter) DiskName 

func (a *LocalFilesystemAdapter) DiskName(disk string) *LocalFilesystemAdapter

DiskName records the name this adapter's disk is registered under.

It is the name ServeFile needs to find the disk again from a link. Disks.Build sets it; setting it by hand is for a disk built without the manager.

func (*LocalFilesystemAdapter) Exists 

func (a *LocalFilesystemAdapter) Exists(_ context.Context, storedPath string) (bool, error)

Exists reports whether the path is a file that is there.

func (*LocalFilesystemAdapter) Get 

func (a *LocalFilesystemAdapter) Get(ctx context.Context, storedPath string) (File, error)

Get reads a file back. The caller closes File.Body.

func (*LocalFilesystemAdapter) List 

func (a *LocalFilesystemAdapter) List(_ context.Context, prefix string) ([]string, error)

List returns the stored paths under a prefix.

The prefix is a string match and not a directory: "invoices/" and "invoices" select different sets, and Disk is what decides which one a caller meant. The walk starts at the deepest directory the prefix names, so listing one tenant does not read another tenant's directory entries.

func (*LocalFilesystemAdapter) MakeDirectory 

func (a *LocalFilesystemAdapter) MakeDirectory(_ context.Context, storedPath string) error

MakeDirectory creates the directory and its parents under the root.

func (*LocalFilesystemAdapter) Path 

func (a *LocalFilesystemAdapter) Path(storedPath string) (string, error)

Path returns the absolute path of a stored path under the root.

func (*LocalFilesystemAdapter) Put 

func (a *LocalFilesystemAdapter) Put(_ context.Context, storedPath string, body io.Reader, _ string) error

Put writes a file, creating whatever directories it needs.

The content type is not stored: on disk it is the extension, and keeping a sidecar file per object to hold one string is a second thing to keep in sync. Get infers it, which is what a static file server does anyway.

func (*LocalFilesystemAdapter) Root 

func (a *LocalFilesystemAdapter) Root() string

Root returns the directory this adapter writes under.

It is here for an error message and for a test that wants to look at what landed on disk. It is not a way to build a path: what goes under the root is decided by Key, and this returns the part of the answer that carries no tenant.

func (*LocalFilesystemAdapter) ServesSignedURLs 

func (a *LocalFilesystemAdapter) ServesSignedURLs() bool

ServesSignedURLs reports what LocalFilesystemAdapter.ShouldServeSignedUrls was told.

func (*LocalFilesystemAdapter) SetVisibility 

func (a *LocalFilesystemAdapter) SetVisibility(_ context.Context, storedPath, visibility string) error

SetVisibility chmods the file.

func (*LocalFilesystemAdapter) ShouldServeSignedUrls 

func (a *LocalFilesystemAdapter) ShouldServeSignedUrls(serve bool) *LocalFilesystemAdapter

ShouldServeSignedUrls records that this disk's files reach a browser through the signed route rather than a public address.

ServeFile reads it. There is no "serve without a signature" here: a route that hands out a stored file with no proof attached is a read with no authorization, which is the one thing this collection does not have a way to write. Off therefore means this disk is not served by that route at all, not that it is served openly.

func (*LocalFilesystemAdapter) Stat 

func (a *LocalFilesystemAdapter) Stat(_ context.Context, storedPath string) (Info, error)

Stat answers what Get would carry, without opening the file.

func (*LocalFilesystemAdapter) Visibility 

func (a *LocalFilesystemAdapter) Visibility(_ context.Context, storedPath string) (string, error)

Visibility reads the file's mode.

Any read bit outside the owner's is public: a file group-readable but not world-readable is still reachable by somebody who is not the process, and calling that private would be the more dangerous of the two mistakes.

type LockableFile 

type LockableFile struct {
	// contains filtered or unexported fields
}

LockableFile is a file held open with an advisory lock around it.

It exists for one job: two processes writing the same file -- a session, a cache entry, a compiled view -- where a reader must see either the old contents or the new ones and never half of each. Filesystem.SharedGet and Filesystem.Put with lock are the two callers, and between them they are the whole pattern.

The lock is advisory, which means it only holds against somebody who also asks for it. That is not a weakness of this type: it is what flock is on every unix, and it is why the reading half has to take a shared lock rather than trusting the writer to be quick.

It is not tenant storage. A path here is a path, with no prefix and no Grant -- see Filesystem for that line, and Disk for the other side of it.

func NewLockableFile 

func NewLockableFile(path string, mode int) (*LockableFile, error)

NewLockableFile opens a file, creating the directory that holds it when the mode asks for creation.

mode is the flag set os.OpenFile takes: os.O_RDONLY to read, and os.O_CREATE|os.O_RDWR to write.

func (*LockableFile) Close 

func (f *LockableFile) Close() error

Close releases the lock and closes the file.

The lock first, and unconditionally: closing a descriptor drops the flock on every unix, but a caller reading this has to be able to see that the release happened rather than infer it from the kernel's behaviour.

func (*LockableFile) File 

func (f *LockableFile) File() *os.File

File returns the open file underneath.

It is here so a caller that already holds the lock can hand the handle to io.Copy rather than reading the whole file into memory. Locking stays this type's job: the handle is the same one Read and Write use.

func (*LockableFile) GetExclusiveLock 

func (f *LockableFile) GetExclusiveLock(block bool) error

GetExclusiveLock takes a write lock. Exactly one holder, and no shared lock alongside it.

func (*LockableFile) GetSharedLock 

func (f *LockableFile) GetSharedLock(block bool) error

GetSharedLock takes a read lock. Several readers may hold one at once, and no writer may hold an exclusive lock while any of them do.

block waits for the lock; without it, a lock somebody else holds returns ErrLockTimeout straight away.

func (*LockableFile) Read 

func (f *LockableFile) Read(length int64) ([]byte, error)

Read returns up to length bytes from the current position. A length of zero or less reads to the end of the file.

func (*LockableFile) ReleaseLock 

func (f *LockableFile) ReleaseLock() error

ReleaseLock drops whichever lock is held. Releasing none is not an error.

func (*LockableFile) Size 

func (f *LockableFile) Size() (int64, error)

Size returns how many bytes the file holds.

func (*LockableFile) Truncate 

func (f *LockableFile) Truncate() error

Truncate empties the file and rewinds to the start.

func (*LockableFile) Write 

func (f *LockableFile) Write(contents []byte) (int, error)

Write appends the contents at the current position and flushes them.

type Pather 

type Pather interface {
	Path(storedPath string) (string, error)
}

Pather is the optional half of an Adapter that can name a stored file on the machine the process is running on.

It exists for the one thing a stream cannot do: handing a path to another program -- a thumbnailer, a virus scanner, ffmpeg. An object store has no such path, and the caller has to download first, which is why this is optional and why Disk.Path says so rather than inventing one.

type PresignPutter 

type PresignPutter interface {
	PresignPut(ctx context.Context, storedPath string, ttl time.Duration) (string, http.Header, error)
}

PresignPutter is the optional half of an Adapter that can hand out a URL a browser uploads directly TO.

It is the write half of Presigner, and it is what makes a large upload possible without the bytes passing through the application at all. The headers it returns are the ones the client must repeat, because a presigned PUT is signed over them.

type Presigner 

type Presigner interface {
	PresignGet(ctx context.Context, path string, ttl time.Duration) (string, error)
}

Presigner is the optional half of an Adapter that can hand out a URL pointing at the store itself.

Object stores can; a directory on disk cannot. It is an optional interface and not a method on Adapter for that reason: a driver that cannot presign would otherwise have to carry a method that returns an error nobody can act on. URLSigner.TemporaryURL asks, and answers the same question either way, so no caller has to know which kind of disk it is holding.

The path it receives is already resolved by Key: a presigned URL is a bearer token for one object, and the object is the tenant's.

type PublicURLGenerator 

type PublicURLGenerator interface {
	PublicURL(ctx context.Context, storedPath string) (string, error)
}

PublicURLGenerator is the optional half of an Adapter that can name a file at a permanent public address.

Flysystem calls it PublicUrlGenerator, and this is the same idea under the Go spelling of the initialism. It is optional for the reason Presigner is: a directory on disk has no such address, and a method returning an error nobody can act on is worse than an interface a driver either satisfies or does not.

type ScopedAdapter 

type ScopedAdapter struct {
	// contains filtered or unexported fields
}

ScopedAdapter is an Adapter that stores everything under a prefix of another one.

It is the "scoped" driver: a path prefix and nothing else. It is deliberately not a place to put a tenant -- the tenant prefix comes from Key and from the Grant, and a driver that could add one would be a second place isolation is decided.

func NewScopedAdapter 

func NewScopedAdapter(inner Adapter, prefix string) *ScopedAdapter

NewScopedAdapter wraps an adapter so every stored path goes under prefix.

func (*ScopedAdapter) Delete 

func (a *ScopedAdapter) Delete(ctx context.Context, storedPath string) error

Delete removes the prefixed path.

func (*ScopedAdapter) Exists 

func (a *ScopedAdapter) Exists(ctx context.Context, storedPath string) (bool, error)

Exists reports on the prefixed path.

func (*ScopedAdapter) Get 

func (a *ScopedAdapter) Get(ctx context.Context, storedPath string) (File, error)

Get reads from under the prefix.

func (*ScopedAdapter) Inner 

func (a *ScopedAdapter) Inner() Adapter

Inner returns the adapter underneath.

func (*ScopedAdapter) List 

func (a *ScopedAdapter) List(ctx context.Context, prefix string) ([]string, error)

List answers in unprefixed paths, so a caller never sees the scope.

func (*ScopedAdapter) Put 

func (a *ScopedAdapter) Put(ctx context.Context, storedPath string, body io.Reader, contentType string) error

Put writes under the prefix.

func (*ScopedAdapter) Stat 

func (a *ScopedAdapter) Stat(ctx context.Context, storedPath string) (Info, error)

Stat answers about the prefixed path.

type ServeCallback 

type ServeCallback func(w http.ResponseWriter, r *http.Request, g auth.Grant, key string, opt ServeOptions) error

ServeCallback is what Disk.ServeUsing installs: the whole of how a file on this disk reaches a browser.

It replaces Serve for this disk. An application installs one when the response needs something this package does not do -- a redirect to a CDN, an extra audit line, a watermark -- and it is a callback rather than a subclass because a Disk is a value here and not a class to extend.

type ServeFile 

type ServeFile struct {
	// contains filtered or unexported fields
}

ServeFile is the route that turns a link from URLSigner.TemporaryURL back into the file it names.

It is the half that makes "no symlink into a document root" a real answer: the file is served by a route, and the route's proof that a Policy said yes is the signature on the link.

Mount it under the same base NewURLSigner was given, with the token as the last segment: 

mux.Handle("GET /files/", http.StripPrefix("/files/", serveFile))

func NewServeFile 

func NewServeFile(disks *Disks, signer *URLSigner, shouldServeSignedUrls bool) *ServeFile

NewServeFile returns the route.

shouldServeSignedUrls is what the disk was configured with; pass false and every request is refused, which is what a disk that is not meant to be reachable by link wants.

func (*ServeFile) ServeHTTP 

func (s *ServeFile) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP redeems the token in the path and writes the file it names.

The status codes are the ones the rest of the collection uses: 403 for a link that is not valid -- forged, expired, or minted for something else -- and 404 for a valid link to a file that is no longer there. They are different on purpose: an expired link is something the person can be told to ask for again, and a missing file is not.

type ServeOptions 

type ServeOptions struct {
	// Download asks the browser to save the file instead of rendering it.
	//
	// It is one field rather than two functions, because two functions that
	// differ by a header is two places to fix the header.
	Download bool
	// Filename is what to call it on the way out. Empty means the last segment
	// of the key.
	Filename string
	// CacheControl is the header to send. Empty means "private, max-age=0,
	// must-revalidate", which is the only safe default for a file that a Policy
	// decided about: a shared cache holding one tenant's file is the same leak
	// as a missing prefix, arriving later.
	CacheControl string
}

ServeOptions is how a file is offered.

type TB 

type TB interface {
	Helper()
	Errorf(format string, args ...any)
}

TB is the part of *testing.T these assertions use.

It is declared here rather than importing testing so that a package under test can hold a Disk without the test binary's flags leaking into a production build. It is satisfied by *testing.T and *testing.B unchanged.

type TemporaryURLCallback 

type TemporaryURLCallback func(ctx context.Context, g auth.Grant, key string, ttl time.Duration) (string, error)

TemporaryURLCallback is what Disk.BuildTemporaryURLsUsing installs.

type TemporaryUploadURLCallback 

type TemporaryUploadURLCallback func(ctx context.Context, g auth.Grant, key string, ttl time.Duration) (string, http.Header, error)

TemporaryUploadURLCallback is what Disk.BuildTemporaryUploadURLsUsing installs. The headers are the ones the client must send with the upload.

type URLSigner 

type URLSigner struct {
	// contains filtered or unexported fields
}

URLSigner issues links that stand in for a session.

It exists for the case a route cannot cover: an <img> tag, a download link in an e-mail, a file handed to a program that has no cookies. The link is proof that a Policy said yes at the moment it was made, it names one file, and it expires.

It is not a way around authorization. TemporaryURL takes a Grant, which means a Policy has already run; the signature carries the decision instead of repeating it, exactly as a signed verification link carries the decision to let somebody confirm an address.

func NewURLSigner 

func NewURLSigner(s *encryption.Signer, base string) *URLSigner

NewURLSigner returns a signer whose fallback links point under base.

base is the path the redeeming route is mounted at -- "/files", say, for a route that reads the last segment and calls [Redeem] then Serve. It is unused when the disk can presign, because then the URL comes from the store.

func (*URLSigner) Redeem 

func (u *URLSigner) Redeem(token string) (grant auth.Grant, disk, key string, err error)

Redeem reads a token issued by [TemporaryURL] and returns what it authorizes.

The Grant it returns is for this action, in this tenant, and it is meant for the key returned beside it -- pass the two together to Serve and to nothing else. It is a system grant, so it carries no person: a link is not a session, and treating it as one would mean a download link could be used to write.

The disk name travels in the token so the redeeming route does not have to guess which disk a link was made against; ask Disks.Disk for it.

func (*URLSigner) TemporaryURL 

func (u *URLSigner) TemporaryURL(ctx context.Context, g auth.Grant, d *Disk, key string, ttl time.Duration) (string, error)

TemporaryURL returns a URL that serves one file for ttl and then stops.

When the disk's adapter is a Presigner, the store issues it and the bytes never pass through the application. When it is not, the link points back at base and carries a signed token that [Redeem] reads. Both are one call, because a caller choosing between them is a caller who gets it wrong on the disk that changed driver.

type Upload 

type Upload struct {
	// Field is the form field the file arrived in.
	Field string
	// Name is the filename the client announced, with any directory stripped.
	Name string
	// Size is how many bytes arrived.
	Size int64
	// ContentType is the type the client announced. It is recorded and never
	// trusted: [Disk.Put] infers the stored type from the key instead.
	ContentType string
	// Open reads the content. The caller closes what it returns, and may call
	// Open more than once.
	Open func() (io.ReadCloser, error)
}

Upload is a file that arrived on a request.

It is the "own contract" that validation points at: a file is not in url.Values, and validating one inside the form pipeline would need a second input shape and a second set of rules, which is a second way to validate. An upload is checked here, against UploadRules, and stored with Disk.Put.

Everything on it except Size came from the client and is a string somebody chose. Name is not a key: it is what to call the file if it is ever offered back, and using it as a key is how "../../etc/passwd" gets stored -- which CleanKey then refuses, but the habit is the bug. Size is counted by the server as the body is read, so it is the one field worth checking.

func FromMultipart 

func FromMultipart(field string, h *multipart.FileHeader) Upload

FromMultipart builds an Upload out of a parsed multipart part.

It is the one place multipart is understood, so the HTTP layer above hands over a *multipart.FileHeader and gets back something with rules attached.

func (Upload) Check 

func (u Upload) Check(r UploadRules) error

Check answers whether this upload satisfies the rules.

Every failure unwraps to ErrRefusedUpload and every message is one a person can act on, because most of them are shown to one.

func (Upload) Extension 

func (u Upload) Extension() string

Extension returns the lowercased extension of the announced name, with the dot, or "" when there is none.

type UploadRules 

type UploadRules struct {
	// MaxBytes is the largest file accepted. It must be positive.
	MaxBytes int64
	// Extensions is the accepted set, lowercased and with the dot: ".pdf".
	// It must not be empty.
	Extensions []string
}

UploadRules is what an upload has to satisfy.

Both fields are required, and both fail closed. A rules value that named no maximum would accept a four gigabyte file, and one that named no extension would accept an .exe -- and the moment those are the defaults, the rule that was forgotten looks exactly like the rule that was written.

There is no content-type rule. The announced type is a header the client wrote, so checking it stops nobody who is trying; the extension is checked because it is what a person sees, and the real defense is on the way out -- the stored type comes from the key, and Serve sends nosniff.

type VisibilityAware 

type VisibilityAware interface {
	// Visibility returns [VisibilityPublic] or [VisibilityPrivate].
	Visibility(ctx context.Context, storedPath string) (string, error)
	// SetVisibility sets it.
	SetVisibility(ctx context.Context, storedPath, visibility string) error
}

VisibilityAware is the optional half of an Adapter whose store has a notion of a file being world-readable: a unix mode on a directory, an ACL on an object store.

It is an optional interface and not a method on Adapter for the reason Presigner is: a driver that has no such concept would otherwise carry a method returning an error nobody can act on. Adding it to Adapter would also break every driver already written against that interface, including the S3 module.

Visibility is not authorization

This is worth stating where somebody will read it before reaching for SetVisibility. Making a file public does not remove the Policy: every read still goes through Disk, which still takes an auth.Grant. What it changes is whether the STORE will serve the same bytes to somebody who never came through the application at all -- which is why the default here is private and why a tenant's upload must stay that way.

Source Files

View all Source files

Directories

Path Synopsis
s3 module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.