store

Documentation

Index

• Constants
• Variables
• func ValidateObjectPath(path string) error
• type ACMECache
  • func NewACMECache(store *Store, prefix string) *ACMECache
  • func (c *ACMECache) Delete(ctx context.Context, key string) error
  • func (c *ACMECache) Get(ctx context.Context, key string) ([]byte, error)
  • func (c *ACMECache) Put(ctx context.Context, key string, data []byte) error
• type FileEntry
• type PrefixStats
• type S3Object
• type Store
  • func New(client *s3.Client, bucket string, cacheSize int) (*Store, error)
  • func NewInMemory(cacheSize int) (*Store, error)
  • func (s *Store) Copy(ctx context.Context, slug, srcPath, dstPath string) error
  • func (s *Store) Delete(ctx context.Context, slug, path string) error
  • func (s *Store) DeleteRaw(ctx context.Context, key string) error
  • func (s *Store) EnsureBucket(ctx context.Context) error
  • func (s *Store) List(ctx context.Context, slug string) ([]string, error)
  • func (s *Store) ListApps(ctx context.Context) ([]string, error)
  • func (s *Store) ListPrefix(ctx context.Context, prefix string) ([]string, error)
  • func (s *Store) ListWithMeta(ctx context.Context, slug string) ([]FileEntry, error)
  • func (s *Store) Read(ctx context.Context, slug, path string) (*S3Object, error)
  • func (s *Store) ReadFresh(ctx context.Context, slug, path string) (*S3Object, error)
  • func (s *Store) ReadRaw(ctx context.Context, key string) (*S3Object, error)
  • func (s *Store) Rename(ctx context.Context, slug, srcPath, dstPath string) error
  • func (s *Store) SumBytesUnderPrefix(ctx context.Context, prefix string) (PrefixStats, error)
  • func (s *Store) UpdateMetadata(ctx context.Context, slug, path, contentType string, ...) error
  • func (s *Store) Write(ctx context.Context, slug, path, content, contentType string, ...) error
  • func (s *Store) WriteConditional(ctx context.Context, slug, path, content, contentType string, ...) (string, error)
  • func (s *Store) WriteRaw(ctx context.Context, key, content, contentType string, ...) error
  • func (s *Store) WriteRawIfAbsent(ctx context.Context, key, content, contentType string, ...) (string, error)
  • func (s *Store) WriteRawIfMatch(ctx context.Context, key, content, contentType string, ...) (string, error)

Constants

const (
	// SnapshotsPrefix holds per-site version-history archives, keyed
	// `_snapshots/{slug}/...` (internal/snapshot).
	SnapshotsPrefix = "_snapshots/"

	// EditsPrefix holds per-edit build transcripts, keyed `_edits/{slug}/...`
	// (internal/editrec).
	EditsPrefix = "_edits/"

	// DefaultACMEPrefix is the default home of the autocert account key and
	// certificate cache (store.ACMECache); overridable via --acme-cache-prefix.
	// The server's CertTracker shares it, writing per-host issuance outcomes
	// under a nested `_status/` key so a failure diagnosis persists next to
	// (and as long as) the certificate it explains.
	DefaultACMEPrefix = "_acme/"

	// StateDir is the in-slug directory for persisted form/KV data:
	// `{slug}/_state/data.json` (internal/state). Unlike the prefixes above it
	// exists once per site, so any bucket-level aggregation must walk slugs.
	StateDir = "_state/"

	// PendingDir is the in-slug directory for un-approved photo-wall bytes:
	// `{slug}/_pending/{id}.jpg` (internal/photowall). Proxy-blocked like
	// StateDir so visitor uploads can't be viewed before the owner approves
	// them; approval Copies the bytes out to the public assets/ tree.
	PendingDir = "_pending/"
)

This file is the single registry of the bucket's reserved keyspace. Two tiers exist, and confusing them has already produced one dead dashboard row (a bucket-level sum over the in-slug _state/ dir always reported zero):

• Bucket-level prefixes sit at the top of the bucket, outside any slug. ListApps hides every top-level prefix that starts with "_" (slugs cannot start with an underscore per validateSlug), which is what keeps these areas out of app listings and subdomain routing.

• In-slug paths live under `{slug}/...` and are reserved per site: the static proxy refuses to serve them and the file explorer treats them as platform-managed.

Packages that own one of these areas (snapshot, editrec, state, portable) alias their local constant to the one here, so adding or renaming a reserved area is one edit plus the compiler pointing at every consumer. The one non-compiled copy is cmd/topbanana's --acme-cache-prefix kong default, which must be a struct-tag literal; it mirrors DefaultACMEPrefix.

const DefaultContentType = "text/html; charset=utf-8"

Variables

var ErrPrecondition = errors.New("store: precondition failed")

ErrPrecondition is returned by the conditional writers when the object changed (or appeared, or vanished) between the caller's read and its write. It is the "someone else won" signal, not a fault: callers turn it into "retry" or "you lost the claim", never into a 500.

Functions

func ValidateObjectPath

func ValidateObjectPath(path string) error

ValidateObjectPath rejects relative-traversal segments, absolute paths, and Windows separators. The proxy handler already gates on this before reaching here, but every caller of Read/Write benefits from the same check — otherwise a future agent tool or handler could write objects at keys like `slug/../other/...` that escape the per-tenant prefix. Exported so the server proxy/validatePage path can share this exact rule instead of re-deriving it.

Types

type ACMECache

type ACMECache struct {
	Store  *Store
	Prefix string
}

ACMECache adapts Store to the autocert.Cache interface so Let's Encrypt account keys, issued certs, and short-lived challenge tokens persist in S3 instead of an ephemeral container filesystem. Keys are namespaced under Prefix (default "_acme/") which sits outside the slug space — ListApps excludes leading-underscore prefixes, so it can't collide with a real app.

func NewACMECache

func NewACMECache(store *Store, prefix string) *ACMECache

NewACMECache returns a Cache backed by store with the given key prefix. An empty prefix is allowed but discouraged; the constructor enforces a trailing slash if one is missing so callers don't have to think about it.

func (*ACMECache) Delete

func (c *ACMECache) Delete(ctx context.Context, key string) error

func (*ACMECache) Get

func (c *ACMECache) Get(ctx context.Context, key string) ([]byte, error)

Get returns the cached blob for key. autocert.ErrCacheMiss is required when the key is absent — ReadRaw signals absence by returning an S3Object with empty Content (and a nil error), so we translate that to the sentinel here. Cached ACME values are never legitimately empty (account keys, DER blobs, challenge tokens all have content), so the heuristic is safe.

func (*ACMECache) Put

func (c *ACMECache) Put(ctx context.Context, key string, data []byte) error

type FileEntry

type FileEntry struct {
	Path         string
	Size         int64
	LastModified time.Time
}

FileEntry is one row of ListWithMeta — the data the file explorer renders. LastModified comes straight from the backend and is already UTC.

type PrefixStats

type PrefixStats struct {
	TotalBytes  int64
	ObjectCount int
}

PrefixStats is the aggregate of SumBytesUnderPrefix: total bytes and object count beneath a bucket prefix.

type S3Object

type S3Object struct {
	Content     string
	ETag        string
	ContentType string
	// Metadata is user-defined key/value pairs stored as x-amz-meta-* headers
	// on the object. Values are URL-decoded on read so callers see plain
	// unicode; the store handles encoding on write because S3 metadata must
	// be ASCII.
	Metadata map[string]string
}

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is the platform's object-storage abstraction. It owns the cross-cutting rules — compression at rest, slug-prefix path validation, metadata URL-encoding, and the ARC read cache — and delegates the actual byte movement to an objectBackend (S3 in production, an in-memory map in tests). See backend.go for the seam.

func New

func New(client *s3.Client, bucket string, cacheSize int) (*Store, error)

New returns a Store backed by a real S3 bucket via client.

func NewInMemory

func NewInMemory(cacheSize int) (*Store, error)

NewInMemory returns a Store backed by an in-process map instead of S3. It runs the same compression, path-validation, metadata-encoding, and caching logic as New, so tests across the storage layer (and everything built on it — editrec, snapshot, portable, build, auth) get deterministic coverage without a live Minio. cacheSize behaves as in New (<= 0 disables the ARC cache).

func (*Store) Copy

func (s *Store) Copy(ctx context.Context, slug, srcPath, dstPath string) error

Copy duplicates `{slug}/{srcPath}` to `{slug}/{dstPath}`. Preserves the source object's content-type and user metadata. Evicts the destination from the ARC cache so subsequent Reads pick up the new object.

func (*Store) Delete

func (s *Store) Delete(ctx context.Context, slug, path string) error

Delete removes a single object at `{slug}/{path}`. Cache entry, if any, is evicted so subsequent Reads don't return a phantom object.

func (*Store) DeleteRaw

func (s *Store) DeleteRaw(ctx context.Context, key string) error

DeleteRaw removes an object by absolute bucket key.

func (*Store) EnsureBucket

func (s *Store) EnsureBucket(ctx context.Context) error

func (*Store) List

func (s *Store) List(ctx context.Context, slug string) ([]string, error)

func (*Store) ListApps

func (s *Store) ListApps(ctx context.Context) ([]string, error)

ListApps returns the slugs of every site in the bucket. Top-level prefixes that start with "_" are reserved (e.g. _snapshots/) and excluded — slugs are restricted to [a-z0-9-] so this can never hide a real app.

func (*Store) ListPrefix

func (s *Store) ListPrefix(ctx context.Context, prefix string) ([]string, error)

ListPrefix returns absolute bucket keys under the given prefix. Used to enumerate snapshot archives at `_snapshots/{slug}/`.

func (*Store) ListWithMeta

func (s *Store) ListWithMeta(ctx context.Context, slug string) ([]FileEntry, error)

ListWithMeta is like List but returns size and last-modified for each object, parsed from the listing response (no extra GETs). The flat List is kept for callers that only need names — changing its signature would touch every existing caller for no gain.

func (*Store) Read

func (s *Store) Read(ctx context.Context, slug, path string) (*S3Object, error)

func (*Store) ReadFresh

func (s *Store) ReadFresh(ctx context.Context, slug, path string) (*S3Object, error)

ReadFresh is Read with the ARC cache bypassed (and the stale entry dropped). The retry half of a compare-and-set needs it: a cached object carries the ETag that just lost, so re-reading through the cache would resubmit it.

func (*Store) ReadRaw

func (s *Store) ReadRaw(ctx context.Context, key string) (*S3Object, error)

ReadRaw is the symmetric counterpart to WriteRaw: fetches an object by absolute bucket key. Returns an S3Object with empty Content for missing keys (no error). Bypasses the ARC cache and does not URL-decode metadata.

func (*Store) Rename

func (s *Store) Rename(ctx context.Context, slug, srcPath, dstPath string) error

Rename moves an object from srcPath to dstPath by Copy+Delete. Returns nil without doing any work when src == dst. If Copy succeeds but Delete fails the new object exists alongside the old one — surviving but inconsistent — and the delete error is returned so the caller can surface it.

func (*Store) SumBytesUnderPrefix

func (s *Store) SumBytesUnderPrefix(ctx context.Context, prefix string) (PrefixStats, error)

SumBytesUnderPrefix aggregates total bytes + object count beneath an arbitrary bucket prefix in a single listing sweep — no per-object reads. Used by the system dashboard to break storage down by reserved area (_snapshots/, _edits/, _acme/, _state/) without round-tripping each archive. Returns a zero PrefixStats for a prefix with no objects so callers can render a zero row without special-casing missing folders.

func (*Store) UpdateMetadata

func (s *Store) UpdateMetadata(ctx context.Context, slug, path, contentType string, metadata map[string]string) error

UpdateMetadata replaces the user-defined metadata on `{slug}/{path}` without touching its bytes. Encoding mirrors Write (URL-escape values so unicode round-trips through ASCII-only metadata). Evicts the ARC cache so the next Read sees fresh metadata.

func (*Store) Write

func (s *Store) Write(ctx context.Context, slug, path, content, contentType string, metadata map[string]string) error

func (*Store) WriteConditional

func (s *Store) WriteConditional(
	ctx context.Context,
	slug, path, content, contentType string,
	metadata map[string]string,
	expectedETag string,
) (string, error)

WriteConditional is Write under a compare-and-set: the object must currently carry expectedETag, or — when expectedETag is "" — must not exist at all. Returns ErrPrecondition when someone else wrote in between, which is a "re-read and retry", never a fault.

It exists because a read-modify-write on a shared object is only safe if one writer can win. The per-site sidecar is the case that forced it: two people with access to the same site (owner and collaborator) can save settings and change the collaborator list at the same time, and a last-writer-wins round-trip silently discards whichever change was read first — including a revocation, which comes back.

On ErrPrecondition the cached copy is provably stale, so it's evicted: the caller's retry has to re-read from the backend or it would just resubmit the same losing ETag forever.

func (*Store) WriteRaw

func (s *Store) WriteRaw(ctx context.Context, key, content, contentType string, metadata map[string]string) error

WriteRaw writes to an arbitrary bucket key, bypassing the slug-prefix convention. Used by snapshot infrastructure that stores archives under a reserved `_snapshots/` prefix outside any user slug. No metadata encoding; pass already-ASCII values.

func (*Store) WriteRawIfAbsent

func (s *Store) WriteRawIfAbsent(ctx context.Context, key, content, contentType string, metadata map[string]string) (string, error)

WriteRawIfAbsent writes key only if nothing is there yet, returning ErrPrecondition when another writer got there first. The create-once half of the same primitive.

func (*Store) WriteRawIfMatch

func (s *Store) WriteRawIfMatch(ctx context.Context, key, content, contentType string, metadata map[string]string, expectedETag string) (string, error)

WriteRawIfMatch writes key only if it currently carries expectedETag — the compare-and-set half of a read-modify-write. Pass the ETag from the ReadRaw that produced the value you're basing the write on; returns ErrPrecondition if anything changed underneath.

This is what makes a claim atomic across processes. Without it, two instances that both read the same key both believe they may act on it, and "read it, then delete it" is not single use — it's two winners.