silod

Documentation

Overview

Package silod is the Silo file server daemon.

Index

• Constants
• Variables
• func DelFileFromTree(storeID, rootID, parentPath, filename string) (string, error)
• func DoPostMultiFiles(repo *repomgr.Repo, rootID, parentDir string, dents []*fsmgr.SeafDirent, ...) (string, error)
• func GenNewCommit(repo *repomgr.Repo, base *commitmgr.Commit, newRoot, user, desc string, ...) (string, error)
• func RecoverWrapper(f func())
• func Run(args []string) error
• func RunBackupDB(args []string) error
• func RunGC(args []string) error
• func RunToken(args []string) error
• type Dirents
  • func (d Dirents) Len() int
  • func (d Dirents) Less(i, j int) bool
  • func (d Dirents) Swap(i, j int)
• type LogFormatter
  • func (f *LogFormatter) Format(entry *log.Entry) ([]byte, error)
• type RepoInfo
• type RepoSizeChangeTask

Constants

const (
	InfiniteQuota = -2
)

InfiniteQuota indicates that the quota is unlimited.

const (
	RepoSizeList = "repo_size_task"
)

Variables

var (
	ErrConflict   = errors.New("concurrent upload conflict")
	ErrGCConflict = errors.New("GC Conflict")
	// ErrRetriesExhausted is a write that kept losing the race for the branch
	// head until its retry budget ran out. It is contention, not breakage:
	// nothing was applied, and the same request will usually succeed on a
	// retry. It exists so callers can tell that apart from a real failure —
	// without it the exhaustion case is an ordinary error and every HTTP
	// handler answers 500, which is the one class a client must not retry.
	ErrRetriesExhausted = errors.New("write contention: retries exhausted")
)

var ErrTimeout = fmt.Errorf("get fs id list timeout")

Functions

func DelFileFromTree

func DelFileFromTree(storeID, rootID, parentPath, filename string) (string, error)

delFileFromTree removes an entry from a directory tree and returns the new root ID. parentPath is the directory containing the entry, filename is the entry to remove. DelFileFromTree removes an entry from a directory tree and returns the new root ID.

func DoPostMultiFiles

func DoPostMultiFiles(repo *repomgr.Repo, rootID, parentDir string, dents []*fsmgr.SeafDirent, user string, replace bool, names *[]string) (string, error)

DoPostMultiFiles inserts entries into a directory tree and returns the new root ID.

func GenNewCommit

func GenNewCommit(repo *repomgr.Repo, base *commitmgr.Commit, newRoot, user, desc string, handleConncurrentUpdate bool, lastGCID string, checkGC bool) (string, error)

GenNewCommit creates a new commit with the given root and updates the branch.

func RecoverWrapper

func RecoverWrapper(f func())

func Run

func Run(args []string) error

Run starts the fileserver daemon. It parses its own flags from args (which should be everything after the `serve` subcommand), initializes databases and managers, and blocks until shutdown. It returns nil on graceful exit; any error during argument parsing or startup is returned to the caller. Note: much of the server bootstrap still uses log.Fatalf for fatal conditions, which calls os.Exit directly — that behavior is unchanged from the previous standalone binary.

func RunBackupDB

func RunBackupDB(args []string) error

RunBackupDB snapshots the SQLite databases into a destination directory.

The databases are the part of a Silo backup that cannot be copied with cp. They run in WAL mode, so the bytes in seafile.db lag the committed state by however much sits in seafile.db-wal, and Silo only checkpoints on a clean shutdown. Copying just the .db file off a running server therefore loses every write since the last checkpoint, silently and without any error to notice — and copying the -wal and -shm files alongside it is not a fix, because the three are read at different instants and need not agree.

VACUUM INTO takes a read transaction over the source and writes a single consistent, already-checkpointed file. It is safe while the server is running and blocks no writer.

The object store is not copied here. It is an ordinary immutable file tree that rsync handles better than anything worth writing, but the *order* matters and is the other half of what makes a naive backup wrong:

databases first, object store second.

Objects are content-addressed and never rewritten, so every object a head captured at T1 references is still present at T2. Copy the store first and the database after, and the heads can reference objects created in between — they are missing from the backup, and the repository restores broken. The one thing that can invalidate the rule is gc -delete removing objects mid-backup, so do not run GC and a backup together.

func RunGC

func RunGC(args []string) error

RunGC reclaims the object-store directories of deleted libraries.

DeleteRepo removes a library's database rows and records its ID in GarbageRepos, but nothing has ever reclaimed the objects, so deleted libraries leak disk indefinitely. This walks GarbageRepos and removes each dead library's directory from the commit, fs and block stores.

It is deliberately not a mark-and-sweep over live repos: it never inspects or deletes anything belonging to a library that still exists, which is what lets it run without reasoning about concurrent writes. Reclaiming unreferenced history *within* a live library is a separate, harder job.

Reporting is the default; -delete is required to remove anything.

func RunToken

func RunToken(args []string) error

RunToken lists and revokes the credentials a user holds.

Both stores could already revoke — repomgr.DeleteRepoToken, repomgr.DeleteRepoTokensByEmail and apitokenstore.DeleteByEmail were written for it and documented as the way to invalidate a token — but nothing outside their own tests ever called them. Sync tokens have no expiry by design, because Seafile clients persist them and treat them as durable, so with no reachable revocation a token copied off a stolen laptop kept read/write access to the library forever. A password change did not touch it. The only remedy was editing SQLite by hand.

This is a CLI rather than an HTTP endpoint because the account it most needs to work for is one whose credentials are already compromised, and because there is no admin role in the API layer to gate such an endpoint with — every authenticated user has equal permissions today.

Types

type Dirents

type Dirents []*fsmgr.SeafDirent

Dirents is an alias for slice of SeafDirent.

func (Dirents) Len

func (d Dirents) Len() int

func (Dirents) Less

func (d Dirents) Less(i, j int) bool

func (Dirents) Swap

func (d Dirents) Swap(i, j int)

type LogFormatter

type LogFormatter struct{}

func (*LogFormatter) Format

func (f *LogFormatter) Format(entry *log.Entry) ([]byte, error)

type RepoInfo

type RepoInfo struct {
	HeadID    string
	Size      int64
	FileCount int64
}

RepoInfo contains repo information.

type RepoSizeChangeTask

type RepoSizeChangeTask struct {
	RepoID string `json:"repo_id"`
}