drivesync

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 3, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

README

drivesync

drivesync shares an encrypted folder through ordinary directories on macOS and Linux. Create a folder with a name and a key, then attach it on each node. Reads are normal local files. Replicas sync both ways. There is no FUSE layer. Pure-Go builds are supported.

The authority stores ciphertext and control-plane records. It does not receive the folder key, file paths, file contents, or file metadata. SECURITY.md describes what the server can see, what a key holder can do, and the local-disk guarantees.

Quickstart

Run one authority process per metadata and blob store. auth authenticates each HTTP request and returns a Principal{Tenant, Subject}. plans implements QuotaPolicy.

meta, err := drivesync.OpenSQLiteMetaStore("authority.sqlite")
if err != nil { return err }
defer meta.Close()
blobs, err := drivesync.OpenDirectoryBlobStore("blobs")
if err != nil { return err }
defer blobs.Close()

srv := drivesync.NewServer(meta, blobs, drivesync.ServerOptions{Quotas: plans})
ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1)
go func() { done <- srv.Run(ctx) }()
defer func() { cancel(); <-done }()
mux.Handle("/drives/", http.StripPrefix("/drives", srv.Handler(auth)))

Run recovers interrupted uploads, expires reservations, collects garbage, and compacts tombstones. Cancel it and wait before closing the stores.

On a client node, or with srv.Client(principal) inside the authority process:

c := drivesync.NewHTTPClient("https://example.com/drives", headers)
key := drivesync.NewFolderKey()
f, err := c.CreateFolder(ctx, drivesync.FolderSpec{
    Name: "workspace", Description: "Files shared across nodes",
    Limits: drivesync.Limits{
        MaxFileBytes: 64 << 20, MaxTotalBytes: 1 << 30,
        MaxFiles: 10000, MaxRows: 50000,
    },
}, key)
if err != nil { return err }
if err := c.Grant(ctx, f.ID, bob, drivesync.Writer); err != nil { return err }
r, err := drivesync.Attach(ctx, c, f.ID, key, "./workspace", drivesync.Options{})
if err != nil { return err }
defer r.Close()
status := r.Status()

Preserve the key and distribute it to each attachment. Creation computes the key check locally. Folder ids are random 128-bit ids. Names are unique per primary owner. Grant, Revoke, SetLimits, ListFolders, GetFolder, and DeleteFolder manage the folder.

Attach's context bounds setup. The replica runs until Close. Sync forces one pass. Status reports pending bytes, conflicts, rejections, quarantine, and errors. Retry retries rejected or quarantined transfers and acknowledges an intentional mass removal.

A small CLI exercises the same API with one local principal. Set DRIVESYNC_TOKEN. Pass the key in DRIVESYNC_KEY or -key-file, never in argv.

go run ./cmd/drivesync serve -data ./drivesync-data
go run ./cmd/drivesync create -name shared
go run ./cmd/drivesync attach -folder ID -key-file ./folder.key -dir ./shared -state ./replica-state
go run ./cmd/drivesync status -state ./replica-state

Limits and quotas

Limits bounds sealed bytes, live entries, and retained rows. MaxFileBytes is the sealed size, including encryption overhead. MaxFiles counts live entries, including directory markers. MaxRows bounds retained rows and garbage. Zero MaxRows selects 16 times MaxFiles, capped at one million. Other zero limits mean unlimited.

Sharing, including a grant in the same tenant, requires a positive byte cap, row cap, and per-file cap. That capacity is charged to the primary owner until the folder is deleted and cleaned up. Reservations are atomic, and commit checks the stored size. Rejected files stay on the local disk with a reason and a retry backoff.

Folder reports id, name, description, owner, role, limits, and usage. Usage is sealed bytes, live entries, and reserved bytes.

Platform support

macOS and Linux are supported, including CGO_ENABLED=0. macOS watches the tree with FSEvents through purego. Linux uses inotify. Both debounce for 150 ms and rescan periodically. kqueue is not used. Other operating systems are not supported.

How it works

A replica scans its directory, encrypts new bytes, and uploads ciphertext. Peers download into a staging file, authenticate it, then publish with fsync and rename. Renames commit as one create/delete pair. A losing local write is kept as a conflict copy. Matching authenticated content is adopted in place, so an existing tree can be attached without uploading those bytes again.

Path ids, metadata, and content use separate keys. Chunks are bound to the folder, blob, path, index, and a final flag. Case collisions keep both contents under distinct local names. Only the owner execute bit syncs. Group and other permission bits stay local.

Default state lives in os.UserCacheDir()/drivesync/<folder>/<root-hash> with private permissions. It holds plaintext paths, so keep it outside other shared folders. Options.StateDir must be dedicated to that attachment. A nonempty directory without the matching state marker is refused. .drivesync* names are reserved. .drivesyncignore accepts globs, root-anchored / patterns, **, comments, and directory patterns, matched under case-folded NFC. Negation and escapes are reported as unsupported.

.git/ is ignored by default. Sync the working tree, and let git carry history.

The included blob stores are an in-memory store and a durable local directory. BlobStore.Put must publish only after a clean EOF. An embedder can add an object-store backend without taking an SDK dependency. Metadata is SQLite. QuotaPolicy is the other extension point.

Fresh databases are required. This version does not migrate older schemas, and it does not provide historical recovery, key rotation, or chunk deduplication.

License

The code is licensed under Apache-2.0. Copyright 2026 Parham Negahdar. See LICENSE.

Unicode path handling is checked against the Unicode Consortium's NormalizationTest data, which remains under the Unicode License. See UNICODE_LICENSE.txt. The corpus is internal/engine/testdata/NormalizationTest.txt.gz.

Documentation

Overview

Package drivesync shares encrypted folders through ordinary local directories. Create a folder with a name and a key, then attach it wherever files are needed.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrDenied    = engine.ErrDenied
	ErrBusy      = engine.ErrBusy
	ErrExpired   = engine.ErrExpired
	ErrQuota     = engine.ErrQuota
	ErrConflict  = engine.ErrConflict
	ErrInvalid   = engine.ErrInvalid
	ErrKey       = engine.ErrKey
	ErrIntegrity = engine.ErrIntegrity
	ErrWaitLimit = engine.ErrWaitLimit
)

Functions

This section is empty.

Types

type Authenticator 

type Authenticator func(*http.Request) (Principal, error)

type BlobStore 

type BlobStore interface {
	Put(context.Context, string, string, io.Reader) (int64, error)
	Open(context.Context, string, string) (io.ReadCloser, error)
	Size(context.Context, string, string) (int64, error)
	Delete(context.Context, string, string) error
}

BlobStore holds immutable sealed blobs. Keys are authority-generated IDs. Put streams bytes, Size reports actual stored bytes, and Delete is idempotent (os.ErrNotExist is also accepted). Implementations must honor cancellation. Put must publish only after clean EOF; a reader error must discard staging.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client manages folders through HTTP or an authenticated in-process authority. Its replication and encryption protocol is private to the library.

func NewHTTPClient 

func NewHTTPClient(url string, headers http.Header) *Client

NewHTTPClient connects to an authority. Use HTTPS outside local tests.

func (*Client) CreateFolder 

func (c *Client) CreateFolder(ctx context.Context, spec FolderSpec, key FolderKey) (Folder, error)

CreateFolder computes the key check locally; the key is never sent to the authority. Keep it securely and supply the same key to every attachment.

func (*Client) DeleteFolder 

func (c *Client) DeleteFolder(ctx context.Context, id string) error

func (*Client) GetFolder 

func (c *Client) GetFolder(ctx context.Context, id string) (Folder, error)

func (*Client) Grant 

func (c *Client) Grant(ctx context.Context, id string, p Principal, role Role) error

func (*Client) ListFolders 

func (c *Client) ListFolders(ctx context.Context) ([]Folder, error)

func (*Client) Revoke 

func (c *Client) Revoke(ctx context.Context, id string, p Principal) error

func (*Client) SetLimits 

func (c *Client) SetLimits(ctx context.Context, id string, limits Limits) error

type DirectoryBlobStore 

type DirectoryBlobStore struct {
	// contains filtered or unexported fields
}

DirectoryBlobStore streams sealed bytes to a durable local directory.

func OpenDirectoryBlobStore 

func OpenDirectoryBlobStore(path string) (*DirectoryBlobStore, error)

func (*DirectoryBlobStore) Close 

func (b *DirectoryBlobStore) Close() error

func (*DirectoryBlobStore) Delete 

func (b *DirectoryBlobStore) Delete(ctx context.Context, folder, id string) error

func (*DirectoryBlobStore) Open 

func (b *DirectoryBlobStore) Open(ctx context.Context, folder, id string) (io.ReadCloser, error)

func (*DirectoryBlobStore) Put 

func (b *DirectoryBlobStore) Put(ctx context.Context, folder, id string, r io.Reader) (int64, error)

func (*DirectoryBlobStore) Size 

func (b *DirectoryBlobStore) Size(ctx context.Context, folder, id string) (int64, error)

type Folder 

type Folder struct {
	ID                string
	Name, Description string
	Owner             Principal
	Role              Role
	Limits            Limits
	Usage             Usage
}

type FolderKey 

type FolderKey [32]byte

func NewFolderKey 

func NewFolderKey() FolderKey

func ParseFolderKey 

func ParseFolderKey(s string) (FolderKey, error)

ParseFolderKey imports a 64-character hex encoding of a random 256-bit key. Passphrases and hashes of passphrases are not suitable folder keys.

type FolderSpec 

type FolderSpec struct {
	Name, Description string
	Limits            Limits
}

type LimitError 

type LimitError struct {
	Limit              string
	Maximum, Requested int64
	// contains filtered or unexported fields
}

LimitError identifies a folder limit that rejected an operation.

func (*LimitError) Error 

func (e *LimitError) Error() string

func (*LimitError) Unwrap 

func (e *LimitError) Unwrap() error

type Limits 

type Limits struct{ MaxFileBytes, MaxTotalBytes, MaxFiles, MaxRows int64 }

Limits bounds sealed bytes, live entries and retained rows. Zero is unlimited except MaxRows, which selects a bounded safety budget. Sharing requires explicit positive byte, row and file caps. Bytes include encryption and metadata overhead.

type MemoryBlobStore 

type MemoryBlobStore struct {
	// contains filtered or unexported fields
}

MemoryBlobStore is useful for tests and small in-process deployments.

func NewMemoryBlobStore 

func NewMemoryBlobStore() *MemoryBlobStore

func (*MemoryBlobStore) Delete 

func (b *MemoryBlobStore) Delete(ctx context.Context, folder, id string) error

func (*MemoryBlobStore) Open 

func (b *MemoryBlobStore) Open(ctx context.Context, folder, id string) (io.ReadCloser, error)

func (*MemoryBlobStore) Put 

func (b *MemoryBlobStore) Put(ctx context.Context, folder, id string, r io.Reader) (int64, error)

func (*MemoryBlobStore) Size 

func (b *MemoryBlobStore) Size(ctx context.Context, folder, id string) (int64, error)

type MetaStore 

type MetaStore struct {
	// contains filtered or unexported fields
}

MetaStore is the SQLite authority store. Metadata storage is concrete because a custom transaction interface would expose private replication/accounting records. BlobStore and QuotaPolicy remain the pluggable extension points.

func OpenSQLiteMetaStore 

func OpenSQLiteMetaStore(path string) (*MetaStore, error)

func (*MetaStore) Close 

func (m *MetaStore) Close() error

type Options 

type Options struct {
	Name, StateDir                          string
	Debounce, RescanInterval, RetryInterval time.Duration
	Ignore                                  []string
	Manual                                  bool
}

Options configures a local replica. StateDir must be dedicated and outside all synced trees; its default is a folder/root directory in the user's cache. Manual disables automatic sync; Sync remains available explicitly.

type Principal 

type Principal struct{ Tenant, Subject string }

Principal is an opaque identity supplied by the embedding authenticator.

type Quota 

type Quota struct{ MaxTotalBytes, MaxFolders, MaxFileBytes, MaxFiles int64 }

Quota is an owner's plan. MaxFiles bounds retained rows across their folders.

type QuotaFunc 

type QuotaFunc func(context.Context, Principal) (Quota, error)

func (QuotaFunc) Quota 

func (f QuotaFunc) Quota(ctx context.Context, p Principal) (Quota, error)

type QuotaPolicy 

type QuotaPolicy interface {
	Quota(context.Context, Principal) (Quota, error)
}

type Rejection 

type Rejection struct{ Path, Reason string }

type Replica 

type Replica struct {
	// contains filtered or unexported fields
}

Replica keeps an encrypted folder in an ordinary local directory.

func Attach 

func Attach(ctx context.Context, c *Client, id string, key FolderKey, dir string, opts Options) (*Replica, error)

Attach uses ctx for setup. The replica remains active until Close.

func (*Replica) Close 

func (r *Replica) Close() error

func (*Replica) Retry 

func (r *Replica) Retry()

Retry clears rejection/quarantine backoff and acknowledges intentional mass removal for the next sync. It never overrides a missing or changed root identity.

func (*Replica) Status 

func (r *Replica) Status() Status

func (*Replica) Sync 

func (r *Replica) Sync(ctx context.Context) error

type Role 

type Role string
const (
	Owner  Role = "owner"
	Writer Role = "writer"
	Reader Role = "reader"
)

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server authorizes every operation independently of transport authentication.

func NewServer 

func NewServer(meta *MetaStore, blobs BlobStore, opts ServerOptions) *Server

func (*Server) Client 

func (s *Server) Client(p Principal) *Client

Client trusts p, which the embedding app must authenticate before calling.

func (*Server) Handler 

func (s *Server) Handler(auth Authenticator) http.Handler

func (*Server) Run 

func (s *Server) Run(ctx context.Context) error

Run recovers interrupted uploads and maintains expired reservations, garbage and tombstones until ctx is cancelled. Use a single authority process for these metadata/blob stores: recovery cannot prove another process stopped publishing. Cancel and join Run before closing the stores.

type ServerOptions 

type ServerOptions struct {
	Quotas                                   QuotaPolicy
	ReservationTTL, TombstoneTTL, GCInterval time.Duration
	Clock                                    func() time.Time
}

ServerOptions configures authority policy and maintenance before serving. Zero TTLs select defaults: five-minute reservations and 30-day tombstones. A negative TombstoneTTL disables compaction. GCInterval defaults to one second.

type Status 

type Status struct {
	PendingUpBytes, PendingDownBytes int64
	Conflicts                        []string
	Rejected                         []Rejection
	Quarantined                      []string
	LastSync                         time.Time
	Errors                           []string
}

type Usage 

type Usage struct{ Bytes, Files, Reserved int64 }

Source Files

View all Source files

Directories

Path Synopsis
cmd
drivesync command
drivesync is a small manual-testing CLI.
 drivesync is a small manual-testing CLI.
internal
engine
Package engine implements the private encrypted replication protocol.
 Package engine implements the private encrypted replication protocol.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.