README
¶
Quark
Your own private cloud, running in your house. Photos, files, documents — all on hardware you own, off servers you don't trust.
What is it?
Quark is a self-hosted personal cloud that runs on a device in your home. Think Google Drive or iCloud, except the data never leaves your house and nobody's training AI on your family photos.
You buy the hardware once. No subscriptions. No data mining. It's yours.
Stack
- Go + Gin — backend server
- Flutter — cross-platform frontend (web, iOS, Android)
- SQLite — embedded local database
- OpenTelemetry — observability
Getting started
Prerequisites: Go, Flutter, Make, air, sqlc, swag
git clone https://github.com/autobutler-org/quark.git # Include --recursive if you want to do OS image builds
cd quark
make setup
make generate
make build
Run it locally
Backend with hot reload. Pick a mode — they differ in scheme and port:
make watch/backend # plain HTTP on http://localhost:8080
make watch/backend/secure # HTTPS on https://localhost (:443), self-signed cert
The secure mode generates a self-signed certificate on first boot. It is not
added to any trust store, so curl needs -k and browsers will warn; the
Flutter app skips chain verification for local addresses on purpose. Binding
:443 requires root on Linux — see AS_ROOT=1 below.
Without hot reload, make serve/backend and make serve/backend/secure are the
same two modes.
Frontend (web):
make serve/frontend
Frontend (mobile emulator):
make emulate # default platform
make emulate/android # or emulate/ios
make serve/frontend/mobile
Container
Runs a published release, not your working tree:
make build/docker # build the image for the latest released tag
make serve/docker # run it on http://localhost:8080
make serve/docker DOCKER_PORT=9000 # different host port
make serve/docker DOCKER_DATA=/abs/path # use a Quark directory on the host
make clean/docker # remove the container
It downloads the tarball for a tagged version, so it will not pick up local
changes — make watch/backend is still the development loop. Use this to check
the shipped artifact, to reproduce something a user reports on a specific
version, or to try Quark without a Go and Flutter toolchain.
State lives in the quark-data volume by default and persists between runs.
See docs/container.md for Kubernetes, the uid 999 the data
directory must be owned by on Linux, and the rest.
USB device mounting requires root on Linux, as does binding
:443in secure mode. UseAS_ROOT=1with any backend target if you need it — e.g.make watch/backend/secure AS_ROOT=1.
Swagger UI is at http://localhost:8080/swagger in insecure mode, or
https://localhost/swagger in secure mode.
Other useful commands
make check # lint
make format # format
make help # all targets
Docs
Contributing
We use linear commit history — one focused commit per PR is the norm. See CONTRIBUTING.md for the full rundown.
License
MIT No Attribution (MIT-0). See LICENSE.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
knowledge
command
Command knowledge builds a map of this repository out of the repository itself: Go packages and the imports between them, the HTTP routes in the swagger spec, the SQLite tables and the sqlc queries that touch them, the Flutter tree, and the user journeys.
|
Command knowledge builds a map of this repository out of the repository itself: Go packages and the imports between them, the HTTP routes in the swagger spec, the SQLite tables and the sqlc queries that touch them, the Flutter tree, and the user journeys. |
|
lsusb
command
Command lsusb prints the USB devices Quark can see, with their partitions and mount points; -storage narrows the list to storage devices.
|
Command lsusb prints the USB devices Quark can see, with their partitions and mount points; -storage narrows the list to storage devices. |
|
provisioning
command
Command provisioning mints single-use Headscale pre-auth keys for quarks that POST /provision (#1876).
|
Command provisioning mints single-use Headscale pre-auth keys for quarks that POST /provision (#1876). |
|
quark
command
Command quark is the Quark binary: quark serve runs the API and the embedded web app, quark install sets it up as a system service, and quark version reports the build.
|
Command quark is the Quark binary: quark serve runs the API and the embedded web app, quark install sets it up as a system service, and quark version reports the build. |
|
quark/install
Package install is the quark install subcommand, which sets Quark up as a system service through internal/install.
|
Package install is the quark install subcommand, which sets Quark up as a system service through internal/install. |
|
quark/serve
Package serve is the quark serve subcommand: it builds the default dependency graph and starts the HTTP server, over plain HTTP when --insecure is set.
|
Package serve is the quark serve subcommand: it builds the default dependency graph and starts the HTTP server, over plain HTTP when --insecure is set. |
|
quark/version
Package version is the quark version subcommand, which prints the version this binary was built as.
|
Package version is the quark version subcommand, which prints the version this binary was built as. |
|
docs
|
|
|
swagger
Package swagger Code generated by swaggo/swag.
|
Package swagger Code generated by swaggo/swag. |
|
internal
|
|
|
db
Package db is Quark's SQLite layer: the connection, the golang-migrate migrations under migrations/, and the query methods sqlc generates from sql/queries/.
|
Package db is Quark's SQLite layer: the connection, the golang-migrate migrations under migrations/, and the query methods sqlc generates from sql/queries/. |
|
db/dbtest
Package dbtest opens a real Quark database for a test.
|
Package dbtest opens a real Quark database for a test. |
|
install
Package install sets Quark up as a systemd service on Linux, along with the root-owned helpers the service is allowed to call but not rewrite.
|
Package install sets Quark up as a systemd service on Linux, along with the root-owned helpers the service is allowed to call but not rewrite. |
|
server
Package server assembles the HTTP server: the gin engine and its middleware, every /api/v0 router (mounted in routes.go), Swagger, and the embedded web build served as a single-page app.
|
Package server assembles the HTTP server: the gin engine and its middleware, every /api/v0 router (mounted in routes.go), Swagger, and the embedded web build served as a single-page app. |
|
server/api/v0
Package v0 holds the version-0 HTTP API: one handler package per URL segment under /api/v0, each mounted in internal/server/routes.go.
|
Package v0 holds the version-0 HTTP API: one handler package per URL segment under /api/v0, each mounted in internal/server/routes.go. |
|
server/api/v0/access
Package v0_access serves /api/v0/access: the path-based grants that share a folder with a user or group — listing, setting and revoking them — and what has been shared with the caller.
|
Package v0_access serves /api/v0/access: the path-based grants that share a folder with a user or group — listing, setting and revoking them — and what has been shared with the caller. |
|
server/api/v0/admin
Package v0_admin serves the admin-only /api/v0/admin routes: creating, approving, denying, promoting, demoting, disabling, enabling and deleting users, groups and their members, and repairing the installation.
|
Package v0_admin serves the admin-only /api/v0/admin routes: creating, approving, denying, promoting, demoting, disabling, enabling and deleting users, groups and their members, and repairing the installation. |
|
server/api/v0/albums
Package v0_albums serves /api/v0/albums: the caller's photo albums, as a list or a nested tree, and the photos in them.
|
Package v0_albums serves /api/v0/albums: the caller's photo albums, as a list or a nested tree, and the photos in them. |
|
server/api/v0/auth
Package v0_auth serves /api/v0/auth: first-boot setup, login and logout, account recovery, requests for an account, and the caller's sessions and account.
|
Package v0_auth serves /api/v0/auth: first-boot setup, login and logout, account recovery, requests for an account, and the caller's sessions and account. |
|
server/api/v0/books
Package v0_books serves /api/v0/books, which finds the books in the files directory.
|
Package v0_books serves /api/v0/books, which finds the books in the files directory. |
|
server/api/v0/chat
Package v0_chat serves /api/v0/chat: the chat channels the caller belongs to, creating, renaming and deleting channels, listing and changing a channel's members, and each account's chat keys, where /chat/keys/me holds the caller's wrapped identity and /chat/keys/:userId serves any account's public keys.
|
Package v0_chat serves /api/v0/chat: the chat channels the caller belongs to, creating, renaming and deleting channels, listing and changing a channel's members, and each account's chat keys, where /chat/keys/me holds the caller's wrapped identity and /chat/keys/:userId serves any account's public keys. |
|
server/api/v0/devices
Package v0_devices serves /api/v0/devices: the clients that have connected to this Quark.
|
Package v0_devices serves /api/v0/devices: the clients that have connected to this Quark. |
|
server/api/v0/events
Package v0_events serves /api/v0/events, the WebSocket that streams event-bus events to open clients, filtered to what each user can read, so every client refreshes when the file tree changes.
|
Package v0_events serves /api/v0/events, the WebSocket that streams event-bus events to open clients, filtered to what each user can read, so every client refreshes when the file tree changes. |
|
server/api/v0/favorites
Package v0_favorites serves the photo favorites, which sit under /api/v0/photos (/photos/favorite and /photos/favorites): marking a photo as a favorite, checking whether one is, and listing them.
|
Package v0_favorites serves the photo favorites, which sit under /api/v0/photos (/photos/favorite and /photos/favorites): marking a photo as a favorite, checking whether one is, and listing them. |
|
server/api/v0/files
Package v0_files serves /api/v0/files: listing, searching and stat-ing files, uploads (including resumable upload sessions), downloads and archive views, and moves, deletes and new folders.
|
Package v0_files serves /api/v0/files: listing, searching and stat-ing files, uploads (including resumable upload sessions), downloads and archive views, and moves, deletes and new folders. |
|
server/api/v0/health
Package v0_health serves /api/v0/health, the device metrics the Health page shows.
|
Package v0_health serves /api/v0/health, the device metrics the Health page shows. |
|
server/api/v0/jobs
Package v0_jobs serves /api/v0/jobs: listing background jobs, and getting, canceling or retrying one.
|
Package v0_jobs serves /api/v0/jobs: listing background jobs, and getting, canceling or retrying one. |
|
server/api/v0/photos
Package v0_photos serves /api/v0/photos: the photo library listing, a photo's metadata, likely duplicates, and rotating or copying a photo.
|
Package v0_photos serves /api/v0/photos: the photo library listing, a photo's metadata, likely duplicates, and rotating or copying a photo. |
|
server/api/v0/settings
Package v0_settings serves /api/v0/settings: reading the Quark's settings, pairing a device for remote access (POST /settings/remote-access/devices, open to any signed-in user), and the admin-only routes that change appliance-wide ones such as turning remote access on and off, account requests, and the chat beta.
|
Package v0_settings serves /api/v0/settings: reading the Quark's settings, pairing a device for remote access (POST /settings/remote-access/devices, open to any signed-in user), and the admin-only routes that change appliance-wide ones such as turning remote access on and off, account requests, and the chat beta. |
|
server/api/v0/ssh
Package v0_ssh serves the admin-only /api/v0/ssh routes: turning SSH access on and off, setting or clearing its password, and managing its authorized keys.
|
Package v0_ssh serves the admin-only /api/v0/ssh routes: turning SSH access on and off, setting or clearing its password, and managing its authorized keys. |
|
server/api/v0/storage
Package v0_storage serves /api/v0/storage: the storage devices Quark manages and their snapshot backups.
|
Package v0_storage serves /api/v0/storage: the storage devices Quark manages and their snapshot backups. |
|
server/api/v0/thumbnails
Package v0_thumbnails serves /api/v0/thumbnails, the cached image previews the photo and file views load, and takes the thumbnails clients render for files the device cannot decode (PUT).
|
Package v0_thumbnails serves /api/v0/thumbnails, the cached image previews the photo and file views load, and takes the thumbnails clients render for files the device cannot decode (PUT). |
|
server/api/v0/trash
Package v0_trash serves /api/v0/trash: listing a device's trash, restoring items to where they were deleted from, and deleting them for good.
|
Package v0_trash serves /api/v0/trash: listing a device's trash, restoring items to where they were deleted from, and deleting them for good. |
|
server/api/v0/users
Package v0_users serves /api/v0/users, profile pictures: PUT and DELETE /users/me/avatar change the caller's own, and GET /users/{id}/avatar serves anyone's to any signed-in user.
|
Package v0_users serves /api/v0/users, profile pictures: PUT and DELETE /users/me/avatar change the caller's own, and GET /users/{id}/avatar serves anyone's to any signed-in user. |
|
server/api/v0/vault
Package v0_vault serves /api/v0/vault, the encrypted password vault: setup, unlock and lock, entries and folders, password generation, and import and export.
|
Package v0_vault serves /api/v0/vault, the encrypted password vault: setup, unlock and lock, entries and folders, password generation, and import and export. |
|
server/api/v0/version
Package v0_version serves /api/v0/version: the installed version, the SBOM, the versions available, and, for admins, updating to one.
|
Package v0_version serves /api/v0/version: the installed version, the SBOM, the versions available, and, for admins, updating to one. |
|
server/api/v0/videos
Package v0_videos serves /api/v0/videos: video metadata, the transcode formats on offer, and transcoding, trimming or extracting a frame.
|
Package v0_videos serves /api/v0/videos: video metadata, the transcode formats on offer, and transcoding, trimming or extracting a frame. |
|
server/middleware
Package middleware is the gin middleware every request passes through — dependency injection, connected-device tracking, and session auth with its short list of exempt paths — plus RequireAdmin for the admin group and RequireChatEnabled for the chat routes.
|
Package middleware is the gin middleware every request passes through — dependency injection, connected-device tracking, and session auth with its short list of exempt paths — plus RequireAdmin for the admin group and RequireChatEnabled for the chat routes. |
|
pkg
|
|
|
backup
Package backup snapshots the vault and managed-device files onto a target device, keeps them in sync as files change, and verifies restores.
|
Package backup snapshots the vault and managed-device files onto a target device, keeps them in sync as files change, and verifies restores. |
|
calendar
Package calendar provides calendar utilities for date/month/weekday operations.
|
Package calendar provides calendar utilities for date/month/weekday operations. |
|
util/accessutil
Package accessutil answers who may reach a path (#1902).
|
Package accessutil answers who may reach a path (#1902). |
|
util/albumutil
Package albumutil holds the photo album writes that carry naming rules: an album name cannot contain '/', and it is unique among its siblings ignoring case.
|
Package albumutil holds the photo album writes that carry naming rules: an album name cannot contain '/', and it is unique among its siblings ignoring case. |
|
util/authutil
Package authutil handles authentication: password hashing, recovery phrases, first-boot setup, login, session issue and validation, account status, and admin role management.
|
Package authutil handles authentication: password hashing, recovery phrases, first-boot setup, login, session issue and validation, account status, and admin role management. |
|
util/avatarutil
Package avatarutil stores, serves and removes profile pictures.
|
Package avatarutil stores, serves and removes profile pictures. |
|
util/bookutil
Package bookutil finds book files (PDF and EPUB) on disk.
|
Package bookutil finds book files (PDF and EPUB) on disk. |
|
util/chatutil
Package chatutil manages chat channels and who belongs to them (#2415).
|
Package chatutil manages chat channels and who belongs to them (#2415). |
|
util/ctxutil
Package ctxutil stores and fetches typed values on a gin context.
|
Package ctxutil stores and fetches typed values on a gin context. |
|
util/deputil
Package deputil carries the server's dependency graph: the Dependencies interface every layer is handed, and the constructors that build it either empty (for tests) or wired to the real database, storage, and VFS.
|
Package deputil carries the server's dependency graph: the Dependencies interface every layer is handed, and the constructors that build it either empty (for tests) or wired to the real database, storage, and VFS. |
|
util/deviceutil
Package deviceutil holds the services behind /api/v0/storage/devices: the mount and unmount of a USB storage device, the display name and role the database keeps for each device, and the status list that overlays the two onto what the detector reports.
|
Package deviceutil holds the services behind /api/v0/storage/devices: the mount and unmount of a USB storage device, the display name and role the database keeps for each device, and the status list that overlays the two onto what the detector reports. |
|
util/downloadutil
Package downloadutil issues the tokens a browser download authenticates with.
|
Package downloadutil issues the tokens a browser download authenticates with. |
|
util/eventbus
Package eventbus is the in-process publish/subscribe bus behind /api/v0/events.
|
Package eventbus is the in-process publish/subscribe bus behind /api/v0/events. |
|
util/favoritesutil
Package favoritesutil manages a user's Favorites, a system photo album created on first use, and toggles photos in and out of it.
|
Package favoritesutil manages a user's Favorites, a system photo album created on first use, and toggles photos in and out of it. |
|
util/ffmpegutil
Package ffmpegutil wraps video inspection, thumbnails, and trimming behind the VideoProcessor interface, with CLIProcessor as the ffmpeg/ffprobe-backed implementation.
|
Package ffmpegutil wraps video inspection, thumbnails, and trimming behind the VideoProcessor interface, with CLIProcessor as the ffmpeg/ffprobe-backed implementation. |
|
util/fileutil
Package fileutil holds the services behind /api/v0/files: the listings the file browser pages through, the archive views that read only headers, the download pipeline that zips a folder or re-encodes an image on its way out, and the mutations (delete, move, new folder) the browser sends back.
|
Package fileutil holds the services behind /api/v0/files: the listings the file browser pages through, the archive views that read only headers, the download pipeline that zips a folder or re-encodes an image on its way out, and the mutations (delete, move, new folder) the browser sends back. |
|
util/githubutil
Package githubutil fetches release metadata from the GitHub REST API.
|
Package githubutil fetches release metadata from the GitHub REST API. |
|
util/grouputil
Package grouputil manages the groups admins put accounts in (#1910).
|
Package grouputil manages the groups admins put accounts in (#1910). |
|
util/healthutil
Package healthutil provides gopsutil-based host hardware health metrics.
|
Package healthutil provides gopsutil-based host hardware health metrics. |
|
util/iosemutil
Package iosemutil provides a shared IO concurrency semaphore to prevent disk thrashing under concurrent load — particularly on spinning hard drives where random-access seeks collapse throughput when too many goroutines compete.
|
Package iosemutil provides a shared IO concurrency semaphore to prevent disk thrashing under concurrent load — particularly on spinning hard drives where random-access seeks collapse throughput when too many goroutines compete. |
|
util/jobutil
Package jobutil runs background jobs.
|
Package jobutil runs background jobs. |
|
util/photoutil
Package photoutil reads, transforms, and thumbnails photo and video files: discovering photos on disk, extracting EXIF metadata, correcting orientation, converting camera RAW, generating thumbnails, and comparing images by perceptual hash.
|
Package photoutil reads, transforms, and thumbnails photo and video files: discovering photos on disk, extracting EXIF metadata, correcting orientation, converting camera RAW, generating thumbnails, and comparing images by perceptual hash. |
|
util/provisionutil
Package provisionutil asks the provisioning service (cmd/provisioning) for a Headscale pre-auth key, so enabling remote access never needs a key from the user (#1876).
|
Package provisionutil asks the provisioning service (cmd/provisioning) for a Headscale pre-auth key, so enabling remote access never needs a key from the user (#1876). |
|
util/ratelimitutil
Package ratelimitutil provides a per-IP token-bucket rate limiter for protecting sensitive endpoints (e.g.
|
Package ratelimitutil provides a per-IP token-bucket rate limiter for protecting sensitive endpoints (e.g. |
|
util/remoteutil
Package remoteutil runs an embedded Tailscale node and reverse-proxies tailnet traffic to the local quark server.
|
Package remoteutil runs an embedded Tailscale node and reverse-proxies tailnet traffic to the local quark server. |
|
util/repairutil
Package repairutil lets an admin repair the Quark installation (#2121).
|
Package repairutil lets an admin repair the Quark installation (#2121). |
|
util/searchutil
Package searchutil provides text extraction and full-text search over indexed file contents using SQLite FTS5.
|
Package searchutil provides text extraction and full-text search over indexed file contents using SQLite FTS5. |
|
util/serverutil
Package serverutil holds the HTTP building blocks the server layer is assembled from: the Response envelope handlers return, the Route/Router registration types that wire handlers onto gin, the HttpError type that lets a service function name its own status code, and the ports the server binds.
|
Package serverutil holds the HTTP building blocks the server layer is assembled from: the Response envelope handlers return, the Route/Router registration types that wire handlers onto gin, the HttpError type that lets a service function name its own status code, and the ports the server binds. |
|
util/settingsutil
Package settingsutil reads and writes the Quark's settings, which live in settings.json in the data directory rather than in the database.
|
Package settingsutil reads and writes the Quark's settings, which live in settings.json in the data directory rather than in the database. |
|
util/sqlutil
Package sqlutil holds small conversions between Go values and database/sql types, and recognizes SQLite's unique-constraint errors.
|
Package sqlutil holds small conversions between Go values and database/sql types, and recognizes SQLite's unique-constraint errors. |
|
util/sshutil
Package sshutil lets an admin turn SSH access to the Quark on and off, and manage how the quark login account signs in: public keys and an optional password (#2131).
|
Package sshutil lets an admin turn SSH access to the Quark on and off, and manage how the quark login account signs in: public keys and an optional password (#2131). |
|
util/storageutil
Package storageutil is the storage layer beneath the VFS: detecting and mounting USB drives and their partitions, device status and roles, the trash, reading and extracting archives, and the file operations the services call.
|
Package storageutil is the storage layer beneath the VFS: detecting and mounting USB drives and their partitions, device status and roles, the trash, reading and extracting archives, and the file operations the services call. |
|
util/thumbnailutil
Package thumbnailutil generates and caches image and video thumbnails on disk.
|
Package thumbnailutil generates and caches image and video thumbnails on disk. |
|
util/tlsutil
Package tlsutil provides helpers for provisioning self-signed TLS certificates used by the Quark server when running in production (HTTPS) mode.
|
Package tlsutil provides helpers for provisioning self-signed TLS certificates used by the Quark server when running in production (HTTPS) mode. |
|
util/transcodeutil
Package transcodeutil is the video-transcode job kind: Enqueue validates and queues a conversion into any video format, and the Handler from NewHandler writes the result into a new file beside the source when the job runs.
|
Package transcodeutil is the video-transcode job kind: Enqueue validates and queues a conversion into any video format, and the Handler from NewHandler writes the result into a new file beside the source when the job runs. |
|
util/updateutil
Package updateutil discovers, downloads, and installs quark releases from GitHub and Azure Blob Storage.
|
Package updateutil discovers, downloads, and installs quark releases from GitHub and Azure Blob Storage. |
|
util/uploadutil
Package uploadutil owns the server side of file uploads: where a finished upload lands, and the resumable chunked sessions that feed it (#1629).
|
Package uploadutil owns the server side of file uploads: where a finished upload lands, and the resumable chunked sessions that feed it (#1629). |
|
util/vaultcrypto
Package vaultcrypto derives vault keys with Argon2id, encrypts vault data with AES-256-GCM, and holds the derived key in an auto-locking session.
|
Package vaultcrypto derives vault keys with Argon2id, encrypts vault data with AES-256-GCM, and holds the derived key in an auto-locking session. |
|
util/vaultutil
Package vaultutil holds the password vault's business logic: entry and folder CRUD over the encrypted store, master-password setup, unlock and rotation, the importer for CSV/JSON exports from other password managers, the plaintext exporter, password generation, and moving the whole vault to another device.
|
Package vaultutil holds the password vault's business logic: entry and folder CRUD over the encrypted store, master-password setup, unlock and rotation, the importer for CSV/JSON exports from other password managers, the plaintext exporter, password generation, and moving the whole vault to another device. |
|
util/versionutil
Package versionutil reports and compares the build version of the binary.
|
Package versionutil reports and compares the build version of the binary. |
|
util/videoutil
Package videoutil provides a thin wrapper around ffmpeg and ffprobe for video probing, frame extraction, trimming, and transcoding.
|
Package videoutil provides a thin wrapper around ffmpeg and ffprobe for video probing, frame extraction, trimming, and transcoding. |
|
util/workerutil
Package workerutil runs the background worker that processes backup-to-device requests off a channel and reports their errors.
|
Package workerutil runs the background worker that processes backup-to-device requests off a channel and reports their errors. |
|
util/xlsxutil
Package xlsxutil reads an OOXML spreadsheet (.xlsx / .xlsm) and rewrites it as the .qsheet JSON envelope Quark's Sheets editor already understands:
|
Package xlsxutil reads an OOXML spreadsheet (.xlsx / .xlsm) and rewrites it as the .qsheet JSON envelope Quark's Sheets editor already understands: |
|
vfs
Package vfs provides the virtual filesystem layer: a common file interface over local disk, in-memory, database-backed and storage-service namespaces, a registry that maps namespaces to implementations, and per-path metadata.
|
Package vfs provides the virtual filesystem layer: a common file interface over local disk, in-memory, database-backed and storage-service namespaces, a registry that maps namespaces to implementations, and per-path metadata. |
|
test
|
|
|
chaos
Package chaos holds defensive API stress/chaos checks for Quark.
|
Package chaos holds defensive API stress/chaos checks for Quark. |
Click to show internal directories.
Click to hide internal directories.