README
¶
Kopia
n.
Kopia is a simple, cross-platform tool for managing encrypted backups in the cloud. It provides fast, incremental backups, secure, client-side end-to-end encryption, compression and data deduplication.
Unlike other cloud backup solutions, the user is in full control of the backup storage and responsible for purchasing one of the cloud storage products (such as Google Cloud Storage), which offer great durability and availability for the data.
Kopia in action
Using kopia command line tool:
Kopia UI - experimental user interface
Getting Started
See Documentation for more information.
Building Kopia
See Build Infrastructure for more information on building Kopia and working with the source code.
Licensing
Kopia is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.
Contribution Guidelines
Kopia is open source and contributions are welcome. For more information on how to contribute see the Contribution Guidelines.
Reporting Security Issues
If you find a security issue you'd like to disclose privately, please contact kopia-pmc@googlegroups.com or via direct message to maintainers on Slack.
Disclaimer
Kopia is a personal project and is not affiliated with, supported or endorsed by Google.
Cryptography Notice
This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.
The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with symmetric algorithms. The form and manner of this distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.
Documentation
¶
Overview ¶
Command-line tool for creating and accessing backups.
Usage:
$ kopia [<flags>] <subcommand> [<args> ...]
Use 'kopia help' to see more details.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cli implements command-line commands for the Kopia.
|
Package cli implements command-line commands for the Kopia. |
|
Package fs defines virtual filesystem abstractions.
|
Package fs defines virtual filesystem abstractions. |
|
cachefs
Package cachefs implements a wrapper that caches filesystem actions.
|
Package cachefs implements a wrapper that caches filesystem actions. |
|
ignorefs
Package ignorefs implements a wrapper that hides ignored files listed in '.kopiaignore' and in policies attached to directories.
|
Package ignorefs implements a wrapper that hides ignored files listed in '.kopiaignore' and in policies attached to directories. |
|
localfs
Package localfs implements virtual filesystem abstraction for a local filesystem.
|
Package localfs implements virtual filesystem abstraction for a local filesystem. |
|
loggingfs
Package loggingfs implements a wrapper that logs all filesystem actions.
|
Package loggingfs implements a wrapper that logs all filesystem actions. |
|
virtualfs
Package virtualfs implements an in-memory abstraction of fs.Directory and fs.StreamingFile.
|
Package virtualfs implements an in-memory abstraction of fs.Directory and fs.StreamingFile. |
|
internal
|
|
|
acl
Package acl provides management of ACLs that define permissions granted to repository users.
|
Package acl provides management of ACLs that define permissions granted to repository users. |
|
apiclient
Package apiclient implements a client for connecting to Kopia HTTP API server.
|
Package apiclient implements a client for connecting to Kopia HTTP API server. |
|
atomicfile
Package atomicfile provides wrappers for atomically writing files in a manner compatible with long filenames.
|
Package atomicfile provides wrappers for atomically writing files in a manner compatible with long filenames. |
|
auth
Package auth provides authentication and authorization constructs.
|
Package auth provides authentication and authorization constructs. |
|
blobtesting
Package blobtesting is used for testing BLOB storage implementations.
|
Package blobtesting is used for testing BLOB storage implementations. |
|
cache
Package cache implements durable on-disk cache with LRU expiration.
|
Package cache implements durable on-disk cache with LRU expiration. |
|
clock
Package clock provides indirection for accessing current time.
|
Package clock provides indirection for accessing current time. |
|
completeset
Package completeset manages complete set of blob metadata.
|
Package completeset manages complete set of blob metadata. |
|
connection
Package connection manages (abstract) connections with retrying and reconnection.
|
Package connection manages (abstract) connections with retrying and reconnection. |
|
ctxutil
Package ctxutil implements utilities for manipulating context.
|
Package ctxutil implements utilities for manipulating context. |
|
diff
Package diff implements helpers for comparing two filesystems.
|
Package diff implements helpers for comparing two filesystems. |
|
editor
Package editor encapsulates working with external text editor.
|
Package editor encapsulates working with external text editor. |
|
epoch
Package epoch manages repository epochs.
|
Package epoch manages repository epochs. |
|
faketime
Package faketime fakes time for tests
|
Package faketime fakes time for tests |
|
fshasher
Package fshasher computes a fingerprint for an FS tree for testing purposes
|
Package fshasher computes a fingerprint for an FS tree for testing purposes |
|
fusemount
Package fusemount implements FUSE filesystem nodes for mounting contents of filesystem stored in repository.
|
Package fusemount implements FUSE filesystem nodes for mounting contents of filesystem stored in repository. |
|
gather
Package gather implements data structures storing binary data organized in a series of byte slices of fixed size that only gathered together by the user.
|
Package gather implements data structures storing binary data organized in a series of byte slices of fixed size that only gathered together by the user. |
|
hmac
Package hmac contains utilities for dealing with HMAC checksums.
|
Package hmac contains utilities for dealing with HMAC checksums. |
|
iocopy
Package iocopy is a wrapper around io.Copy() that recycles shared buffers.
|
Package iocopy is a wrapper around io.Copy() that recycles shared buffers. |
|
listcache
Package listcache defines a blob.Storage wrapper that caches results of list calls for short duration of time.
|
Package listcache defines a blob.Storage wrapper that caches results of list calls for short duration of time. |
|
logfile
Package logfile manages log files.
|
Package logfile manages log files. |
|
memtrack
Package memtrack implements utility to log memory usage.
|
Package memtrack implements utility to log memory usage. |
|
mockfs
Package mockfs implements in-memory filesystem for testing.
|
Package mockfs implements in-memory filesystem for testing. |
|
mount
Package mount manages creating operating system mount points for directory snapshots.
|
Package mount manages creating operating system mount points for directory snapshots. |
|
ospath
Package ospath provides discovery of OS-dependent paths.
|
Package ospath provides discovery of OS-dependent paths. |
|
ownwrites
Package ownwrites defines a blob.Storage wrapper that ensures that recently-written blobs show up in ListBlob() results, if the underlying provider is eventually consistent when it comes to list results.
|
Package ownwrites defines a blob.Storage wrapper that ensures that recently-written blobs show up in ListBlob() results, if the underlying provider is eventually consistent when it comes to list results. |
|
parallelwork
Package parallelwork implements pallel work queue with fixed number of workers that concurrently process and add work items to the queue.
|
Package parallelwork implements pallel work queue with fixed number of workers that concurrently process and add work items to the queue. |
|
passwordpersist
Package passwordpersist manages password persistence.
|
Package passwordpersist manages password persistence. |
|
providervalidation
Package providervalidation implements validation to ensure the blob storage is compatible with Kopia requirements.
|
Package providervalidation implements validation to ensure the blob storage is compatible with Kopia requirements. |
|
remoterepoapi
Package remoterepoapi contains requests and responses for remote repository API.
|
Package remoterepoapi contains requests and responses for remote repository API. |
|
repotesting
Package repotesting contains test utilities for working with repositories.
|
Package repotesting contains test utilities for working with repositories. |
|
retry
Package retry implements exponential retry policy.
|
Package retry implements exponential retry policy. |
|
scrubber
Package scrubber contains helpers that remove sensitive information from Go structs before it's presented to users.
|
Package scrubber contains helpers that remove sensitive information from Go structs before it's presented to users. |
|
server
Package server implements Kopia API server handlers.
|
Package server implements Kopia API server handlers. |
|
serverapi
Package serverapi contains GO types corresponding to Kopia server API.
|
Package serverapi contains GO types corresponding to Kopia server API. |
|
stats
Package stats provides helpers for simple stats
|
Package stats provides helpers for simple stats |
|
testlogging
Package testlogging implements logger that writes to testing.T log.
|
Package testlogging implements logger that writes to testing.T log. |
|
testutil
Package testutil contains test utilities.
|
Package testutil contains test utilities. |
|
throttle
Package throttle implements helpers for throttling uploads and downloads.
|
Package throttle implements helpers for throttling uploads and downloads. |
|
timetrack
Package timetrack tracks the progress and estimates completion of a task.
|
Package timetrack tracks the progress and estimates completion of a task. |
|
tlsutil
Package tlsutil contains TLS utilities.
|
Package tlsutil contains TLS utilities. |
|
uitask
Package uitask provided management of in-process long-running tasks that are exposed to the UI.
|
Package uitask provided management of in-process long-running tasks that are exposed to the UI. |
|
units
Package units contains helpers to convert sizes to humand-readable strings.
|
Package units contains helpers to convert sizes to humand-readable strings. |
|
user
Package user provides management of user accounts.
|
Package user provides management of user accounts. |
|
wcmatch
Package wcmatch implements wildcard matching files using .gitignore syntax.
|
Package wcmatch implements wildcard matching files using .gitignore syntax. |
|
webdavmount
Package webdavmount implements webdav filesystem for serving snapshots.
|
Package webdavmount implements webdav filesystem for serving snapshots. |
|
zaplogutil
Package zaplogutil provides reusable utilities for working with ZAP logger.
|
Package zaplogutil provides reusable utilities for working with ZAP logger. |
|
Package repo implements content-addressable Repository on top of BLOB storage.
|
Package repo implements content-addressable Repository on top of BLOB storage. |
|
blob
Package blob implements simple storage of immutable, unstructured binary large objects (BLOBs).
|
Package blob implements simple storage of immutable, unstructured binary large objects (BLOBs). |
|
blob/azure
Package azure implements Azure Blob Storage.
|
Package azure implements Azure Blob Storage. |
|
blob/b2
Package b2 implements Storage based on an Backblaze B2 bucket.
|
Package b2 implements Storage based on an Backblaze B2 bucket. |
|
blob/filesystem
Package filesystem implements filesystem-based Storage.
|
Package filesystem implements filesystem-based Storage. |
|
blob/gcs
Package gcs implements Storage based on Google Cloud Storage bucket.
|
Package gcs implements Storage based on Google Cloud Storage bucket. |
|
blob/logging
Package logging implements wrapper around Storage that logs all activity.
|
Package logging implements wrapper around Storage that logs all activity. |
|
blob/rclone
Package rclone implements blob storage provider proxied by rclone (http://rclone.org)
|
Package rclone implements blob storage provider proxied by rclone (http://rclone.org) |
|
blob/readonly
Package readonly implements wrapper around readonlyStorage that prevents all mutations.
|
Package readonly implements wrapper around readonlyStorage that prevents all mutations. |
|
blob/retrying
Package retrying implements wrapper around blob.Storage that adds retry loop around all operations in case they return unexpected errors.
|
Package retrying implements wrapper around blob.Storage that adds retry loop around all operations in case they return unexpected errors. |
|
blob/s3
Package s3 implements Storage based on an S3 bucket.
|
Package s3 implements Storage based on an S3 bucket. |
|
blob/sftp
Package sftp implements blob storage provided for SFTP/SSH.
|
Package sftp implements blob storage provided for SFTP/SSH. |
|
blob/sharded
Package sharded implements common support for sharded blob providers, such as filesystem or webdav.
|
Package sharded implements common support for sharded blob providers, such as filesystem or webdav. |
|
blob/webdav
Package webdav implements WebDAV-based Storage.
|
Package webdav implements WebDAV-based Storage. |
|
compression
Package compression manages compression algorithm implementations.
|
Package compression manages compression algorithm implementations. |
|
content
Package content implements repository support for content-addressable storage.
|
Package content implements repository support for content-addressable storage. |
|
encryption
Package encryption manages content encryption algorithms.
|
Package encryption manages content encryption algorithms. |
|
hashing
Package hashing encapsulates all keyed hashing algorithms.
|
Package hashing encapsulates all keyed hashing algorithms. |
|
logging
Package logging provides loggers for Kopia.
|
Package logging provides loggers for Kopia. |
|
maintenance
Package maintenance manages automatic repository maintenance.
|
Package maintenance manages automatic repository maintenance. |
|
manifest
Package manifest implements support for managing JSON-based manifests in repository.
|
Package manifest implements support for managing JSON-based manifests in repository. |
|
object
Package object implements repository support for content-addressable objects of arbitrary size.
|
Package object implements repository support for content-addressable objects of arbitrary size. |
|
splitter
Package splitter manages splitting of object data into chunks.
|
Package splitter manages splitting of object data into chunks. |
|
site
|
|
|
Package snapshot manages metadata about snapshots stored in repository.
|
Package snapshot manages metadata about snapshots stored in repository. |
|
policy
Package policy implements management of snapshot policies.
|
Package policy implements management of snapshot policies. |
|
restore
Package restore manages restoring filesystem snapshots.
|
Package restore manages restoring filesystem snapshots. |
|
snapshotfs
Package snapshotfs implements virtual filesystem on top of snapshots in repo.Repository.
|
Package snapshotfs implements virtual filesystem on top of snapshots in repo.Repository. |
|
snapshotgc
Package snapshotgc implements garbage collection of contents that are no longer referenced through snapshots.
|
Package snapshotgc implements garbage collection of contents that are no longer referenced through snapshots. |
|
snapshotmaintenance
Package snapshotmaintenance provides helpers to run snapshot GC and low-level repository snapshotmaintenance.
|
Package snapshotmaintenance provides helpers to run snapshot GC and low-level repository snapshotmaintenance. |
|
tests
|
|
|
clitestutil
Package clitestutil contains utilities for
|
Package clitestutil contains utilities for |
|
repository_stress_test/repomodel
Package repomodel provides simplified model of repository operaton.
|
Package repomodel provides simplified model of repository operaton. |
|
robustness
Package robustness contains tests that that validate data stability over time.
|
Package robustness contains tests that that validate data stability over time. |
|
robustness/checker
Package checker defines the framework for creating and restoring snapshots with a data integrity check
|
Package checker defines the framework for creating and restoring snapshots with a data integrity check |
|
robustness/engine
Package engine provides the framework for a snapshot repository testing engine
|
Package engine provides the framework for a snapshot repository testing engine |
|
robustness/fiofilewriter
Package fiofilewriter provides a FileWriter based on FIO.
|
Package fiofilewriter provides a FileWriter based on FIO. |
|
robustness/multiclient_test/framework
Package framework contains tools to enable multiple clients to connect to a central repository server and run robustness tests concurrently.
|
Package framework contains tools to enable multiple clients to connect to a central repository server and run robustness tests concurrently. |
|
robustness/pathlock
Package pathlock defines a PathLocker interface and an implementation that will synchronize based on filepath.
|
Package pathlock defines a PathLocker interface and an implementation that will synchronize based on filepath. |
|
robustness/snapmeta
Package snapmeta provides Kopia implementations of Persister and Snapshotter.
|
Package snapmeta provides Kopia implementations of Persister and Snapshotter. |
|
testdirtree
Package testdirtree provides utilities for creating test directory trees for testing.
|
Package testdirtree provides utilities for creating test directory trees for testing. |
|
testenv
Package testenv contains Environment for use in testing.
|
Package testenv contains Environment for use in testing. |
|
testingaction
Command testingaction implements a action that is used in various tests.
|
Command testingaction implements a action that is used in various tests. |
|
tools/fio
Package fio wraps calls to the fio tool.
|
Package fio wraps calls to the fio tool. |
|
tools/fswalker
Package fswalker provides the checker.Comparer interface using FSWalker walker and reporter.
|
Package fswalker provides the checker.Comparer interface using FSWalker walker and reporter. |
|
tools/fswalker/protofile
Package protofile contains helper functions common across multiple fswalker tool wrappers
|
Package protofile contains helper functions common across multiple fswalker tool wrappers |
|
tools/fswalker/reporter
Package reporter wraps calls to the the fswalker Reporter
|
Package reporter wraps calls to the the fswalker Reporter |
|
tools/fswalker/walker
Package walker wraps calls to the the fswalker Walker
|
Package walker wraps calls to the the fswalker Walker |
|
tools/kopiaclient
Package kopiaclient provides a client to interact with a Kopia repo.
|
Package kopiaclient provides a client to interact with a Kopia repo. |
|
tools/kopiarunner
Package kopiarunner wraps the execution of the kopia binary.
|
Package kopiarunner wraps the execution of the kopia binary. |
|
tools
|
|
|
gettool
Package gettool combines and replaces curl, tar and gunzip, sha256sum and a bunch of Makefile scripts to quickly download, verify and install OS-specific version of tools (typically from GitHub) in a platform-agnostic manner without external tooling.
|
Package gettool combines and replaces curl, tar and gunzip, sha256sum and a bunch of Makefile scripts to quickly download, verify and install OS-specific version of tools (typically from GitHub) in a platform-agnostic manner without external tooling. |
|
gettool/autodownload
Package autodownload combines and replaces curl, tar and gunzip and sha256sum and allows downloading, verifying and extracting the archive (zip, tar, tar.gz) to a local directory without using external tools.
|
Package autodownload combines and replaces curl, tar and gunzip and sha256sum and allows downloading, verifying and extracting the archive (zip, tar, tar.gz) to a local directory without using external tools. |
Click to show internal directories.
Click to hide internal directories.