README

Camlistore is your personal storage system for life.

It's a way to store, sync, share, model and back up content.

It stands for "Content-Addressable Multi-Layer Indexed Storage", for
lack of a better name.  For more, see:

     https://camlistore.org/
     https://camlistore.org/doc/

Other useful files:

     BUILDING       how to compile it ("go run make.go")
     CONTRIBUTING   how to do development and contribute

Mailing lists:

     http://camlistore.org/lists

Bugs and contributing:

     https://github.com/camlistore/camlistore/issues
     https://camlistore.org/doc/contributing
Expand ▾ Collapse ▴

Directories

Path Synopsis
app
hello
The hello application serves as an example on how to make stand-alone server applications, interacting with a Camlistore server.
The hello application serves as an example on how to make stand-alone server applications, interacting with a Camlistore server.
publisher
The publisher application serves and renders items published by Camlistore.
The publisher application serves and renders items published by Camlistore.
publisher/js
This package contains source code for gopherjs, to generate javascript code that is included in the publisher web UI.
This package contains source code for gopherjs, to generate javascript code that is included in the publisher web UI.
cmd
camdeploy
The camdeploy program deploys Camlistore on cloud computing platforms such as Google Compute Engine or Amazon EC2.
The camdeploy program deploys Camlistore on cloud computing platforms such as Google Compute Engine or Amazon EC2.
camget
The camget tool fetches blobs, files, and directories.
The camget tool fetches blobs, files, and directories.
cammount
The cammount tool mounts a root directory blob onto the given mountpoint.
The cammount tool mounts a root directory blob onto the given mountpoint.
camput
The camput tool mainly pushes blobs, files, and directories.
The camput tool mainly pushes blobs, files, and directories.
camtool
The camtool binary is a collection of commands to help with the use of a camlistore server.
The camtool binary is a collection of commands to help with the use of a camlistore server.
dev
update_closure_compiler downloads a new version of the closure compiler if the one in tmp/closure-compiler doesn't exist or is older than the requested version.
update_closure_compiler downloads a new version of the closure compiler if the one in tmp/closure-compiler doesn't exist or is older than the requested version.
devcam
The devcam tool is a collection of wrappers around the camlistore programs (camistored, camput, camtool...) which take care of setup and configuration, so they can be used by developers to ease hacking on camlistore.
The devcam tool is a collection of wrappers around the camlistore programs (camistored, camput, camtool...) which take care of setup and configuration, so they can be used by developers to ease hacking on camlistore.
envvardoc
Program envvardoc will verify all referenced environment variables in go source are properly documented.
Program envvardoc will verify all referenced environment variables in go source are properly documented.
Command monthly builds the tarballs and zip archives for all the monthly released Camlistore downloads.
Command monthly builds the tarballs and zip archives for all the monthly released Camlistore downloads.
buildbot/builder
The buildbot is Camlistore's continuous builder.
The buildbot is Camlistore's continuous builder.
buildbot/master
The buildbot is Camlistore's continuous builder.
The buildbot is Camlistore's continuous builder.
docker
Command dock builds Camlistore's various Docker images.
Command dock builds Camlistore's various Docker images.
pkg
app
Package app provides helpers for server applications interacting with Camlistore.
Package app provides helpers for server applications interacting with Camlistore.
auth
Package auth implements Camlistore authentication.
Package auth implements Camlistore authentication.
blob
Package blob defines types to refer to and retrieve low-level Camlistore blobs.
Package blob defines types to refer to and retrieve low-level Camlistore blobs.
blobserver
Package blobserver defines how raw blobs are stored and accessed.
Package blobserver defines how raw blobs are stored and accessed.
blobserver/archiver
Package archiver zips lots of little blobs into bigger zip files and stores them somewhere.
Package archiver zips lots of little blobs into bigger zip files and stores them somewhere.
blobserver/cond
Package cond registers the "cond" conditional blobserver storage type to select routing of get/put operations on blobs to other storage targets as a function of their content.
Package cond registers the "cond" conditional blobserver storage type to select routing of get/put operations on blobs to other storage targets as a function of their content.
blobserver/dir
Package dir implements the blobserver Storage interface for a directory, detecting whether the directory is file-per-blob (localdisk) or diskpacked.
Package dir implements the blobserver Storage interface for a directory, detecting whether the directory is file-per-blob (localdisk) or diskpacked.
blobserver/diskpacked
Package diskpacked registers the "diskpacked" blobserver storage type, storing blobs packed together into monolithic data files with an index listing the sizes and offsets of the little blobs within the large files.
Package diskpacked registers the "diskpacked" blobserver storage type, storing blobs packed together into monolithic data files with an index listing the sizes and offsets of the little blobs within the large files.
blobserver/encrypt
Package encrypt registers the "encrypt" blobserver storage type which stores all blobs and metadata with AES encryption into other wrapped storage targets (e.g.
Package encrypt registers the "encrypt" blobserver storage type which stores all blobs and metadata with AES encryption into other wrapped storage targets (e.g.
blobserver/gethandler
Package gethandler implements the HTTP handler for fetching blobs.
Package gethandler implements the HTTP handler for fetching blobs.
blobserver/google/cloudstorage
Package cloudstorage registers the "googlecloudstorage" blob storage type, storing blobs on Google Cloud Storage (not Google Drive).
Package cloudstorage registers the "googlecloudstorage" blob storage type, storing blobs on Google Cloud Storage (not Google Drive).
blobserver/google/drive
Package drive registers the "googledrive" blobserver storage type, storing blobs in a Google Drive folder.
Package drive registers the "googledrive" blobserver storage type, storing blobs in a Google Drive folder.
blobserver/google/drive/service
DriveService translates blobserver.Storage methods into Google Drive API methods.
DriveService translates blobserver.Storage methods into Google Drive API methods.
blobserver/handlers
Package handlers implements the HTTP interface to the Camlistore blob server.
Package handlers implements the HTTP interface to the Camlistore blob server.
blobserver/local
Package local implements functionality common to both the "localdisk" and "diskpacked" storage mechanisms.
Package local implements functionality common to both the "localdisk" and "diskpacked" storage mechanisms.
blobserver/localdisk
Package localdisk registers the "filesystem" blobserver storage type, storing blobs in a forest of sharded directories at the specified root.
Package localdisk registers the "filesystem" blobserver storage type, storing blobs in a forest of sharded directories at the specified root.
blobserver/memory
Package memory registers the "memory" blobserver storage type, storing blobs in an in-memory map.
Package memory registers the "memory" blobserver storage type, storing blobs in an in-memory map.
blobserver/mongo
Package mongo registers the "mongo" blobserver storage type, storing blobs using MongoDB.
Package mongo registers the "mongo" blobserver storage type, storing blobs using MongoDB.
blobserver/namespace
Package namespace implements the "namespace" blobserver storage type.
Package namespace implements the "namespace" blobserver storage type.
blobserver/protocol
Package protocol contains types for Camlistore protocol types.
Package protocol contains types for Camlistore protocol types.
blobserver/proxycache
Package proxycache registers the "proxycache" blobserver storage type, which uses a provided blobserver as a cache for a second origin blobserver.
Package proxycache registers the "proxycache" blobserver storage type, which uses a provided blobserver as a cache for a second origin blobserver.
blobserver/remote
Package remote registers the "remote" blobserver storage type, storing and fetching blobs from a remote Camlistore server, speaking the HTTP protocol.
Package remote registers the "remote" blobserver storage type, storing and fetching blobs from a remote Camlistore server, speaking the HTTP protocol.
blobserver/replica
Package replica registers the "replica" blobserver storage type, providing synchronous replication to one more backends.
Package replica registers the "replica" blobserver storage type, providing synchronous replication to one more backends.
blobserver/s3
Package s3 registers the "s3" blobserver storage type, storing blobs in an Amazon Web Services' S3 storage bucket.
Package s3 registers the "s3" blobserver storage type, storing blobs in an Amazon Web Services' S3 storage bucket.
blobserver/shard
Package shard registers the "shard" blobserver storage type, predictably spraying out blobs out over the provided backends based on their blobref.
Package shard registers the "shard" blobserver storage type, predictably spraying out blobs out over the provided backends based on their blobref.
blobserver/stats
Package stats contains an in-memory StatReceiver that only stores sizes of received blobs but not their contents.
Package stats contains an in-memory StatReceiver that only stores sizes of received blobs but not their contents.
blobserver/storagetest
Package storagetest tests blobserver.Storage implementations
Package storagetest tests blobserver.Storage implementations
buildinfo
Package buildinfo provides information about the current build.
Package buildinfo provides information about the current build.
cacher
Package cacher provides various blobref fetching caching mechanisms.
Package cacher provides various blobref fetching caching mechanisms.
camerrors
Package camerrors define specific errors that are used to decide on how to deal with some failure cases.
Package camerrors define specific errors that are used to decide on how to deal with some failure cases.
camlegal
Package camlegal registers the Camlistore License
Package camlegal registers the Camlistore License
client
Package client implements a Camlistore client.
Package client implements a Camlistore client.
client/android
Package android contains code specific to running the Camlistore client code as a child process on Android.
Package android contains code specific to running the Camlistore client code as a child process on Android.
cmdmain
Package cmdmain contains the shared implementation for camget, camput, camtool, and other Camlistore command-line tools.
Package cmdmain contains the shared implementation for camget, camput, camtool, and other Camlistore command-line tools.
constants
Package constants contains Camlistore constants.
Package constants contains Camlistore constants.
conv
Package conv contains utilities for parsing values delimited by spaces.
Package conv contains utilities for parsing values delimited by spaces.
deploy/gce
Package gce provides tools to deploy Camlistore on Google Compute Engine.
Package gce provides tools to deploy Camlistore on Google Compute Engine.
env
Package env detects what sort of environment Camlistore is running in.
Package env detects what sort of environment Camlistore is running in.
fileembed
Package fileembed provides access to static data resources (images, HTML, css, etc) embedded into the binary with genfileembed.
Package fileembed provides access to static data resources (images, HTML, css, etc) embedded into the binary with genfileembed.
fileembed/genfileembed
The genfileembed command embeds resources into Go files, to eliminate run-time dependencies on files on the filesystem.
The genfileembed command embeds resources into Go files, to eliminate run-time dependencies on files on the filesystem.
fs
Package fs implements a FUSE filesystem for Camlistore and is used by the cammount binary.
Package fs implements a FUSE filesystem for Camlistore and is used by the cammount binary.
gc
Package gc defines a generic garbage collector.
Package gc defines a generic garbage collector.
geocode
Package geocode handles mapping user-entered locations into lat/long polygons.
Package geocode handles mapping user-entered locations into lat/long polygons.
googlestorage
Package googlestorage is simple Google Cloud Storage client.
Package googlestorage is simple Google Cloud Storage client.
hashutil
Package hashutil contains misc hashing functions lacking homes elsewhere.
Package hashutil contains misc hashing functions lacking homes elsewhere.
httputil
Package httputil contains a bunch of HTTP utility code, some generic, and some Camlistore-specific.
Package httputil contains a bunch of HTTP utility code, some generic, and some Camlistore-specific.
images/fastjpeg
Package fastjpeg uses djpeg(1), from the Independent JPEG Group's (www.ijg.org) jpeg package, to quickly down-sample images on load.
Package fastjpeg uses djpeg(1), from the Independent JPEG Group's (www.ijg.org) jpeg package, to quickly down-sample images on load.
images/resize
Package resize resizes images.
Package resize resizes images.
importer
Package importer imports content from third-party websites.
Package importer imports content from third-party websites.
importer/allimporters
Package allimporters registers all the importer implementations.
Package allimporters registers all the importer implementations.
importer/dummy
Package dummy is an example importer for development purposes.
Package dummy is an example importer for development purposes.
importer/feed
Package feed implements an importer for RSS, Atom, and RDF feeds.
Package feed implements an importer for RSS, Atom, and RDF feeds.
importer/feed/atom
Package atom defines XML data structures for an Atom feed.
Package atom defines XML data structures for an Atom feed.
importer/feed/rdf
Package rdf defines XML data structures for an RDF feed.
Package rdf defines XML data structures for an RDF feed.
importer/feed/rss
Package rss defines XML data structures for an RSS feed.
Package rss defines XML data structures for an RSS feed.
importer/flickr
Package flickr implements an importer for flickr.com accounts.
Package flickr implements an importer for flickr.com accounts.
importer/foursquare
Package foursquare implements an importer for foursquare.com accounts.
Package foursquare implements an importer for foursquare.com accounts.
importer/picasa
Package picasa implements an importer for picasa.com accounts.
Package picasa implements an importer for picasa.com accounts.
importer/pinboard
Package pinboard imports pinboard.in posts.
Package pinboard imports pinboard.in posts.
importer/test
Package test provides common functionality for importer tests.
Package test provides common functionality for importer tests.
importer/twitter
Package twitter implements a twitter.com importer.
Package twitter implements a twitter.com importer.
index
Package index provides a generic indexing system on top of the abstract Storage interface.
Package index provides a generic indexing system on top of the abstract Storage interface.
index/indextest
Package indextest contains the unit tests for the indexer so they can be re-used for each specific implementation of the index Storage interface.
Package indextest contains the unit tests for the indexer so they can be re-used for each specific implementation of the index Storage interface.
index/sqlindex
Package sqlindex implements the sorted.KeyValue interface using an *sql.DB.
Package sqlindex implements the sorted.KeyValue interface using an *sql.DB.
jsonsign
Package jsonsign implements Camlistore's cryptographic signing and verification of JSON blobs.
Package jsonsign implements Camlistore's cryptographic signing and verification of JSON blobs.
jsonsign/signhandler
Package signhandler implements the HTTP interface to signing and verifying Camlistore JSON blobs.
Package signhandler implements the HTTP interface to signing and verifying Camlistore JSON blobs.
kvutil
Package kvutil contains helpers related to github.com/cznic/kv.
Package kvutil contains helpers related to github.com/cznic/kv.
lru
Package lru implements an LRU cache.
Package lru implements an LRU cache.
magic
Package magic implements MIME type sniffing of data based on the well-known "magic" number prefixes in the file.
Package magic implements MIME type sniffing of data based on the well-known "magic" number prefixes in the file.
media
Package media provides means for querying information about audio and video data.
Package media provides means for querying information about audio and video data.
misc/amazon/s3
Package s3 implements a generic Amazon S3 client, not specific to Camlistore.
Package s3 implements a generic Amazon S3 client, not specific to Camlistore.
misc/closure
Package closure provides tools to help with the use of the closure library.
Package closure provides tools to help with the use of the closure library.
misc/closure/genclosuredeps
The genclosuredeps command, similarly to the closure depswriter.py tool, outputs to os.Stdout for each .js file, which namespaces it provides, and the namespaces it requires, hence helping the closure library to resolve dependencies between those files.
The genclosuredeps command, similarly to the closure depswriter.py tool, outputs to os.Stdout for each .js file, which namespaces it provides, and the namespaces it requires, hence helping the closure library to resolve dependencies between those files.
misc/closure/jstest
Package jstest uses the Go testing package to test JavaScript code using Node and Mocha.
Package jstest uses the Go testing package to test JavaScript code using Node and Mocha.
misc/gpgagent
Package gpgagent interacts with the local GPG Agent.
Package gpgagent interacts with the local GPG Agent.
misc/pinentry
Package pinentry interfaces with the pinentry(1) command to securely prompt the user for a password using whichever user interface the user is currently using.
Package pinentry interfaces with the pinentry(1) command to securely prompt the user for a password using whichever user interface the user is currently using.
netutil
Package netutil identifies the system userid responsible for localhost TCP connections.
Package netutil identifies the system userid responsible for localhost TCP connections.
osutil
Package osutil provides operating system-specific path information, and other utility functions.
Package osutil provides operating system-specific path information, and other utility functions.
osutil/gce
Package gce configures hooks for running Camlistore for Google Compute Engine.
Package gce configures hooks for running Camlistore for Google Compute Engine.
publish
Package publish exposes the types and functions that can be used from a Go template, for publishing.
Package publish exposes the types and functions that can be used from a Go template, for publishing.
readerutil
Package readerutil provides and operates on io.Readers.
Package readerutil provides and operates on io.Readers.
rollsum
Package rollsum implements rolling checksums similar to apenwarr's bup, which is similar to librsync.
Package rollsum implements rolling checksums similar to apenwarr's bup, which is similar to librsync.
schema
Package schema manipulates Camlistore schema blobs.
Package schema manipulates Camlistore schema blobs.
schema/nodeattr
Package nodeattr contains constants for permanode attribute names.
Package nodeattr contains constants for permanode attribute names.
search
Package search describes and answers Camlistore search queries.
Package search describes and answers Camlistore search queries.
server
Package server implements the server HTTP interface for the UI, publishing, setup, status, sync, thubnailing, etc.
Package server implements the server HTTP interface for the UI, publishing, setup, status, sync, thubnailing, etc.
server/app
Package app helps with configuring and starting server applications from Camlistore.
Package app helps with configuring and starting server applications from Camlistore.
serverinit
Package serverinit is responsible for mapping from a Camlistore configuration file and instantiating HTTP Handlers for all the necessary endpoints.
Package serverinit is responsible for mapping from a Camlistore configuration file and instantiating HTTP Handlers for all the necessary endpoints.
sorted
Package sorted provides a KeyValue interface and constructor registry.
Package sorted provides a KeyValue interface and constructor registry.
sorted/buffer
Package buffer provides a sorted.KeyValue implementation that buffers one KeyValue implementation in front of an another.
Package buffer provides a sorted.KeyValue implementation that buffers one KeyValue implementation in front of an another.
sorted/kvfile
Package kvfile provides an implementation of sorted.KeyValue on top of a single mutable database file on disk using github.com/cznic/kv.
Package kvfile provides an implementation of sorted.KeyValue on top of a single mutable database file on disk using github.com/cznic/kv.
sorted/kvtest
Package kvtest tests sorted.KeyValue implementations.
Package kvtest tests sorted.KeyValue implementations.
sorted/leveldb
Package leveldb provides an implementation of sorted.KeyValue on top of a single mutable database file on disk using github.com/syndtr/goleveldb.
Package leveldb provides an implementation of sorted.KeyValue on top of a single mutable database file on disk using github.com/syndtr/goleveldb.
sorted/mongo
Package mongo provides an implementation of sorted.KeyValue using MongoDB.
Package mongo provides an implementation of sorted.KeyValue using MongoDB.
sorted/mysql
Package mysql provides an implementation of sorted.KeyValue on top of MySQL.
Package mysql provides an implementation of sorted.KeyValue on top of MySQL.
sorted/postgres
Package postgres provides an implementation of sorted.KeyValue on top of PostgreSQL.
Package postgres provides an implementation of sorted.KeyValue on top of PostgreSQL.
sorted/sqlite
Package sqlite provides an implementation of sorted.KeyValue using an SQLite database file.
Package sqlite provides an implementation of sorted.KeyValue using an SQLite database file.
sorted/sqlkv
Package sqlkv implements the sorted.KeyValue interface using an *sql.DB.
Package sqlkv implements the sorted.KeyValue interface using an *sql.DB.
test
Package test provides common Camlistore test objects.
Package test provides common Camlistore test objects.
test/asserts
Package asserts provides a bad implementation of test predicate helpers.
Package asserts provides a bad implementation of test predicate helpers.
test/dockertest
Package dockertest contains helper functions for setting up and tearing down docker containers to aid in testing.
Package dockertest contains helper functions for setting up and tearing down docker containers to aid in testing.
types/camtypes
Package camtypes is like the types package, but higher-level and contains Camlistore-specific types.
Package camtypes is like the types package, but higher-level and contains Camlistore-specific types.
types/clientconfig
Package clientconfig provides types related to the client configuration file.
Package clientconfig provides types related to the client configuration file.
types/serverconfig
Package serverconfig provides types related to the server configuration file.
Package serverconfig provides types related to the server configuration file.
video/thumbnail
Package thumbnail generates image thumbnails from videos.
Package thumbnail generates image thumbnails from videos.
webserver
Package webserver implements a superset wrapper of http.Server.
Package webserver implements a superset wrapper of http.Server.
server
camlistored
The camlistored binary is the Camlistore server.
The camlistored binary is the Camlistore server.
camlistored/ui
Package ui contains the resources for the Camlistore web UI.
Package ui contains the resources for the Camlistore web UI.
camnetdns
The camnetdns server serves camlistore.net's DNS server and its DNS challenges
The camnetdns server serves camlistore.net's DNS server and its DNS challenges
sigserver
The sigserver is a stand-alone JSON signing and verification server.
The sigserver is a stand-alone JSON signing and verification server.
internal