rqlite is a lightweight, distributed relational database, which uses SQLite as its storage engine. Forming a cluster is very straightforward, it gracefully handles leader elections, and tolerates failures of machines, including the leader. rqlite is available for Linux, OSX, and Microsoft Windows.
rqlite gives you the functionality of a rock solid, fault-tolerant, replicated relational database, but with very easy installation, deployment, and operation. With it you've got a lightweight and reliable distributed relational data store. Think etcd or Consul, but with relational data modelling also available.
You could use rqlite as part of a larger system, as a central store for some critical relational data, without having to run larger, more complex distributed databases.
rqlite uses Raft to achieve consensus across all the instances of the SQLite databases, ensuring that every change made to the system is made to a quorum of SQLite databases, or none at all. You can learn more about the design here.
- Very easy deployment, with no need to separately install SQLite.
- Fully replicated production-grade SQL database.
- Production-grade distributed consensus system.
- An easy-to-use HTTP(S) API, including leader-redirection and bulk-update support. A CLI is also available, as are various client libraries.
- Discovery Service support, allowing clusters to be dynamically created.
- Extensive security and encryption support, including node-to-node encryption.
- Choice of read consistency levels.
- Transaction support.
- Hot backups.
The quickest way to get running on OSX and Linux is to download a pre-built release binary. You can find these binaries on the Github releases page. If you prefer Windows you can download the latest build here. Once installed, you can start a single rqlite node like so:
This single node automatically becomes the leader. You can pass
rqlited to list all configuration options.
Alternatively you can pull the latest release via
docker pull rqlite/rqlite. Docker support is still very early.
Forming a cluster
While not strictly necessary to run rqlite, running multiple nodes means you'll have a fault-tolerant cluster. Start two more nodes, allowing the cluster to tolerate failure of a single node, like so:
rqlited -http-addr localhost:4003 -raft-addr localhost:4004 -join http://localhost:4001 ~/node.2 rqlited -http-addr localhost:4005 -raft-addr localhost:4006 -join http://localhost:4001 ~/node.3
This demonstration shows all 3 nodes running on the same host. In reality you probably wouldn't do this, and then you wouldn't need to select different -http-addr and -raft-addr ports for each rqlite node.
With just these few steps you've now got a fault-tolerant, distributed relational database. For full details on creating and managing real clusters check out this documentation.
There is also a rqlite Discovery Service, allowing nodes to automatically connect and form a cluster. This can be much more convenient, allowing clusters to be dynamically created. Check out the documentation for more details.
Let's insert some records via the rqlite CLI, using standard SQLite commands. Once inserted, these records will be replicated across the cluster, in a durable and fault-tolerant manner. Your 3-node cluster can suffer the failure of a single node without any loss of functionality.
$ rqlite 127.0.0.1:4001> CREATE TABLE foo (id INTEGER NOT NULL PRIMARY KEY, name TEXT) 0 row affected (0.000668 sec) 127.0.0.1:4001> .schema +-----------------------------------------------------------------------------+ | sql | +-----------------------------------------------------------------------------+ | CREATE TABLE foo (id INTEGER NOT NULL PRIMARY KEY, name TEXT) | +-----------------------------------------------------------------------------+ 127.0.0.1:4001> INSERT INTO foo(name) VALUES("fiona") 1 row affected (0.000080 sec) 127.0.0.1:4001> SELECT * FROM foo +----+-------+ | id | name | +----+-------+ | 1 | fiona | +----+-------+
rqlite replicates SQLite for fault-tolerance. It does not replicate it for performance. In fact performance is reduced somewhat due to the network round-trips.
Depending on your machine (particularly its IO performance) and network, individual INSERT performance could be anything from 10 operations per second to more than 200 operations per second. However, by using the bulk API, transactions, or both, throughput will increase significantly, often by 2 orders of magnitude. This speed-up is due to the way SQLite works. So for high throughput, execute as many operations as possible within a single transaction.
By default rqlite uses an in-memory SQLite database to maximise performance. In this mode no actual SQLite file is created and the entire database is stored in memory. If you wish rqlite to use an actual file-based SQLite database, pass
-on-disk to rqlite on start-up.
Does using an in-memory database put my data at risk?
Since the Raft log is the authoritative store for all data, and it is written to disk by each node, an in-memory database can be fully recreated on start-up. Using an in-memory database does not put your data at risk.
- Only SQL statements that are deterministic are safe to use with rqlite, because statements are committed to the Raft log before they are sent to each node. In other words, rqlite performs statement-based replication. For example, the following statement could result in a different SQLite database under each node:
INSERT INTO foo (n) VALUES(random());
- Technically this is not supported, but you can directly read the SQLite under any node at anytime, assuming you run in "on-disk" mode. However there is no guarantee that the SQLite file reflects all the changes that have taken place on the cluster unless you are sure the host node itself has received and applied all changes.
- In case it isn't obvious, rqlite does not replicate any changes made directly to any underlying SQLite file, when run in "on disk" mode. If you change the SQLite file directly, you will cause rqlite to fail. Only modify the database via the HTTP API.
- SQLite dot-commands such as
.tablesare not directly supported by the API, but the rqlite CLI supports some very similar functionality. This is because those commands are features of the
sqlite3command, not SQLite itself.
You can learn how check status and diagnostics here.
Backup and restore
You can learn about securing access, and restricting users' access, to rqlite here.
There is a Google Group dedicated to discussion of rqlite.
How do I pronounce rqlite? For what it's worth I try to pronounce it "ree-qwell-lite". But it seems most people, including me, often pronouce it "R Q lite".
Package rqlite is a lightweight, distributed relational database built on SQLite. rqlite uses Raft to achieve consensus across the cluster of SQLite databases. It ensures that every change made to the database is made to a majority of underlying SQLite files, or none at all.
rqlite gives you the functionality of a rock solid, fault-tolerant, replicated relational database, but with very easy installation, deployment, and operation. With it you've got a lightweight and reliable distributed store for relational data. You could use rqlite as part of a larger system, as a central store for some critical relational data, without having to run a heavier solution like MySQL.
Package auth is a lightweight credential store.
|Package auth is a lightweight credential store.|
Package aws provides functionality for accessing the AWS API.
|Package aws provides functionality for accessing the AWS API.|
Package cluster supports intracluster control messaging.
|Package cluster supports intracluster control messaging.|
Command rqlite is the command-line interface for rqlite.
|Command rqlite is the command-line interface for rqlite.|
Command rqlited is the rqlite server.
|Command rqlited is the rqlite server.|
Package db exposes a lightweight abstraction over the SQLite code.
|Package db exposes a lightweight abstraction over the SQLite code.|
Package disco controls interaction with the rqlite Discovery service
|Package disco controls interaction with the rqlite Discovery service|
Package http provides the HTTP server for accessing the distributed database.
|Package http provides the HTTP server for accessing the distributed database.|
Package store provides a distributed SQLite instance.
|Package store provides a distributed SQLite instance.|
Package system supports running end-to-end type testing of rqlite
|Package system supports running end-to-end type testing of rqlite|
Package tcp provides various TCP-related utilities.
|Package tcp provides various TCP-related utilities.|