restserver

README

Rest Server

Rest Server is a high performance HTTP server that implements restic's REST backend API. It provides secure and efficient way to backup data remotely, using restic backup client via the rest: URL.

Requirements

Rest Server requires Go 1.7 or higher to build. The only tested compiler is the official Go compiler. Building server with gccgo may work, but is not supported.

The required version of restic backup client to use with Rest Server is v0.7.1 or higher.

Installation

From source

Build

make

or

go run build.go

If all goes well, you'll find the binary in the current directory.

Alternatively, you can compile and install it in your $GOBIN with a standard go install ./cmd/rest-server. But, beware, you won't have version info built into binary when compiled that way!

Install

make install

Installs the binary as /usr/local/bin/rest-server.

Alternatively, you can install it manually anywhere you want. It's a single binary, there are no dependencies.

Docker

Build image

make docker_build

Pull image

docker pull restic/rest-server

Usage

To learn how to use restic backup client with REST backend, please consult restic manual.

rest-server --help

Run a REST server for use with restic

Usage:
  rest-server [flags]

Flags:
      --append-only         enable append only mode
      --cpu-profile string  write CPU profile to file
      --debug               output debug messages
  -h, --help                help for rest-server
      --listen string       listen address (default ":8000")
      --log string          log HTTP requests in the combined log format
      --no-auth             disable .htpasswd authentication
      --path string         data directory (default "/tmp/restic")
      --private-repos       users can only access their private repo
      --prometheus          enable Prometheus metrics
      --tls                 turn on TLS support
      --tls-cert string     TLS certificate path
      --tls-key string      TLS key path
  -V, --version             show version and quit

By default the server persists backup data in /tmp/restic. To start the server with a custom persistence directory and with authentication disabled:

rest-server --path /user/home/backup --no-auth

To authenticate users (for access to the rest-server), the server supports using a .htpasswd file to specify users. You can create such a file at the root of the persistence directory by executing the following command (note that you need the htpasswd program from Apache's http-tools). In order to append new user to the file, just omit the -c argument. Only bcrypt and SHA encryption methods are supported, so use -B (very secure) or -s (insecure by today's standards) when adding/changing passwords.

htpasswd -B -c .htpasswd username

If you want to disable authentication, you must add the --no-auth flag. If this flag is not specified and the .htpasswd cannot be opened, rest-server will refuse to start.

NOTE: In older versions of rest-server (up to 0.9.7), this flag does not exist and the server disables authentication if .htpasswd is missing or cannot be opened.

By default the server uses HTTP protocol. This is not very secure since with Basic Authentication, username and passwords will travel in cleartext in every request. In order to enable TLS support just add the --tls argument and add a private and public key at the root of your persistence directory. You may also specify private and public keys by --tls-cert and --tls-key.

Signed certificate is required by the restic backend, but if you just want to test the feature you can generate unsigned keys with the following commands:

openssl genrsa -out private_key 2048
openssl req -new -x509 -key private_key -out public_key -days 365

The --append-only mode allows creation of new backups but prevents deletion and modification of existing backups. This can be useful when backing up systems that have a potential of being hacked.

To prevent your users from accessing each others' repositories, you may use the --private-repos flag which grants access only when a subdirectory with the same name as the user is specified in the repository URL. For example, user "foo" using the repository URLs rest:https://foo:pass@host:8000/foo, rest:https://foo:pass@host:8000/foo/ or rest:https://foo:pass@host:8000/foo/bar would be granted access, but the same user using repository URLs rest:https://foo:pass@host:8000/ or rest:https://foo:pass@host:8000/foobar/ would be denied access.

Rest Server uses exactly the same directory structure as local backend, so you should be able to access it both locally and via HTTP, even simultaneously.

Systemd

There's an example systemd service file included with the source, so you can get Rest Server up & running as a proper Systemd service in no time. Before installing, adapt paths and options to your environment.

Docker

By default, image uses authentication. To turn it off, set environment variable DISABLE_AUTHENTICATION to any value.

Persistent data volume is located to /data.

Start server

docker run -p 8000:8000 -v /my/data:/data --name rest_server restic/rest-server

It's suggested to set a container name to more easily manage users (see next section).

You can set environment variable OPTIONS to any extra flags you'd like to pass to rest-server.

Manage users

Add user

docker exec -it rest_server create_user myuser

or

docker exec -it rest_server create_user myuser mypassword

Delete user

docker exec -it rest_server delete_user myuser

Prometheus support and Grafana dashboard

The server can be started with --prometheus to expose Prometheus metrics at /metrics.

This repository contains an example full stack Docker Compose setup with a Grafana dashboard in examples/compose-with-grafana/.

Why use Rest Server?

Compared to the SFTP backend, the REST backend has better performance, especially so if you can skip additional crypto overhead by using plain HTTP transport (restic already properly encrypts all data it sends, so using HTTPS is mostly about authentication).

But, even if you use HTTPS transport, the REST protocol should be faster and more scalable, due to some inefficiencies of the SFTP protocol (everything needs to be transferred in chunks of 32 KiB at most, each packet needs to be acknowledged by the server).

Finally, the Rest Server implementation is really simple and as such could be used on the low-end devices, no problem. Also, in some cases, for example behind corporate firewalls, HTTP/S might be the only protocol allowed. Here too REST backend might be the perfect option for your backup needs.

Contributors

Contributors are welcome, just open a new issue / pull request.

License

The BSD 2-Clause License

Copyright © 2015, Bertil Chapuis
Copyright © 2016, Zlatko Čalušić, Alexander Neumann
Copyright © 2017, The Rest Server Authors
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Documentation

Index

• Constants
• func NewHandler(server Server) *goji.Mux
• type Blob
• type HtpasswdFile
  • func NewHtpasswdFromFile(path string) (*HtpasswdFile, error)
  • func (h *HtpasswdFile) Reload() error
  • func (h *HtpasswdFile) ReloadCheck() error
  • func (h *HtpasswdFile) Validate(user string, password string) bool
• type Server
  • func (s *Server) AuthHandler(f *HtpasswdFile, h http.Handler) http.HandlerFunc
  • func (s *Server) CheckBlob(w http.ResponseWriter, r *http.Request)
  • func (s *Server) CheckConfig(w http.ResponseWriter, r *http.Request)
  • func (s *Server) CreateRepo(w http.ResponseWriter, r *http.Request)
  • func (s *Server) DeleteBlob(w http.ResponseWriter, r *http.Request)
  • func (s *Server) DeleteConfig(w http.ResponseWriter, r *http.Request)
  • func (s *Server) GetBlob(w http.ResponseWriter, r *http.Request)
  • func (s *Server) GetConfig(w http.ResponseWriter, r *http.Request)
  • func (s *Server) ListBlobs(w http.ResponseWriter, r *http.Request)
  • func (s *Server) ListBlobsV1(w http.ResponseWriter, r *http.Request)
  • func (s *Server) ListBlobsV2(w http.ResponseWriter, r *http.Request)
  • func (s *Server) SaveBlob(w http.ResponseWriter, r *http.Request)
  • func (s *Server) SaveConfig(w http.ResponseWriter, r *http.Request)

Constants

const CheckInterval = 30 * time.Second

CheckInterval represents how often we check for changes in htpasswd file.

Functions

func NewHandler

func NewHandler(server Server) *goji.Mux

NewHandler returns the master HTTP multiplexer/router.

Types

type Blob

type Blob struct {
	Name string `json:"name"`
	Size int64  `json:"size"`
}

Blob represents a single blob, its name and its size.

type HtpasswdFile

type HtpasswdFile struct {
	Users map[string]string
	// contains filtered or unexported fields
}

HtpasswdFile is a map for usernames to passwords.

func NewHtpasswdFromFile

func NewHtpasswdFromFile(path string) (*HtpasswdFile, error)

NewHtpasswdFromFile reads the users and passwords from a htpasswd file and returns them. If an error is encountered, it is returned, together with a nil-Pointer for the HtpasswdFile.

func (*HtpasswdFile) Reload

func (h *HtpasswdFile) Reload() error

Reload reloads the htpasswd file. If the reload fails, the Users map is not changed and the error is returned.

func (*HtpasswdFile) ReloadCheck

func (h *HtpasswdFile) ReloadCheck() error

ReloadCheck checks at most once per CheckInterval if the file changed and will reload the file if it did. It logs errors and successful reloads, and returns an error if any was encountered.

func (*HtpasswdFile) Validate

func (h *HtpasswdFile) Validate(user string, password string) bool

Validate returns true if password matches the stored password for user. If no password for user is stored, or the password is wrong, false is returned.

type Server

type Server struct {
	Path         string
	Listen       string
	Log          string
	CPUProfile   string
	TLSKey       string
	TLSCert      string
	TLS          bool
	NoAuth       bool
	AppendOnly   bool
	PrivateRepos bool
	Prometheus   bool
	Debug        bool
	MaxRepoSize  int64
	// contains filtered or unexported fields
}

Server determines how a Mux's handlers behave.

func (*Server) AuthHandler

func (s *Server) AuthHandler(f *HtpasswdFile, h http.Handler) http.HandlerFunc

AuthHandler wraps h with a http.HandlerFunc that performs basic authentication against the user/passwords pairs stored in f and returns the http.HandlerFunc.

func (*Server) CheckBlob

func (s *Server) CheckBlob(w http.ResponseWriter, r *http.Request)

CheckBlob tests whether a blob exists.

func (*Server) CheckConfig

func (s *Server) CheckConfig(w http.ResponseWriter, r *http.Request)

CheckConfig checks whether a configuration exists.

func (*Server) CreateRepo

func (s *Server) CreateRepo(w http.ResponseWriter, r *http.Request)

CreateRepo creates repository directories.

func (*Server) DeleteBlob

func (s *Server) DeleteBlob(w http.ResponseWriter, r *http.Request)

DeleteBlob deletes a blob from the repository.

func (*Server) DeleteConfig

func (s *Server) DeleteConfig(w http.ResponseWriter, r *http.Request)

DeleteConfig removes a config.

func (*Server) GetBlob

func (s *Server) GetBlob(w http.ResponseWriter, r *http.Request)

GetBlob retrieves a blob from the repository.

func (*Server) GetConfig

func (s *Server) GetConfig(w http.ResponseWriter, r *http.Request)

GetConfig allows for a config to be retrieved.

func (*Server) ListBlobs

func (s *Server) ListBlobs(w http.ResponseWriter, r *http.Request)

ListBlobs lists all blobs of a given type in an arbitrary order.

func (*Server) ListBlobsV1

func (s *Server) ListBlobsV1(w http.ResponseWriter, r *http.Request)

ListBlobsV1 lists all blobs of a given type in an arbitrary order.

func (*Server) ListBlobsV2

func (s *Server) ListBlobsV2(w http.ResponseWriter, r *http.Request)

ListBlobsV2 lists all blobs of a given type, together with their sizes, in an arbitrary order.

func (*Server) SaveBlob

func (s *Server) SaveBlob(w http.ResponseWriter, r *http.Request)

SaveBlob saves a blob to the repository.

func (*Server) SaveConfig

func (s *Server) SaveConfig(w http.ResponseWriter, r *http.Request)

SaveConfig allows for a config to be saved.