README

etcd

Go Report Card Coverage Build Status Travis Build Status Semaphore Docs Godoc Releases LICENSE

Note: The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order to get stable binaries.

etcd Logo

etcd is a distributed reliable key-value store for the most critical data of a distributed system, with a focus on being:

  • Simple: well-defined, user-facing API (gRPC)
  • Secure: automatic TLS with optional client cert authentication
  • Fast: benchmarked 10,000 writes/sec
  • Reliable: properly distributed using Raft

etcd is written in Go and uses the Raft consensus algorithm to manage a highly-available replicated log.

etcd is used in production by many companies, and the development team stands behind it in critical deployment scenarios, where etcd is frequently teamed with applications such as Kubernetes, locksmith, vulcand, Doorman, and many others. Reliability is further ensured by rigorous testing.

See etcdctl for a simple command line client.

Community meetings

etcd contributors and maintainers have monthly (every four weeks) meetings at 11:00 AM (USA Pacific) on Thursday.

An initial agenda will be posted to the shared Google docs a day before each meeting, and everyone is welcome to suggest additional topics or other agendas.

Time:

Join Hangouts Meet: meet.google.com/umg-nrxn-qvs

Join by phone: +1 405-792-0633‬ PIN: ‪299 906‬#

Getting started

Getting etcd

The easiest way to get etcd is to use one of the pre-built release binaries which are available for OSX, Linux, Windows, and Docker on the release page.

For more installation guides, please check out play.etcd.io and operating etcd.

For those wanting to try the very latest version, build the latest version of etcd from the master branch. This first needs Go installed (version 1.13+ is required). All development occurs on master, including new features and bug fixes. Bug fixes are first targeted at master and subsequently ported to release branches, as described in the branch management guide.

Running etcd

First start a single-member cluster of etcd.

If etcd is installed using the pre-built release binaries, run it from the installation location as below:

/tmp/etcd-download-test/etcd

The etcd command can be simply run as such if it is moved to the system path as below:

mv /tmp/etcd-download-test/etcd /usr/local/bin/
etcd

If etcd is built from the master branch, run it as below:

./bin/etcd

This will bring up etcd listening on port 2379 for client communication and on port 2380 for server-to-server communication.

Next, let's set a single key, and then retrieve it:

etcdctl put mykey "this is awesome"
etcdctl get mykey

etcd is now running and serving client requests. For more, please check out:

etcd TCP ports

The official etcd ports are 2379 for client requests, and 2380 for peer communication.

Running a local etcd cluster

First install goreman, which manages Procfile-based applications.

Our Procfile script will set up a local example cluster. Start it with:

goreman start

This will bring up 3 etcd members infra1, infra2 and infra3 and etcd grpc-proxy, which runs locally and composes a cluster.

Every cluster member and proxy accepts key value reads and key value writes.

Follow the steps in Procfile.learner to add a learner node to the cluster. Start the learner node with:

goreman -f ./Procfile.learner start
Next steps

Now it's time to dig into the full etcd API and other guides.

Contact

Contributing

See CONTRIBUTING for details on submitting patches and the contribution workflow.

Reporting bugs

See reporting bugs for details about reporting any issues.

Reporting a security vulnerability

See security disclosure and release process for details on how to report a security vulnerability and how the etcd team manages it.

Issue and PR management

See issue triage guidelines for details on how issues are managed.

See PR management for guidelines on how pull requests are managed.

etcd Emeritus Maintainers

These emeritus maintainers dedicated a part of their career to etcd and reviewed code, triaged bugs, and pushed the project forward over a substantial period of time. Their contribution is greatly appreciated.

  • Fanmin Shi
  • Anthony Romano
License

etcd is under the Apache 2.0 license. See the LICENSE file for details.

Expand ▾ Collapse ▴

Documentation

Overview

Package main is a simple wrapper of the real etcd entrypoint package (located at go.etcd.io/etcd/etcdmain) to ensure that etcd is still "go getable"; e.g. `go get go.etcd.io/etcd` works as expected and builds a binary in $GOBIN/etcd

This package should NOT be extended or modified in any way; to modify the etcd binary, work in the `go.etcd.io/etcd/etcdmain` package.

Source Files

Directories

Path Synopsis
Documentation/learning/lock/client
Documentation/learning/lock/storage
auth Package auth provides client role authentication for accessing keys in etcd.
auth/authpb Package authpb is a generated protocol buffer package.
client Package client provides bindings for the etcd APIs.
client/integration Package integration implements tests built upon embedded etcd, focusing on the correctness of the etcd v2 client.
clientv3 Package clientv3 implements the official Go etcd client for v3.
clientv3/balancer Package balancer implements client balancer.
clientv3/balancer/connectivity Package connectivity implements client connectivity operations.
clientv3/balancer/picker Package picker defines/implements client balancer picker policy.
clientv3/balancer/resolver/endpoint Package endpoint resolves etcd entpoints using grpc targets of the form 'endpoint://<id>/<endpoint>'.
clientv3/clientv3util Package clientv3util contains utility functions derived from clientv3.
clientv3/concurrency Package concurrency implements concurrency operations on top of etcd such as distributed locks, barriers, and elections.
clientv3/credentials Package credentials implements gRPC credential interface with etcd specific logic.
clientv3/integration Package integration implements tests built upon embedded etcd, and focuses on correctness of etcd client.
clientv3/leasing Package leasing serves linearizable reads from a local cache by acquiring exclusive write access to keys through a client-side leasing protocol.
clientv3/mirror Package mirror implements etcd mirroring operations.
clientv3/namespace Package namespace is a clientv3 wrapper that translates all keys to begin with a given prefix.
clientv3/naming Package naming provides an etcd-backed gRPC resolver for discovering gRPC services.
clientv3/ordering Package ordering is a clientv3 wrapper that caches response header revisions to detect ordering violations from stale responses.
clientv3/snapshot Package snapshot implements utilities around etcd snapshot.
clientv3/yaml Package yaml handles yaml-formatted clientv3 configuration data.
contrib/raftexample raftexample is a simple KV store using the raft and rafthttp libraries.
contrib/recipes Package recipe contains experimental client-side distributed synchronization primitives.
embed Package embed provides bindings for embedding an etcd server in a program.
etcdctl etcdctl is a command line application that controls etcd.
etcdctl/ctlv2 Package ctlv2 contains the main entry point for the etcdctl for v2 API.
etcdctl/ctlv2/command Package command is a set of libraries for etcdctl commands.
etcdctl/ctlv3 Package ctlv3 contains the main entry point for the etcdctl for v3 API.
etcdctl/ctlv3/command Package command is a set of libraries for etcd v3 commands.
etcdmain Package etcdmain contains the main entry point for the etcd binary.
etcdserver Package etcdserver defines how etcd servers interact and store their states.
etcdserver/api Package api manages the capabilities and features that are exposed to clients by the etcd cluster.
etcdserver/api/etcdhttp Package etcdhttp implements HTTP transportation layer for etcdserver.
etcdserver/api/membership Package membership describes individual etcd members and clusters of members.
etcdserver/api/membership/membershippb Package membershippb is a generated protocol buffer package.
etcdserver/api/rafthttp Package rafthttp implements HTTP transportation layer for etcd/raft pkg.
etcdserver/api/snap Package snap handles Raft nodes' states with snapshots.
etcdserver/api/snap/snappb Package snappb is a generated protocol buffer package.
etcdserver/api/v2auth Package v2auth implements etcd authentication.
etcdserver/api/v2discovery Package v2discovery provides an implementation of the cluster discovery that is used by etcd with v2 client.
etcdserver/api/v2error Package v2error describes errors in etcd project.
etcdserver/api/v2http Package v2http provides etcd client and server implementations.
etcdserver/api/v2http/httptypes Package httptypes defines how etcd's HTTP API entities are serialized to and deserialized from JSON.
etcdserver/api/v2stats Package v2stats defines a standard interface for etcd cluster statistics.
etcdserver/api/v2store Package v2store defines etcd's in-memory key/value store in v2 API.
etcdserver/api/v2v3 Package v2v3 provides a ServerV2 implementation backed by clientv3.Client.
etcdserver/api/v3alarm Package v3alarm manages health status alarms in etcd.
etcdserver/api/v3client Package v3client provides clientv3 interfaces from an etcdserver.
etcdserver/api/v3compactor Package v3compactor implements automated policies for compacting etcd's mvcc storage.
etcdserver/api/v3election Package v3election provides a v3 election service from an etcdserver.
etcdserver/api/v3election/v3electionpb Package v3electionpb is a generated protocol buffer package.
etcdserver/api/v3election/v3electionpb/gw Package v3electionpb is a reverse proxy.
etcdserver/api/v3lock Package v3lock provides a v3 locking service from an etcdserver.
etcdserver/api/v3lock/v3lockpb Package v3lockpb is a generated protocol buffer package.
etcdserver/api/v3lock/v3lockpb/gw Package v3lockpb is a reverse proxy.
etcdserver/api/v3rpc Package v3rpc implements etcd v3 RPC system based on gRPC.
etcdserver/api/v3rpc/rpctypes Package rpctypes has types and values shared by the etcd server and client for v3 RPC interaction.
etcdserver/etcdserverpb Package etcdserverpb is a generated protocol buffer package.
etcdserver/etcdserverpb/gw Package etcdserverpb is a reverse proxy.
functional/agent Package agent implements functional-tester agent server.
functional/cmd/etcd-agent etcd-agent is a program that runs functional-tester agent.
functional/cmd/etcd-proxy etcd-proxy is a proxy layer that simulates various network conditions.
functional/cmd/etcd-runner etcd-runner is a program for testing etcd clientv3 features against a fault injected cluster.
functional/cmd/etcd-tester etcd-tester is a program that runs functional-tester client.
functional/rpcpb
functional/runner Package runner implements individual etcd-runner commands for the etcd-runner utility.
functional/tester Package tester implements functional-tester tester server.
integration Package integration implements tests built upon embedded etcd, and focus on etcd correctness.
lease Package lease provides an interface and implementation for time-limited leases over arbitrary resources.
lease/leasehttp Package leasehttp serves lease renewals made through HTTP requests.
lease/leasepb Package leasepb is a generated protocol buffer package.
mvcc Package mvcc defines etcd's stable MVCC storage.
mvcc/backend Package backend defines a standard interface for etcd's backend MVCC storage.
mvcc/mvccpb Package mvccpb is a generated protocol buffer package.
pkg/adt Package adt implements useful abstract data types.
pkg/contention Package contention provides facilities for detecting system contention.
pkg/cpuutil Package cpuutil provides facilities for detecting cpu-specific features.
pkg/crc Package crc provides utility function for cyclic redundancy check algorithms.
pkg/debugutil Package debugutil includes utility functions for debugging.
pkg/expect Package expect implements a small expect-style interface
pkg/fileutil Package fileutil implements utility functions related to files and paths.
pkg/flags Package flags implements command-line flag parsing.
pkg/httputil Package httputil provides HTTP utility functions.
pkg/idutil Package idutil implements utility functions for generating unique, randomized ids.
pkg/ioutil Package ioutil implements I/O utility functions.
pkg/logutil Package logutil includes utilities to facilitate logging.
pkg/mock/mockserver Package mockserver provides mock implementations for etcdserver's server interface.
pkg/mock/mockstorage Package mockstorage provides mock implementations for etcdserver's storage interface.
pkg/mock/mockstore Package mockstore provides mock structures for the etcd store package.
pkg/mock/mockwait Package mockwait provides mock implementations for pkg/wait.
pkg/netutil Package netutil implements network-related utility functions.
pkg/osutil Package osutil implements operating system-related utility functions.
pkg/pathutil Package pathutil implements utility functions for handling slash-separated paths.
pkg/pbutil Package pbutil defines interfaces for handling Protocol Buffer objects.
pkg/proxy Package proxy implements proxy servers for network fault testing.
pkg/report Package report generates human-readable benchmark reports.
pkg/runtime Package runtime implements utility functions for runtime systems.
pkg/schedule Package schedule provides mechanisms and policies for scheduling units of work.
pkg/srv Package srv looks up DNS SRV records.
pkg/stringutil Package stringutil exports string utility functions.
pkg/systemd Package systemd provides utility functions for systemd.
pkg/testutil Package testutil provides test utility functions.
pkg/tlsutil Package tlsutil provides utility functions for handling TLS.
pkg/traceutil Package traceutil implements tracing utilities using "context".
pkg/transport Package transport implements various HTTP transport utilities based on Go net package.
pkg/types Package types declares various data types and implements type-checking functions.
pkg/wait Package wait provides utility functions for polling, listening using Go channel.
proxy/grpcproxy Package grpcproxy is an OSI level 7 proxy for etcd v3 API requests.
proxy/grpcproxy/adapter Package adapter provides gRPC adapters between client and server gRPC interfaces without needing to go through a gRPC connection.
proxy/grpcproxy/cache Package cache exports functionality for efficiently caching and mapping `RangeRequest`s to corresponding `RangeResponse`s.
proxy/httpproxy Package httpproxy implements etcd httpproxy.
proxy/tcpproxy Package tcpproxy is an OSI level 4 proxy for routing etcd clients to etcd servers.
raft Package raft sends and receives messages in the Protocol Buffer format defined in the raftpb package.
raft/confchange
raft/quorum
raft/raftpb Package raftpb is a generated protocol buffer package.
raft/rafttest Package rafttest provides functional tests for etcd's raft implementation.
raft/tracker
tests/e2e Package e2e implements tests built upon etcd binaries, and focus on end-to-end testing.
tools/benchmark benchmark is a program for benchmarking etcd v3 API performance.
tools/benchmark/cmd Package cmd implements individual benchmark commands for the benchmark utility.
tools/etcd-dump-db etcd-dump-db inspects etcd db files.
tools/etcd-dump-logs etcd-dump-logs is a program for analyzing etcd server write ahead logs.
tools/etcd-dump-metrics etcd-dump-metrics automates etcd Prometheus metrics documentation.
tools/local-tester/bridge Package main is the entry point for the local tester network bridge.
version Package version implements etcd version parsing and contains latest version information.
wal Package wal provides an implementation of a write ahead log that is used by etcd.
wal/walpb Package walpb is a generated protocol buffer package.