derp

package
v1.78.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 11, 2024 License: BSD-3-Clause Imports: 48 Imported by: 49

README

DERP

This directory (and subdirectories) contain the DERP code. The server itself is in ../cmd/derper.

DERP is a packet relay system (client and servers) where peers are addressed using WireGuard public keys instead of IP addresses.

It relays two types of packets:

  • "Disco" discovery messages (see ../disco) as the a side channel during NAT traversal.

  • Encrypted WireGuard packets as the fallback of last resort when UDP is blocked or NAT traversal fails.

DERP Map

Each client receives a "DERP Map" from the coordination server describing the DERP servers the client should try to use.

The client picks its home "DERP home" based on latency. This is done to keep costs low by avoid using cloud load balancers (pricey) or anycast, which would necessarily require server-side routing between DERP regions.

Clients pick their DERP home and report it to the coordination server which shares it to all the peers in the tailnet. When a peer wants to send a packet and it doesn't already have a WireGuard session open, it sends disco messages (some direct, and some over DERP), trying to do the NAT traversal. The client will make connections to multiple DERP regions as needed. Only the DERP home region connection needs to be alive forever.

DERP Regions

Tailscale runs 1 or more DERP nodes (instances of cmd/derper) in various geographic regions to make sure users have low latency to their DERP home.

Regions generally have multiple nodes per region "meshed" (routing to each other) together for redundancy: it allows for cloud failures or upgrades without kicking users out to a higher latency region. Instead, clients will reconnect to the next node in the region. Each node in the region is required to to be meshed with every other node in the region and forward packets to the other nodes in the region. Packets are forwarded only one hop within the region. There is no routing between regions. The assumption is that the mesh TCP connections are over a VPC that's very fast, low latency, and not charged per byte. The coordination server assigns the list of nodes in a region as a function of the tailnet, so all nodes within a tailnet should generally be on the same node and not require forwarding. Only after a failure do clients of a particular tailnet get split between nodes in a region and require inter-node forwarding. But over time it balances back out. There's also an admin-only DERP frame type to force close the TCP connection of a particular client to force them to reconnect to their primary if the operator wants to force things to balance out sooner. (Using the (*derphttp.Client).ClosePeer method, as used by Tailscale's internal rarely-used cmd/derpprune maintenance tool)

We generally run a minimum of three nodes in a region not for quorum reasons (there's no voting) but just because two is too uncomfortably few for cascading failure reasons: if you're running two nodes at 51% load (CPU, memory, etc) and then one fails, that makes the second one fail. With three or more nodes, you can run each node a bit hotter.

Documentation

Overview

Package derp implements the Designated Encrypted Relay for Packets (DERP) protocol.

DERP routes packets to clients using curve25519 keys as addresses.

DERP is used by Tailscale nodes to proxy encrypted WireGuard packets through the Tailscale cloud servers when a direct path cannot be found or opened. DERP is a last resort. Both sides between very aggressive NATs, firewalls, no IPv6, etc? Well, DERP.

Index

Constants

View Source 
const (
	PeerGoneReasonDisconnected  = PeerGoneReasonType(0x00) // peer disconnected from this server
	PeerGoneReasonNotHere       = PeerGoneReasonType(0x01) // server doesn't know about this peer, unexpected
	PeerGoneReasonMeshConnBroke = PeerGoneReasonType(0xf0) // invented by Client.RunWatchConnectionLoop on disconnect; not sent on the wire
)
View Source 
const (
	PeerPresentIsRegular  = 1 << 0
	PeerPresentIsMeshPeer = 1 << 1
	PeerPresentIsProber   = 1 << 2
	PeerPresentNotIdeal   = 1 << 3 // client said derp server is not its Region.Nodes[0] ideal node
)

PeerPresentFlags bits.

View Source 
const IdealNodeHeader = "Ideal-Node"

IdealNodeHeader is the HTTP request header sent on DERP HTTP client requests to indicate that they're connecting to their ideal (Region.Nodes[0]) node. The HTTP header value is the name of the node they wish they were connected to. This is an optional header.

View Source 
const MaxPacketSize = 64 << 10

MaxPacketSize is the maximum size of a packet sent over DERP. (This only includes the data bytes visible to magicsock, not including its on-wire framing overhead)

View Source 
const ProtocolVersion = 2

ProtocolVersion is bumped whenever there's a wire-incompatible change.

  • version 1 (zero on wire): consistent box headers, in use by employee dev nodes a bit
  • version 2: received packets have src addrs in frameRecvPacket at beginning

Variables

View Source 
var IdealNodeContextKey = ctxkey.New[string]("ideal-node", "")

IdealNodeContextKey is the context key used to pass the IdealNodeHeader value from the HTTP handler to the DERP server's Accept method.

Functions

This section is empty.

Types

type BytesSentRecv added in v1.10.0 

type BytesSentRecv struct {
	Sent uint64
	Recv uint64
	// Key is the public key of the client which sent/received these bytes.
	Key key.NodePublic
}

BytesSentRecv records the number of bytes that have been sent since the last traffic check for a given process, as well as the public key of the process sending those bytes.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is a DERP client.

func NewClient 

func NewClient(privateKey key.NodePrivate, nc Conn, brw *bufio.ReadWriter, logf logger.Logf, opts ...ClientOpt) (*Client, error)

func (*Client) ClosePeer added in v0.100.0 

func (c *Client) ClosePeer(target key.NodePublic) error

ClosePeer asks the server to close target's TCP connection. It's a fatal error if the client wasn't created using MeshKey.

func (*Client) ForwardPacket added in v0.100.0 

func (c *Client) ForwardPacket(srcKey, dstKey key.NodePublic, pkt []byte) (err error)

func (*Client) LocalAddr added in v1.20.0 

func (c *Client) LocalAddr() (netip.AddrPort, error)

LocalAddr returns the TCP connection's local address.

If the client is broken in some previously detectable way, it returns an error.

func (*Client) NotePreferred 

func (c *Client) NotePreferred(preferred bool) (err error)

NotePreferred sends a packet that tells the server whether this client is the user's preferred server. This is only used in the server for stats.

func (*Client) PublicKey added in v1.76.0 

func (c *Client) PublicKey() key.NodePublic

func (*Client) Recv 

func (c *Client) Recv() (m ReceivedMessage, err error)

Recv reads a message from the DERP server.

The returned message may alias memory owned by the Client; it should only be accessed until the next call to Client.

Once Recv returns an error, the Client is dead forever.

func (*Client) Send 

func (c *Client) Send(dstKey key.NodePublic, pkt []byte) error

Send sends a packet to the Tailscale node identified by dstKey.

It is an error if the packet is larger than 64KB.

func (*Client) SendPing added in v1.20.0 

func (c *Client) SendPing(data [8]byte) error

func (*Client) SendPong added in v1.6.0 

func (c *Client) SendPong(data [8]byte) error

func (*Client) ServerPublicKey added in v0.100.0 

func (c *Client) ServerPublicKey() key.NodePublic

ServerPublicKey returns the server's public key.

func (*Client) WatchConnectionChanges added in v0.99.1 

func (c *Client) WatchConnectionChanges() error

WatchConnectionChanges sends a request to subscribe to the peer's connection list. It's a fatal error if the client wasn't created using MeshKey.

type ClientOpt added in v0.100.0 

type ClientOpt interface {
	// contains filtered or unexported methods
}

ClientOpt is an option passed to NewClient.

func CanAckPings added in v1.6.0 

func CanAckPings(v bool) ClientOpt

CanAckPings returns a ClientOpt to set whether it advertises to the server that it's capable of acknowledging ping requests.

func IsProber added in v1.12.0 

func IsProber(v bool) ClientOpt

IsProber returns a ClientOpt to pass to the DERP server during connect to declare that this client is a a prober.

func MeshKey added in v0.100.0 

func MeshKey(key string) ClientOpt

MeshKey returns a ClientOpt to pass to the DERP server during connect to get access to join the mesh.

An empty key means to not use a mesh key.

func ServerPublicKey added in v1.2.0 

func ServerPublicKey(key key.NodePublic) ClientOpt

ServerPublicKey returns a ClientOpt to declare that the server's DERP public key is known. If key is the zero value, the returned ClientOpt is a no-op.

type Conn 

type Conn interface {
	io.WriteCloser
	LocalAddr() net.Addr
	// The *Deadline methods follow the semantics of net.Conn.
	SetDeadline(time.Time) error
	SetReadDeadline(time.Time) error
	SetWriteDeadline(time.Time) error
}

Conn is the subset of the underlying net.Conn the DERP Server needs. It is a defined type so that non-net connections can be used.

type HealthMessage added in v1.16.0 

type HealthMessage struct {
	// Problem, if non-empty, is a description of why the connection
	// is unhealthy.
	//
	// The empty string means the connection is healthy again.
	//
	// The default condition is healthy, so the server doesn't
	// broadcast a HealthMessage until a problem exists.
	Problem string
}

HealthMessage is a one-way message from server to client, declaring the connection health state.

type KeepAliveMessage added in v1.6.0 

type KeepAliveMessage struct{}

KeepAliveMessage is a one-way empty message from server to client, just to keep the connection alive. It's like a PingMessage, but doesn't solicit a reply from the client.

type PacketForwarder added in v0.100.0 

type PacketForwarder interface {
	ForwardPacket(src, dst key.NodePublic, payload []byte) error
	String() string
}

PacketForwarder is something that can forward packets.

It's mostly an interface for circular dependency reasons; the typical implementation is derphttp.Client. The other implementation is a multiForwarder, which this package creates as needed if a public key gets more than one PacketForwarder registered for it.

type PeerGoneMessage added in v0.98.0 

type PeerGoneMessage struct {
	Peer   key.NodePublic
	Reason PeerGoneReasonType
}

PeerGoneMessage is a ReceivedMessage that indicates that the client identified by the underlying public key is not connected to this server.

It has only historically been sent by the server when the client connection count decremented from 1 to 0 and not from e.g. 2 to 1. See https://github.com/tailscale/tailscale/issues/13566 for details.

type PeerGoneReasonType added in v1.40.0 

type PeerGoneReasonType byte

PeerGoneReasonType is a one byte reason code explaining why a server does not have a path to the requested destination.

type PeerPresentFlags added in v1.70.0 

type PeerPresentFlags byte

PeerPresentFlags is an optional byte of bit flags sent after a framePeerPresent message.

For a modern server, the value should always be non-zero. If the value is zero, that means the server doesn't support this field.

type PeerPresentMessage added in v0.99.1 

type PeerPresentMessage struct {
	// Key is the public key of the client.
	Key key.NodePublic
	// IPPort is the remote IP and port of the client.
	IPPort netip.AddrPort
	// Flags is a bitmask of info about the client.
	Flags PeerPresentFlags
}

PeerPresentMessage is a ReceivedMessage that indicates that the client is connected to the server. (Only used by trusted mesh clients)

It will be sent to client watchers for every new connection from a client, even if the client's already connected with that public key. See https://github.com/tailscale/tailscale/issues/13566 for PeerPresentMessage and PeerGoneMessage not being 1:1.

type PingMessage added in v1.6.0 

type PingMessage [8]byte

PingMessage is a request from a client or server to reply to the other side with a PongMessage with the given payload.

type PongMessage added in v1.20.0 

type PongMessage [8]byte

PongMessage is a reply to a PingMessage from a client or server with the payload sent previously in a PingMessage.

type ReceivedMessage 

type ReceivedMessage interface {
	// contains filtered or unexported methods
}

ReceivedMessage represents a type returned by Client.Recv. Unless otherwise documented, the returned message aliases the byte slice provided to Recv and thus the message is only as good as that buffer, which is up to the caller.

type ReceivedPacket 

type ReceivedPacket struct {
	Source key.NodePublic
	// Data is the received packet bytes. It aliases the memory
	// passed to Client.Recv.
	Data []byte
}

ReceivedPacket is a ReceivedMessage representing an incoming packet.

type Server 

type Server struct {
	// WriteTimeout, if non-zero, specifies how long to wait
	// before failing when writing to a client.
	WriteTimeout time.Duration
	// contains filtered or unexported fields
}

Server is a DERP server.

func NewServer 

func NewServer(privateKey key.NodePrivate, logf logger.Logf) *Server

NewServer returns a new DERP server. It doesn't listen on its own. Connections are given to it via Server.Accept.

func (*Server) Accept 

func (s *Server) Accept(ctx context.Context, nc Conn, brw *bufio.ReadWriter, remoteAddr string)

Accept adds a new connection to the server and serves it.

The provided bufio ReadWriter must be already connected to nc. Accept blocks until the Server is closed or the connection closes on its own.

Accept closes nc.

func (*Server) AddPacketForwarder added in v0.100.0 

func (s *Server) AddPacketForwarder(dst key.NodePublic, fwd PacketForwarder)

AddPacketForwarder registers fwd as a packet forwarder for dst. fwd must be comparable.

func (*Server) Close 

func (s *Server) Close() error

Close closes the server and waits for the connections to disconnect.

func (*Server) ConsistencyCheck added in v0.100.0 

func (s *Server) ConsistencyCheck() error

func (*Server) ExpVar 

func (s *Server) ExpVar() expvar.Var

ExpVar returns an expvar variable suitable for registering with expvar.Publish.

func (*Server) HasMeshKey added in v0.99.1 

func (s *Server) HasMeshKey() bool

HasMeshKey reports whether the server is configured with a mesh key.

func (*Server) IsClientConnectedForTest added in v1.24.0 

func (s *Server) IsClientConnectedForTest(k key.NodePublic) bool

IsClientConnectedForTest reports whether the client with specified key is connected. This is used in tests to verify that nodes are connected.

func (*Server) MeshKey added in v0.100.0 

func (s *Server) MeshKey() string

MeshKey returns the configured mesh key, if any.

func (*Server) MetaCert added in v1.2.0 

func (s *Server) MetaCert() []byte

MetaCert returns the server metadata cert that can be sent by the TLS server to let the client skip a round trip during start-up.

func (*Server) PrivateKey added in v0.100.0 

func (s *Server) PrivateKey() key.NodePrivate

PrivateKey returns the server's private key.

func (*Server) PublicKey added in v0.100.0 

func (s *Server) PublicKey() key.NodePublic

PublicKey returns the server's public key.

func (*Server) RemovePacketForwarder added in v0.100.0 

func (s *Server) RemovePacketForwarder(dst key.NodePublic, fwd PacketForwarder)

RemovePacketForwarder removes fwd as a packet forwarder for dst. fwd must be comparable.

func (*Server) ServeDebugTraffic added in v1.10.0 

func (s *Server) ServeDebugTraffic(w http.ResponseWriter, r *http.Request)

func (*Server) SetMeshKey added in v0.99.1 

func (s *Server) SetMeshKey(v string)

SetMesh sets the pre-shared key that regional DERP servers used to mesh amongst themselves.

It must be called before serving begins.

func (*Server) SetVerifyClient added in v1.10.0 

func (s *Server) SetVerifyClient(v bool)

SetVerifyClients sets whether this DERP server verifies clients through tailscaled.

It must be called before serving begins.

func (*Server) SetVerifyClientURL added in v1.62.0 

func (s *Server) SetVerifyClientURL(v string)

SetVerifyClientURL sets the admission controller URL to use for verifying clients. If empty, all clients are accepted (unless restricted by SetVerifyClient checking against tailscaled).

func (*Server) SetVerifyClientURLFailOpen added in v1.62.0 

func (s *Server) SetVerifyClientURLFailOpen(v bool)

SetVerifyClientURLFailOpen sets whether to allow clients to connect if the admission controller URL is unreachable.

type ServerInfoMessage added in v1.2.0 

type ServerInfoMessage struct {
	// TokenBucketBytesPerSecond is how many bytes per second the
	// server says it will accept, including all framing bytes.
	//
	// Zero means unspecified. There might be a limit, but the
	// client need not try to respect it.
	TokenBucketBytesPerSecond int

	// TokenBucketBytesBurst is how many bytes the server will
	// allow to burst, temporarily violating
	// TokenBucketBytesPerSecond.
	//
	// Zero means unspecified. There might be a limit, but the
	// client need not try to respect it.
	TokenBucketBytesBurst int
}

ServerInfoMessage is sent by the server upon first connect.

type ServerRestartingMessage added in v1.16.0 

type ServerRestartingMessage struct {
	// ReconnectIn is an advisory duration that the client should wait
	// before attempting to reconnect. It might be zero.
	// It exists for the server to smear out the reconnects.
	ReconnectIn time.Duration

	// TryFor is an advisory duration for how long the client
	// should attempt to reconnect before giving up and proceeding
	// with its normal connection failure logic. The interval
	// between retries is undefined for now.
	// A server should not send a TryFor duration more than a few
	// seconds.
	TryFor time.Duration
}

ServerRestartingMessage is a one-way message from server to client, advertising that the server is restarting.

Source Files

View all Source files

Directories

Path Synopsis
Package derphttp implements DERP-over-HTTP.
 Package derphttp implements DERP-over-HTTP.
xdp
Package xdp contains the XDP STUN program.
 Package xdp contains the XDP STUN program.
headers
The update program fetches the libbpf headers from the libbpf GitHub repository and writes them to disk.
 The update program fetches the libbpf headers from the libbpf GitHub repository and writes them to disk.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.