package module
Version: v0.12.4 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Jul 29, 2021 License: MIT Imports: 52 Imported by: 0



GoDoc Build Status Discourse posts

A Kademlia DHT implementation on go-libp2p

Table of Contents


go get github.com/libp2p/go-libp2p-kad-dht


Go to https://godoc.org/github.com/libp2p/go-libp2p-kad-dht.


Contributions welcome. Please check out the issues.

Check out our contributing document for more information on how we work, and about contributing in general. Please be aware that all interactions related to libp2p are subject to the IPFS Code of Conduct.

Small note: If editing the README, please conform to the standard-readme specification.


MIT © Protocol Labs Inc.

The last gx published version of this module was: 4.4.34: QmXuNFLZc6Nb5akB4sZsxK3doShsFKT1sZFvxLXJvZQwAW



Package dht implements a distributed hash table that satisfies the ipfs routing interface. This DHT is modeled after kademlia with S/Kademlia modifications.



View Source
const DefaultPrefix protocol.ID = "/ipfs"

DefaultPrefix is the application specific prefix attached to all DHT protocols by default.


View Source
var (
	// ProtocolDHT is the default DHT protocol.
	ProtocolDHT protocol.ID = "/ipfs/kad/1.0.0"
	// DefaultProtocols spoken by the DHT.
	DefaultProtocols = []protocol.ID{ProtocolDHT}
View Source
var DefaultBootstrapPeers []multiaddr.Multiaddr

DefaultBootstrapPeers is a set of public DHT bootstrap peers provided by libp2p.

View Source
var ErrNoPeersQueried = errors.New("failed to query any peers")

ErrNoPeersQueried is returned when we failed to connect to any peers.

View Source
var ErrReadTimeout = net.ErrReadTimeout

ErrReadTimeout is an error that occurs when no message is read within the timeout period.

View Source
var LookupEventBufferSize = 16

LookupEventBufferSize is the number of events to buffer.


func GetDefaultBootstrapPeerAddrInfos

func GetDefaultBootstrapPeerAddrInfos() []peer.AddrInfo

GetDefaultBootstrapPeerAddrInfos returns the peer.AddrInfos for the default bootstrap peers so we can use these for initializing the DHT by passing these to the BootstrapPeers(...) option.

func NewRTPeerDiversityFilter

func NewRTPeerDiversityFilter(h host.Host, maxPerCpl, maxForTable int) *rtPeerIPGroupFilter

NewRTPeerDiversityFilter constructs the `PeerIPGroupFilter` that will be used to configure the diversity filter for the Routing Table. Please see the docs for `peerdiversity.PeerIPGroupFilter` AND `peerdiversity.Filter` for more details.

func PrivateQueryFilter

func PrivateQueryFilter(_ interface{}, ai peer.AddrInfo) bool

PrivateQueryFilter doens't currently restrict which peers we are willing to query from the local DHT.

func PrivateRoutingTableFilter

func PrivateRoutingTableFilter(dht interface{}, p peer.ID) bool

PrivateRoutingTableFilter allows a peer to be added to the routing table if the connections to that peer indicate that it is on a private network

func PublicQueryFilter

func PublicQueryFilter(_ interface{}, ai peer.AddrInfo) bool

PublicQueryFilter returns true if the peer is suspected of being publicly accessible

func PublicRoutingTableFilter

func PublicRoutingTableFilter(dht interface{}, p peer.ID) bool

PublicRoutingTableFilter allows a peer to be added to the routing table if the connections to that peer indicate that it is on a public network

func PublishLookupEvent

func PublishLookupEvent(ctx context.Context, ev *LookupEvent)

PublishLookupEvent publishes a query event to the query event channel associated with the given context, if any.

func Quorum

func Quorum(n int) routing.Option

Quorum is a DHT option that tells the DHT how many peers it needs to get values from before returning the best one. Zero means the DHT query should complete instead of returning early.

Default: 0

func RegisterForLookupEvents

func RegisterForLookupEvents(ctx context.Context) (context.Context, <-chan *LookupEvent)

RegisterForLookupEvents registers a lookup event channel with the given context. The returned context can be passed to DHT queries to receive lookup events on the returned channels.

The passed context MUST be canceled when the caller is no longer interested in query events.


type IpfsDHT

type IpfsDHT struct {

	// ProviderManager stores & manages the provider records for this Dht peer.
	ProviderManager *providers.ProviderManager

	Validator record.Validator
	// contains filtered or unexported fields

IpfsDHT is an implementation of Kademlia with S/Kademlia modifications. It is used to implement the base Routing module.

func New

func New(ctx context.Context, h host.Host, options ...Option) (*IpfsDHT, error)

New creates a new DHT with the specified host and options. Please note that being connected to a DHT peer does not necessarily imply that it's also in the DHT Routing Table. If the Routing Table has more than "minRTRefreshThreshold" peers, we consider a peer as a Routing Table candidate ONLY when we successfully get a query response from it OR if it send us a query.

func NewDHT

func NewDHT(ctx context.Context, h host.Host, dstore ds.Batching) *IpfsDHT

NewDHT creates a new DHT object with the given peer as the 'local' host. IpfsDHT's initialized with this function will respond to DHT requests, whereas IpfsDHT's initialized with NewDHTClient will not.

func NewDHTClient

func NewDHTClient(ctx context.Context, h host.Host, dstore ds.Batching) *IpfsDHT

NewDHTClient creates a new DHT object with the given peer as the 'local' host. IpfsDHT clients initialized with this function will not respond to DHT requests. If you need a peer to respond to DHT requests, use NewDHT instead.

func (*IpfsDHT) Bootstrap

func (dht *IpfsDHT) Bootstrap(ctx context.Context) error

Bootstrap tells the DHT to get into a bootstrapped state satisfying the IpfsRouter interface.

func (*IpfsDHT) Close

func (dht *IpfsDHT) Close() error

Close calls Process Close.

func (*IpfsDHT) Context

func (dht *IpfsDHT) Context() context.Context

Context returns the DHT's context.

func (*IpfsDHT) FindLocal

func (dht *IpfsDHT) FindLocal(id peer.ID) peer.AddrInfo

FindLocal looks for a peer with a given ID connected to this dht and returns the peer and the table it was found in.

func (*IpfsDHT) FindPeer

func (dht *IpfsDHT) FindPeer(ctx context.Context, id peer.ID) (_ peer.AddrInfo, err error)

FindPeer searches for a peer with given ID.

func (*IpfsDHT) FindProviders

func (dht *IpfsDHT) FindProviders(ctx context.Context, c cid.Cid) ([]peer.AddrInfo, error)

FindProviders searches until the context expires.

func (*IpfsDHT) FindProvidersAsync

func (dht *IpfsDHT) FindProvidersAsync(ctx context.Context, key cid.Cid, count int) <-chan peer.AddrInfo

FindProvidersAsync is the same thing as FindProviders, but returns a channel. Peers will be returned on the channel as soon as they are found, even before the search query completes. If count is zero then the query will run until it completes. Note: not reading from the returned channel may block the query from progressing.

func (*IpfsDHT) ForceRefresh

func (dht *IpfsDHT) ForceRefresh() <-chan error

ForceRefresh acts like RefreshRoutingTable but forces the DHT to refresh all buckets in the Routing Table irrespective of when they were last refreshed.

The returned channel will block until the refresh finishes, then yield the error and close. The channel is buffered and safe to ignore.

func (*IpfsDHT) GetClosestPeers

func (dht *IpfsDHT) GetClosestPeers(ctx context.Context, key string) ([]peer.ID, error)

GetClosestPeers is a Kademlia 'node lookup' operation. Returns a channel of the K closest peers to the given key.

If the context is canceled, this function will return the context error along with the closest K peers it has found so far.

func (*IpfsDHT) GetPublicKey

func (dht *IpfsDHT) GetPublicKey(ctx context.Context, p peer.ID) (ci.PubKey, error)

GetPublicKey gets the public key when given a Peer ID. It will extract from the Peer ID if inlined or ask the node it belongs to or ask the DHT.

func (*IpfsDHT) GetRoutingTableDiversityStats

func (dht *IpfsDHT) GetRoutingTableDiversityStats() []peerdiversity.CplDiversityStats

GetRoutingTableDiversityStats returns the diversity stats for the Routing Table.

func (*IpfsDHT) GetValue

func (dht *IpfsDHT) GetValue(ctx context.Context, key string, opts ...routing.Option) (_ []byte, err error)

GetValue searches for the value corresponding to given Key.

func (*IpfsDHT) Host

func (dht *IpfsDHT) Host() host.Host

Host returns the libp2p host this DHT is operating with.

func (*IpfsDHT) Mode

func (dht *IpfsDHT) Mode() ModeOpt

Mode allows introspection of the operation mode of the DHT

func (*IpfsDHT) PeerID

func (dht *IpfsDHT) PeerID() peer.ID

PeerID returns the DHT node's Peer ID.

func (*IpfsDHT) PeerKey

func (dht *IpfsDHT) PeerKey() []byte

PeerKey returns a DHT key, converted from the DHT node's Peer ID.

func (*IpfsDHT) Ping

func (dht *IpfsDHT) Ping(ctx context.Context, p peer.ID) error

Ping sends a ping message to the passed peer and waits for a response.

func (*IpfsDHT) Process

func (dht *IpfsDHT) Process() goprocess.Process

Process returns the DHT's process.

func (*IpfsDHT) Provide

func (dht *IpfsDHT) Provide(ctx context.Context, key cid.Cid, brdcst bool) (err error)

Provide makes this node announce that it can provide a value for the given key

func (*IpfsDHT) PutValue

func (dht *IpfsDHT) PutValue(ctx context.Context, key string, value []byte, opts ...routing.Option) (err error)

PutValue adds value corresponding to given Key. This is the top level "Store" operation of the DHT

func (*IpfsDHT) RefreshRoutingTable

func (dht *IpfsDHT) RefreshRoutingTable() <-chan error

RefreshRoutingTable tells the DHT to refresh it's routing tables.

The returned channel will block until the refresh finishes, then yield the error and close. The channel is buffered and safe to ignore.

func (*IpfsDHT) RoutingTable

func (dht *IpfsDHT) RoutingTable() *kb.RoutingTable

RoutingTable returns the DHT's routingTable.

func (*IpfsDHT) SearchValue

func (dht *IpfsDHT) SearchValue(ctx context.Context, key string, opts ...routing.Option) (<-chan []byte, error)

SearchValue searches for the value corresponding to given Key and streams the results.

type KeyKadID

type KeyKadID struct {
	Key string
	Kad kbucket.ID

KeyKadID contains the Kademlia key in string and binary form.

func NewKeyKadID

func NewKeyKadID(k string) *KeyKadID

NewKeyKadID creates a KeyKadID from a string Kademlia ID.

type LookupEvent

type LookupEvent struct {
	// Node is the ID of the node performing the lookup.
	Node *PeerKadID
	// ID is a unique identifier for the lookup instance.
	ID uuid.UUID
	// Key is the Kademlia key used as a lookup target.
	Key *KeyKadID
	// Request, if not nil, describes a state update event, associated with an outgoing query request.
	Request *LookupUpdateEvent
	// Response, if not nil, describes a state update event, associated with an outgoing query response.
	Response *LookupUpdateEvent
	// Terminate, if not nil, describe a termination event.
	Terminate *LookupTerminateEvent

LookupEvent is emitted for every notable event that happens during a DHT lookup. LookupEvent supports JSON marshalling because all of its fields do, recursively.

func NewLookupEvent

func NewLookupEvent(
	node peer.ID,
	id uuid.UUID,
	key string,
	request *LookupUpdateEvent,
	response *LookupUpdateEvent,
	terminate *LookupTerminateEvent,
) *LookupEvent

NewLookupEvent creates a LookupEvent automatically converting the node libp2p Peer ID to a PeerKadID and the string Kademlia key to a KeyKadID.

type LookupTerminateEvent

type LookupTerminateEvent struct {
	// Reason is the reason for lookup termination.
	Reason LookupTerminationReason

LookupTerminateEvent describes a lookup termination event.

func NewLookupTerminateEvent

func NewLookupTerminateEvent(reason LookupTerminationReason) *LookupTerminateEvent

NewLookupTerminateEvent creates a new lookup termination event with a given reason.

type LookupTerminationReason

type LookupTerminationReason int

LookupTerminationReason captures reasons for terminating a lookup.

const (
	// LookupStopped indicates that the lookup was aborted by the user's stopFn.
	LookupStopped LookupTerminationReason = iota
	// LookupCancelled indicates that the lookup was aborted by the context.
	// LookupStarvation indicates that the lookup terminated due to lack of unqueried peers.
	// LookupCompleted indicates that the lookup terminated successfully, reaching the Kademlia end condition.

func (LookupTerminationReason) MarshalJSON

func (r LookupTerminationReason) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON encoding of the passed lookup termination reason.

func (LookupTerminationReason) String

func (r LookupTerminationReason) String() string

type LookupUpdateEvent

type LookupUpdateEvent struct {
	// Cause is the peer whose response (or lack of response) caused the update event.
	// If Cause is nil, this is the first update event in the lookup, caused by the seeding.
	Cause *PeerKadID
	// Source is the peer who informed us about the peer IDs in this update (below).
	Source *PeerKadID
	// Heard is a set of peers whose state in the lookup's peerset is being set to "heard".
	Heard []*PeerKadID
	// Waiting is a set of peers whose state in the lookup's peerset is being set to "waiting".
	Waiting []*PeerKadID
	// Queried is a set of peers whose state in the lookup's peerset is being set to "queried".
	Queried []*PeerKadID
	// Unreachable is a set of peers whose state in the lookup's peerset is being set to "unreachable".
	Unreachable []*PeerKadID

LookupUpdateEvent describes a lookup state update event.

func NewLookupUpdateEvent

func NewLookupUpdateEvent(
	cause peer.ID,
	source peer.ID,
	heard []peer.ID,
	waiting []peer.ID,
	queried []peer.ID,
	unreachable []peer.ID,
) *LookupUpdateEvent

NewLookupUpdateEvent creates a new lookup update event, automatically converting the passed peer IDs to peer Kad IDs.

type ModeOpt

type ModeOpt = dhtcfg.ModeOpt

ModeOpt describes what mode the dht should operate in

const (
	// ModeAuto utilizes EvtLocalReachabilityChanged events sent over the event bus to dynamically switch the DHT
	// between Client and Server modes based on network conditions
	ModeAuto ModeOpt = iota
	// ModeClient operates the DHT as a client only, it cannot respond to incoming queries
	// ModeServer operates the DHT as a server, it can both send and respond to queries
	// ModeAutoServer operates in the same way as ModeAuto, but acts as a server when reachability is unknown

type Option

type Option = dhtcfg.Option

func BootstrapPeers

func BootstrapPeers(bootstrappers ...peer.AddrInfo) Option

BootstrapPeers configures the bootstrapping nodes that we will connect to to seed and refresh our Routing Table if it becomes empty.

func BootstrapPeersFunc

func BootstrapPeersFunc(getBootstrapPeers func() []peer.AddrInfo) Option

BootstrapPeersFunc configures the function that returns the bootstrapping nodes that we will connect to to seed and refresh our Routing Table if it becomes empty.

func BucketSize

func BucketSize(bucketSize int) Option

BucketSize configures the bucket size (k in the Kademlia paper) of the routing table.

The default value is 20.

func Concurrency

func Concurrency(alpha int) Option

Concurrency configures the number of concurrent requests (alpha in the Kademlia paper) for a given query path.

The default value is 10.

func Datastore

func Datastore(ds ds.Batching) Option

Datastore configures the DHT to use the specified datastore.

Defaults to an in-memory (temporary) map.

func DisableAutoRefresh

func DisableAutoRefresh() Option

DisableAutoRefresh completely disables 'auto-refresh' on the DHT routing table. This means that we will neither refresh the routing table periodically nor when the routing table size goes below the minimum threshold.

func DisableProviders

func DisableProviders() Option

DisableProviders disables storing and retrieving provider records.

Defaults to enabled.

WARNING: do not change this unless you're using a forked DHT (i.e., a private network and/or distinct DHT protocols with the `Protocols` option).

func DisableValues

func DisableValues() Option

DisableValues disables storing and retrieving value records (including public keys).

Defaults to enabled.

WARNING: do not change this unless you're using a forked DHT (i.e., a private network and/or distinct DHT protocols with the `Protocols` option).

func MaxRecordAge

func MaxRecordAge(maxAge time.Duration) Option

MaxRecordAge specifies the maximum time that any node will hold onto a record ("PutValue record") from the time its received. This does not apply to any other forms of validity that the record may contain. For example, a record may contain an ipns entry with an EOL saying its valid until the year 2020 (a great time in the future). For that record to stick around it must be rebroadcasted more frequently than once every 'MaxRecordAge'

func Mode

func Mode(m ModeOpt) Option

Mode configures which mode the DHT operates in (Client, Server, Auto).

Defaults to ModeAuto.

func NamespacedValidator

func NamespacedValidator(ns string, v record.Validator) Option

NamespacedValidator adds a validator namespaced under `ns`. This option fails if the DHT is not using a `record.NamespacedValidator` as its validator (it uses one by default but this can be overridden with the `Validator` option). Adding a namespaced validator without changing the `Validator` will result in adding a new validator in addition to the default public key and IPNS validators. The "pk" and "ipns" namespaces cannot be overridden here unless a new `Validator` has been set first.

Example: Given a validator registered as `NamespacedValidator("ipns", myValidator)`, all records with keys starting with `/ipns/` will be validated with `myValidator`.

func ProtocolExtension

func ProtocolExtension(ext protocol.ID) Option

ProtocolExtension adds an application specific protocol to the DHT protocol. For example, /ipfs/lan/kad/1.0.0 instead of /ipfs/kad/1.0.0. extension should be of the form /lan.

func ProtocolPrefix

func ProtocolPrefix(prefix protocol.ID) Option

ProtocolPrefix sets an application specific prefix to be attached to all DHT protocols. For example, /myapp/kad/1.0.0 instead of /ipfs/kad/1.0.0. Prefix should be of the form /myapp.

Defaults to dht.DefaultPrefix

func ProvidersOptions

func ProvidersOptions(opts []providers.Option) Option

ProvidersOptions are options passed directly to the provider manager.

The provider manager adds and gets provider records from the datastore, cahing them in between. These options are passed to the provider manager allowing customisation of things like the GC interval and cache implementation.

func QueryFilter

func QueryFilter(filter QueryFilterFunc) Option

QueryFilter sets a function that approves which peers may be dialed in a query

func Resiliency

func Resiliency(beta int) Option

Resiliency configures the number of peers closest to a target that must have responded in order for a given query path to complete.

The default value is 3.

func RoutingTableFilter

func RoutingTableFilter(filter RouteTableFilterFunc) Option

RoutingTableFilter sets a function that approves which peers may be added to the routing table. The host should already have at least one connection to the peer under consideration.

func RoutingTableLatencyTolerance

func RoutingTableLatencyTolerance(latency time.Duration) Option

RoutingTableLatencyTolerance sets the maximum acceptable latency for peers in the routing table's cluster.

func RoutingTablePeerDiversityFilter

func RoutingTablePeerDiversityFilter(pg peerdiversity.PeerIPGroupFilter) Option

RoutingTablePeerDiversityFilter configures the implementation of the `PeerIPGroupFilter` that will be used to construct the diversity filter for the Routing Table. Please see the docs for `peerdiversity.PeerIPGroupFilter` AND `peerdiversity.Filter` for more details.

func RoutingTableRefreshPeriod

func RoutingTableRefreshPeriod(period time.Duration) Option

RoutingTableRefreshPeriod sets the period for refreshing buckets in the routing table. The DHT will refresh buckets every period by:

1. First searching for nearby peers to figure out how many buckets we should try to fill. 1. Then searching for a random key in each bucket that hasn't been queried in

the last refresh period.

func RoutingTableRefreshQueryTimeout

func RoutingTableRefreshQueryTimeout(timeout time.Duration) Option

RoutingTableRefreshQueryTimeout sets the timeout for routing table refresh queries.

func V1ProtocolOverride

func V1ProtocolOverride(proto protocol.ID) Option

V1ProtocolOverride overrides the protocolID used for /kad/1.0.0 with another. This is an advanced feature, and should only be used to handle legacy networks that have not been using protocolIDs of the form /app/kad/1.0.0.

This option will override and ignore the ProtocolPrefix and ProtocolExtension options

func Validator

func Validator(v record.Validator) Option

Validator configures the DHT to use the specified validator.

Defaults to a namespaced validator that can validate both public key (under the "pk" namespace) and IPNS records (under the "ipns" namespace). Setting the validator implies that the user wants to control the validators and therefore the default public key and IPNS validators will not be added.

type PeerKadID

type PeerKadID struct {
	Peer peer.ID
	Kad  kbucket.ID

PeerKadID contains a libp2p Peer ID and a binary Kademlia ID.

func NewPeerKadID

func NewPeerKadID(p peer.ID) *PeerKadID

NewPeerKadID creates a PeerKadID from a libp2p Peer ID.

func NewPeerKadIDSlice

func NewPeerKadIDSlice(p []peer.ID) []*PeerKadID

NewPeerKadIDSlice creates a slice of PeerKadID from the passed slice of libp2p Peer IDs.

func OptPeerKadID

func OptPeerKadID(p peer.ID) *PeerKadID

OptPeerKadID returns a pointer to a PeerKadID or nil if the passed Peer ID is it's default value.

type QueryFilterFunc

type QueryFilterFunc = dhtcfg.QueryFilterFunc

QueryFilterFunc is a filter applied when considering peers to dial when querying

type RouteTableFilterFunc

type RouteTableFilterFunc = dhtcfg.RouteTableFilterFunc

RouteTableFilterFunc is a filter applied when considering connections to keep in the local route table.


Path Synopsis
Package dual provides an implementaiton of a split or "dual" dht, where two parallel instances are maintained for the global internet and the local LAN respectively.
Package dual provides an implementaiton of a split or "dual" dht, where two parallel instances are maintained for the global internet and the local LAN respectively.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL