bitswap

README

go-bitswap

An implementation of the bitswap protocol in go!

Lead Maintainer

Dirk McCormick

Table of Contents

• Background
• Install
• Usage
• Implementation
• Contribute
• License

Background

Bitswap is the data trading module for ipfs. It manages requesting and sending blocks to and from other peers in the network. Bitswap has two main jobs:

• to acquire blocks requested by the client from the network
• to judiciously send blocks in its possession to other peers who want them

Bitswap is a message based protocol, as opposed to request-response. All messages contain wantlists or blocks.

A node sends a wantlist to tell peers which blocks it wants. When a node receives a wantlist it should check which blocks it has from the wantlist, and consider sending the matching blocks to the requestor.

When a node receives blocks that it asked for, the node should send out a notification called a 'Cancel' to tell its peers that the node no longer wants those blocks.

go-bitswap provides an implementation of the Bitswap protocol in go.

Learn more about how Bitswap works

Install

go-bitswap requires Go >= 1.11 and can be installed using Go modules

Usage

Initializing a Bitswap Exchange

import (
  "context"
  bitswap "github.com/ipfs/go-bitswap"
  bsnet "github.com/ipfs/go-graphsync/network"
  blockstore "github.com/ipfs/go-ipfs-blockstore"
  "github.com/libp2p/go-libp2p-core/routing"
	"github.com/libp2p/go-libp2p-core/host"
)

var ctx context.Context
var host host.Host
var router routing.ContentRouting
var bstore blockstore.Blockstore

network := bsnet.NewFromIPFSHost(host, router)
exchange := bitswap.New(ctx, network, bstore)

Parameter Notes:

1. ctx is just the parent context for all of Bitswap
2. network is a network abstraction provided to Bitswap on top of libp2p & content routing.
3. bstore is an IPFS blockstore

Get A Block Synchronously

var c cid.Cid
var ctx context.Context
var exchange bitswap.Bitswap

block, err := exchange.GetBlock(ctx, c)

Parameter Notes:

1. ctx is the context for this request, which can be cancelled to cancel the request
2. c is the content ID of the block you're requesting

Get Several Blocks Asynchronously

var cids []cid.Cid
var ctx context.Context
var exchange bitswap.Bitswap

blockChannel, err := exchange.GetBlocks(ctx, cids)

Parameter Notes:

1. ctx is the context for this request, which can be cancelled to cancel the request
2. cids is a slice of content IDs for the blocks you're requesting

Get Related Blocks Faster With Sessions

In IPFS, content blocks are often connected to each other through a MerkleDAG. If you know ahead of time that block requests are related, Bitswap can make several optimizations internally in how it requests those blocks in order to get them faster. Bitswap provides a mechanism called a Bitswap Session to manage a series of block requests as part of a single higher level operation. You should initialize a Bitswap Session any time you intend to make a series of block requests that are related -- and whose responses are likely to come from the same peers.

var ctx context.Context
var cids []cids.cid
var exchange bitswap.Bitswap

session := exchange.NewSession(ctx)
blocksChannel, err := session.GetBlocks(ctx, cids)
// later
var relatedCids []cids.cid
relatedBlocksChannel, err := session.GetBlocks(ctx, relatedCids)

Note that NewSession returns an interface with GetBlock and GetBlocks methods that have the same signature as the overall Bitswap exchange.

Tell bitswap a new block was added to the local datastore

var blk blocks.Block
var exchange bitswap.Bitswap

err := exchange.HasBlock(blk)

Contribute

PRs are welcome!

Small note: If editing the Readme, please conform to the standard-readme specification.

License

MIT © Juan Batiz-Benet

Documentation

Overview

Package bitswap implements the DMS3 exchange interface with the BitSwap bilateral exchange protocol.

Index

• Variables
• func New(parent context.Context, network bsnet.BitSwapNetwork, ...) exchange.Interface
• type Bitswap
  • func (bs *Bitswap) Close() error
  • func (bs *Bitswap) GetBlock(parent context.Context, k cid.Cid) (blocks.Block, error)
  • func (bs *Bitswap) GetBlocks(ctx context.Context, keys []cid.Cid) (<-chan blocks.Block, error)
  • func (bs *Bitswap) GetWantBlocks() []cid.Cid
  • func (bs *Bitswap) GetWantHaves() []cid.Cid
  • func (bs *Bitswap) GetWantlist() []cid.Cid
  • func (bs *Bitswap) HasBlock(blk blocks.Block) error
  • func (bs *Bitswap) IsOnline() bool
  • func (bs *Bitswap) LedgerForPeer(p peer.ID) *decision.Receipt
  • func (bs *Bitswap) NewSession(ctx context.Context) exchange.Fetcher
  • func (bs *Bitswap) PeerConnected(p peer.ID)
  • func (bs *Bitswap) PeerDisconnected(p peer.ID)
  • func (bs *Bitswap) ReceiveError(err error)
  • func (bs *Bitswap) ReceiveMessage(ctx context.Context, p peer.ID, incoming bsmsg.BitSwapMessage)
  • func (bs *Bitswap) Stat() (*Stat, error)
  • func (bs *Bitswap) WantlistForPeer(p peer.ID) []cid.Cid
• type Option
  • func DisableWireTap() Option
  • func EnableWireTap(tap WireTap) Option
  • func EngineBlockstoreWorkerCount(count int) Option
  • func ProvideEnabled(enabled bool) Option
  • func ProviderSearchDelay(newProvSearchDelay time.Duration) Option
  • func RebroadcastDelay(newRebroadcastDelay delay.D) Option
  • func SetSendDontHaves(send bool) Option
  • func WithScoreLedger(scoreLedger deciface.ScoreLedger) Option
• type Stat
• type WireTap

Variables

var (
	// HasBlockBufferSize is the buffer size of the channel for new blocks
	// that need to be provided. They should get pulled over by the
	// provideCollector even before they are actually provided.
	// TODO: Does this need to be this large givent that?
	HasBlockBufferSize = 256
)

var TaskWorkerCount = 8

TaskWorkerCount is the total number of simultaneous threads sending outgoing messages

Functions

func New

func New(parent context.Context, network bsnet.BitSwapNetwork,
	bstore blockstore.Blockstore, options ...Option) exchange.Interface

New initializes a BitSwap instance that communicates over the provided BitSwapNetwork. This function registers the returned instance as the network delegate. Runs until context is cancelled or bitswap.Close is called.

Types

type Bitswap

type Bitswap struct {
	// contains filtered or unexported fields
}

Bitswap instances implement the bitswap protocol.

func (*Bitswap) Close

func (bs *Bitswap) Close() error

Close is called to shutdown Bitswap

func (*Bitswap) GetBlock

func (bs *Bitswap) GetBlock(parent context.Context, k cid.Cid) (blocks.Block, error)

GetBlock attempts to retrieve a particular block from peers within the deadline enforced by the context.

func (*Bitswap) GetBlocks

func (bs *Bitswap) GetBlocks(ctx context.Context, keys []cid.Cid) (<-chan blocks.Block, error)

GetBlocks returns a channel where the caller may receive blocks that correspond to the provided |keys|. Returns an error if BitSwap is unable to begin this request within the deadline enforced by the context.

NB: Your request remains open until the context expires. To conserve resources, provide a context with a reasonably short deadline (ie. not one that lasts throughout the lifetime of the server)

func (*Bitswap) GetWantBlocks

func (bs *Bitswap) GetWantBlocks() []cid.Cid

GetWantBlocks returns the current list of want-blocks.

func (*Bitswap) GetWantHaves

func (bs *Bitswap) GetWantHaves() []cid.Cid

GetWanthaves returns the current list of want-haves.

func (*Bitswap) GetWantlist

func (bs *Bitswap) GetWantlist() []cid.Cid

GetWantlist returns the current local wantlist (both want-blocks and want-haves).

func (*Bitswap) HasBlock

func (bs *Bitswap) HasBlock(blk blocks.Block) error

HasBlock announces the existence of a block to this bitswap service. The service will potentially notify its peers.

func (*Bitswap) IsOnline

func (bs *Bitswap) IsOnline() bool

IsOnline is needed to match go-dms3-exchange-interface

func (*Bitswap) LedgerForPeer

func (bs *Bitswap) LedgerForPeer(p peer.ID) *decision.Receipt

LedgerForPeer returns aggregated data about blocks swapped and communication with a given peer.

func (*Bitswap) NewSession

func (bs *Bitswap) NewSession(ctx context.Context) exchange.Fetcher

NewSession generates a new Bitswap session. You should use this, rather that calling Bitswap.GetBlocks, any time you intend to do several related block requests in a row. The session returned will have it's own GetBlocks method, but the session will use the fact that the requests are related to be more efficient in its requests to peers. If you are using a session from go-blockservice, it will create a bitswap session automatically.

func (*Bitswap) PeerConnected

func (bs *Bitswap) PeerConnected(p peer.ID)

PeerConnected is called by the network interface when a peer initiates a new connection to bitswap.

func (*Bitswap) PeerDisconnected

func (bs *Bitswap) PeerDisconnected(p peer.ID)

PeerDisconnected is called by the network interface when a peer closes a connection

func (*Bitswap) ReceiveError

func (bs *Bitswap) ReceiveError(err error)

ReceiveError is called by the network interface when an error happens at the network layer. Currently just logs error.

func (*Bitswap) ReceiveMessage

func (bs *Bitswap) ReceiveMessage(ctx context.Context, p peer.ID, incoming bsmsg.BitSwapMessage)

ReceiveMessage is called by the network interface when a new message is received.

func (*Bitswap) Stat

func (bs *Bitswap) Stat() (*Stat, error)

Stat returns aggregated statistics about bitswap operations

func (*Bitswap) WantlistForPeer

func (bs *Bitswap) WantlistForPeer(p peer.ID) []cid.Cid

WantlistForPeer returns the currently understood list of blocks requested by a given peer.

type Option

type Option func(*Bitswap)

Option defines the functional option type that can be used to configure bitswap instances

func DisableWireTap

func DisableWireTap() Option

Configures Bitswap not to use any wiretap.

func EnableWireTap

func EnableWireTap(tap WireTap) Option

Configures Bitswap to use given wiretap.

func EngineBlockstoreWorkerCount

func EngineBlockstoreWorkerCount(count int) Option

EngineBlockstoreWorkerCount sets the number of worker threads used for blockstore operations in the decision engine

func ProvideEnabled

func ProvideEnabled(enabled bool) Option

ProvideEnabled is an option for enabling/disabling provide announcements

func ProviderSearchDelay

func ProviderSearchDelay(newProvSearchDelay time.Duration) Option

ProviderSearchDelay overwrites the global provider search delay

func RebroadcastDelay

func RebroadcastDelay(newRebroadcastDelay delay.D) Option

RebroadcastDelay overwrites the global provider rebroadcast delay

func SetSendDontHaves

func SetSendDontHaves(send bool) Option

SetSendDontHaves indicates what to do when the engine receives a want-block for a block that is not in the blockstore. Either - Send a DONT_HAVE message - Simply don't respond This option is only used for testing.

func WithScoreLedger

func WithScoreLedger(scoreLedger deciface.ScoreLedger) Option

Configures the engine to use the given score decision logic.

type Stat

type Stat struct {
	ProvideBufLen    int
	Wantlist         []cid.Cid
	Peers            []string
	BlocksReceived   uint64
	DataReceived     uint64
	BlocksSent       uint64
	DataSent         uint64
	DupBlksReceived  uint64
	DupDataReceived  uint64
	MessagesReceived uint64
}

Stat is a struct that provides various statistics on bitswap operations

type WireTap

type WireTap interface {
	MessageReceived(peer.ID, bsmsg.BitSwapMessage)
	MessageSent(peer.ID, bsmsg.BitSwapMessage)
}

WireTap provides methods to access all messages sent and received by Bitswap. This interface can be used to implement various statistics (this is original intent).