bitswap

README

go-bitswap

An implementation of the bitswap protocol in go!

Table of Contents

• Install
• Protocol
• Implementation
• Contribute
• License

Protocol

Bitswap is the data trading module for ipfs, it manages requesting and sending blocks to and from other peers in the network. Bitswap has two main jobs, the first is to acquire blocks requested by the client from the network. The second is to judiciously send blocks in its possession to other peers who want them.

Bitswap is a message based protocol, as opposed to response-reply. All messages contain wantlists, or blocks. Upon receiving a wantlist, a node should consider sending out wanted blocks if they have them. Upon receiving blocks, the node should send out a notification called a 'Cancel' signifying that they no longer want the block. At a protocol level, bitswap is very simple.

Implementation

Internally, when a message with a wantlist is received, it is sent to the decision engine to be considered, and blocks that we have that are wanted are placed into the peer request queue. Any block we possess that is wanted by another peer has a task in the peer request queue created for it. The peer request queue is a priority queue that sorts available tasks by some metric, currently, that metric is very simple and aims to fairly address the tasks of each other peer. More advanced decision logic will be implemented in the future. Task workers pull tasks to be done off of the queue, retrieve the block to be sent, and send it off. The number of task workers is limited by a constant factor.

Client requests for new blocks are handled by the want manager, for every new block (or set of blocks) wanted, the 'WantBlocks' method is invoked. The want manager then ensures that connected peers are notified of the new block that we want by sending the new entries to a message queue for each peer. The message queue will loop while there is work available and do the following: 1) Ensure it has a connection to its peer, 2) grab the message to be sent, and 3) send it. If new messages are added while the loop is in steps 1 or 3, the messages are combined into one to avoid having to keep an actual queue and send multiple messages. The same process occurs when the client receives a block and sends a cancel message for it.

Contribute

PRs are welcome!

Small note: If editing the Readme, please conform to the standard-readme specification.

License

MIT © Juan Batiz-Benet

Documentation

Overview

Package bitswap implements the IPFS exchange interface with the BitSwap bilateral exchange protocol.

Index

• Variables
• func New(parent context.Context, network bsnet.BitSwapNetwork, ...) exchange.Interface
• type Bitswap
  • func (bs *Bitswap) Close() error
  • func (bs *Bitswap) GetBlock(parent context.Context, k cid.Cid) (blocks.Block, error)
  • func (bs *Bitswap) GetBlocks(ctx context.Context, keys []cid.Cid) (<-chan blocks.Block, error)
  • func (bs *Bitswap) GetWantlist() []cid.Cid
  • func (bs *Bitswap) HasBlock(blk blocks.Block) error
  • func (bs *Bitswap) IsOnline() bool
  • func (bs *Bitswap) LedgerForPeer(p peer.ID) *decision.Receipt
  • func (bs *Bitswap) NewSession(ctx context.Context) exchange.Fetcher
  • func (bs *Bitswap) PeerConnected(p peer.ID)
  • func (bs *Bitswap) PeerDisconnected(p peer.ID)
  • func (bs *Bitswap) ReceiveError(err error)
  • func (bs *Bitswap) ReceiveMessage(ctx context.Context, p peer.ID, incoming bsmsg.BitSwapMessage)
  • func (bs *Bitswap) Stat() (*Stat, error)
  • func (bs *Bitswap) WantlistForPeer(p peer.ID) []cid.Cid
• type Option
  • func ProvideEnabled(enabled bool) Option
• type Stat

Variables

var (
	// HasBlockBufferSize is the buffer size of the channel for new blocks
	// that need to be provided. They should get pulled over by the
	// provideCollector even before they are actually provided.
	// TODO: Does this need to be this large givent that?
	HasBlockBufferSize = 256
)

var TaskWorkerCount = 8

TaskWorkerCount is the total number of simultaneous threads sending outgoing messages

Functions

func New

func New(parent context.Context, network bsnet.BitSwapNetwork,
	bstore blockstore.Blockstore, options ...Option) exchange.Interface

New initializes a BitSwap instance that communicates over the provided BitSwapNetwork. This function registers the returned instance as the network delegate. Runs until context is cancelled or bitswap.Close is called.

Types

type Bitswap

type Bitswap struct {
	// contains filtered or unexported fields
}

Bitswap instances implement the bitswap protocol.

func (*Bitswap) Close

func (bs *Bitswap) Close() error

Close is called to shutdown Bitswap

func (*Bitswap) GetBlock

func (bs *Bitswap) GetBlock(parent context.Context, k cid.Cid) (blocks.Block, error)

GetBlock attempts to retrieve a particular block from peers within the deadline enforced by the context.

func (*Bitswap) GetBlocks

func (bs *Bitswap) GetBlocks(ctx context.Context, keys []cid.Cid) (<-chan blocks.Block, error)

GetBlocks returns a channel where the caller may receive blocks that correspond to the provided |keys|. Returns an error if BitSwap is unable to begin this request within the deadline enforced by the context.

NB: Your request remains open until the context expires. To conserve resources, provide a context with a reasonably short deadline (ie. not one that lasts throughout the lifetime of the server)

func (*Bitswap) GetWantlist

func (bs *Bitswap) GetWantlist() []cid.Cid

GetWantlist returns the current local wantlist.

func (*Bitswap) HasBlock

func (bs *Bitswap) HasBlock(blk blocks.Block) error

HasBlock announces the existence of a block to this bitswap service. The service will potentially notify its peers.

func (*Bitswap) IsOnline

func (bs *Bitswap) IsOnline() bool

IsOnline is needed to match go-ipfs-exchange-interface

func (*Bitswap) LedgerForPeer

func (bs *Bitswap) LedgerForPeer(p peer.ID) *decision.Receipt

LedgerForPeer returns aggregated data about blocks swapped and communication with a given peer.

func (*Bitswap) NewSession

func (bs *Bitswap) NewSession(ctx context.Context) exchange.Fetcher

NewSession generates a new Bitswap session. You should use this, rather that calling Bitswap.GetBlocks, any time you intend to do several related block requests in a row. The session returned will have it's own GetBlocks method, but the session will use the fact that the requests are related to be more efficient in its requests to peers. If you are using a session from go-blockservice, it will create a bitswap session automatically.

func (*Bitswap) PeerConnected

func (bs *Bitswap) PeerConnected(p peer.ID)

PeerConnected is called by the network interface when a peer initiates a new connection to bitswap.

func (*Bitswap) PeerDisconnected

func (bs *Bitswap) PeerDisconnected(p peer.ID)

PeerDisconnected is called by the network interface when a peer closes a connection

func (*Bitswap) ReceiveError

func (bs *Bitswap) ReceiveError(err error)

ReceiveError is called by the network interface when an error happens at the network layer. Currently just logs error.

func (*Bitswap) ReceiveMessage

func (bs *Bitswap) ReceiveMessage(ctx context.Context, p peer.ID, incoming bsmsg.BitSwapMessage)

ReceiveMessage is called by the network interface when a new message is received.

func (*Bitswap) Stat

func (bs *Bitswap) Stat() (*Stat, error)

Stat returns aggregated statistics about bitswap operations

func (*Bitswap) WantlistForPeer

func (bs *Bitswap) WantlistForPeer(p peer.ID) []cid.Cid

WantlistForPeer returns the currently understood list of blocks requested by a given peer.

type Option

type Option func(*Bitswap)

Option defines the functional option type that can be used to configure bitswap instances

func ProvideEnabled

func ProvideEnabled(enabled bool) Option

ProvideEnabled is an option for enabling/disabling provide announcements

type Stat

type Stat struct {
	ProvideBufLen    int
	Wantlist         []cid.Cid
	Peers            []string
	BlocksReceived   uint64
	DataReceived     uint64
	BlocksSent       uint64
	DataSent         uint64
	DupBlksReceived  uint64
	DupDataReceived  uint64
	MessagesReceived uint64
}

Stat is a struct that provides various statistics on bitswap operations