retrievalimpl

package
v1.28.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 14, 2023 License: Apache-2.0, MIT Imports: 39 Imported by: 9

Documentation

Overview

Package retrievalimpl provides the primary implementation of retrieval market top level interfaces interfaces

This package provides a production implementation of `RetrievalClient` and `RetrievalProvider`.

Index

Constants

View Source 
const MaxIdentityCIDBytes = 2 << 10

MaxIdentityCIDBytes is the largest identity CID as a PayloadCID that we are willing to decode

View Source 
const MaxIdentityCIDLinks = 32

MaxIdentityCIDLinks is the maximum number of links contained within an identity CID that we are willing to check for matching pieces

Variables

View Source 
var ClientFSMParameterSpec = fsm.Parameters{
	Environment:     &clientDealEnvironment{},
	StateType:       retrievalmarket.ClientDealState{},
	StateKeyField:   "Status",
	Events:          clientstates.ClientEvents,
	StateEntryFuncs: clientstates.ClientStateEntryFuncs,
}

ClientFSMParameterSpec is a valid set of parameters for a client deal FSM - used in doc generation

View Source 
var DefaultPricingFunc = func(VerifiedDealsFreeTransfer bool) func(ctx context.Context, pricingInput retrievalmarket.PricingInput) (retrievalmarket.Ask, error) {
	return func(ctx context.Context, pricingInput retrievalmarket.PricingInput) (retrievalmarket.Ask, error) {
		ask := pricingInput.CurrentAsk

		if pricingInput.Unsealed {
			ask.UnsealPrice = big.Zero()
		}

		if pricingInput.VerifiedDeal && VerifiedDealsFreeTransfer {
			ask.PricePerByte = big.Zero()
		}

		return ask, nil
	}
}

DefaultPricingFunc is the default pricing policy that will be used to price retrieval deals.

View Source 
var ProviderFSMParameterSpec = fsm.Parameters{
	Environment:     &providerDealEnvironment{},
	StateType:       retrievalmarket.ProviderDealState{},
	StateKeyField:   "Status",
	Events:          providerstates.ProviderEvents,
	StateEntryFuncs: providerstates.ProviderStateEntryFuncs,
	Options: fsm.Options{
		ConsumeAllEventsBeforeEntryFuncs: true,
	},
}

ProviderFSMParameterSpec is a valid set of parameters for a provider FSM - used in doc generation

Functions

func NewClient 

func NewClient(
	network rmnet.RetrievalMarketNetwork,
	dataTransfer datatransfer.Manager,
	node retrievalmarket.RetrievalClientNode,
	resolver discovery.PeerResolver,
	ds datastore.Batching,
	ba retrievalmarket.BlockstoreAccessor,
) (retrievalmarket.RetrievalClient, error)

NewClient creates a new retrieval client

func NewProvider 

func NewProvider(minerAddress address.Address,
	node retrievalmarket.RetrievalProviderNode,
	sa retrievalmarket.SectorAccessor,
	network rmnet.RetrievalMarketNetwork,
	pieceStore piecestore.PieceStore,
	dagStore stores.DAGStoreWrapper,
	dataTransfer datatransfer.Manager,
	ds datastore.Batching,
	retrievalPricingFunc RetrievalPricingFunc,
	opts ...RetrievalProviderOption,
) (retrievalmarket.RetrievalProvider, error)

NewProvider returns a new retrieval Provider

Types

type Client added in v0.3.2 

type Client struct {
	// contains filtered or unexported fields
}

Client is the production implementation of the RetrievalClient interface

func (*Client) CancelDeal added in v0.3.2 

func (c *Client) CancelDeal(dealID retrievalmarket.DealID) error

CancelDeal attempts to cancel an in progress deal

func (*Client) FindProviders added in v0.3.2 

func (c *Client) FindProviders(payloadCID cid.Cid) []retrievalmarket.RetrievalPeer

FindProviders uses PeerResolver interface to locate a list of providers who may have a given payload CID.

func (*Client) GetDeal added in v0.6.0 

func (c *Client) GetDeal(dealID retrievalmarket.DealID) (retrievalmarket.ClientDealState, error)

GetDeal returns a given deal by deal ID, if it exists

func (*Client) ListDeals added in v0.3.2 

func (c *Client) ListDeals() (map[retrievalmarket.DealID]retrievalmarket.ClientDealState, error)

ListDeals lists all known retrieval deals

func (*Client) NextID added in v1.8.0 

func (c *Client) NextID() retrievalmarket.DealID

func (*Client) OnReady added in v0.7.0 

func (c *Client) OnReady(ready shared.ReadyFunc)

OnReady registers a listener for when the client has finished starting up

func (*Client) Query added in v0.3.2 

func (c *Client) Query(ctx context.Context, p retrievalmarket.RetrievalPeer, payloadCID cid.Cid, params retrievalmarket.QueryParams) (retrievalmarket.QueryResponse, error)

Query sends a retrieval query to a specific retrieval provider, to determine if the provider can serve a retrieval request and what its specific parameters for the request are.

The client creates a new `RetrievalQueryStream` for the chosen peer ID, and calls `WriteQuery` on it, which constructs a data-transfer message and writes it to the Query stream.

func (*Client) Retrieve added in v0.3.2 

func (c *Client) Retrieve(
	ctx context.Context,
	id retrievalmarket.DealID,
	payloadCID cid.Cid,
	params retrievalmarket.Params,
	totalFunds abi.TokenAmount,
	p retrievalmarket.RetrievalPeer,
	clientWallet address.Address,
	minerWallet address.Address,
) (retrievalmarket.DealID, error)

Retrieve initiates the retrieval deal flow, which involves multiple requests and responses

To start this processes, the client creates a new `RetrievalDealStream`. Currently, this connection is kept open through the entire deal until completion or failure. Make deals pauseable as well as surviving a restart is a planned future feature.

Retrieve should be called after using FindProviders and Query are used to identify an appropriate provider to retrieve the deal from. The parameters identified in Query should be passed to Retrieve to ensure the greatest likelihood the provider will accept the deal

When called, the client takes the following actions:

1. Creates a deal ID using the next value from its `storedCounter`.

2. Constructs a `DealProposal` with deal terms

3. Tells its statemachine to begin tracking this deal state by dealID.

4. Constructs a `blockio.SelectorVerifier` and adds it to its dealID-keyed map of block verifiers.

5. Triggers a `ClientEventOpen` event on its statemachine.

From then on, the statemachine controls the deal flow in the client. Other components may listen for events in this flow by calling `SubscribeToEvents` on the Client. The Client handles consuming blocks it receives from the provider, via `ConsumeBlocks` function

Retrieve can use an ID generated through NextID, or can generate an ID if the user passes a zero value.

Use NextID when it's necessary to reserve an ID ahead of time, e.g. to associate it with a given blockstore in the BlockstoreAccessor.

Documentation of the client state machine can be found at https://godoc.org/github.com/filecoin-project/go-fil-markets/retrievalmarket/impl/clientstates

func (*Client) Start added in v0.7.0 

func (c *Client) Start(ctx context.Context) error

Start initialized the Client, performing relevant database migrations

func (*Client) SubscribeToEvents added in v0.3.2 

func (c *Client) SubscribeToEvents(subscriber retrievalmarket.ClientSubscriber) retrievalmarket.Unsubscribe

SubscribeToEvents allows another component to listen for events on the RetrievalClient in order to track deals as they progress through the deal flow

func (*Client) TryRestartInsufficientFunds added in v0.6.0 

func (c *Client) TryRestartInsufficientFunds(paymentChannel address.Address) error

TryRestartInsufficientFunds attempts to restart any deals stuck in the insufficient funds state after funds are added to a given payment channel

type DealDecider added in v0.3.0 

type DealDecider func(ctx context.Context, state retrievalmarket.ProviderDealState) (bool, string, error)

DealDecider is a function that makes a decision about whether to accept a deal

type Provider added in v0.3.0 

type Provider struct {
	// contains filtered or unexported fields
}

Provider is the production implementation of the RetrievalProvider interface

func (*Provider) Configure added in v0.3.0 

func (p *Provider) Configure(opts ...RetrievalProviderOption)

Configure reconfigures a provider after initialization

func (*Provider) GetAsk added in v0.5.3 

func (p *Provider) GetAsk() *retrievalmarket.Ask

GetAsk returns the current deal parameters this provider accepts

func (*Provider) GetDynamicAsk added in v1.5.0 

func (p *Provider) GetDynamicAsk(ctx context.Context, input retrievalmarket.PricingInput, storageDeals []abi.DealID) (retrievalmarket.Ask, error)

GetDynamicAsk quotes a dynamic price for the retrieval deal by calling the user configured dynamic pricing function. It passes the static price parameters set in the Ask Store to the pricing function.

func (*Provider) HandleQueryStream added in v0.3.0 

func (p *Provider) HandleQueryStream(stream rmnet.RetrievalQueryStream)

HandleQueryStream is called by the network implementation whenever a new message is received on the query protocol

A Provider handling a retrieval `Query` does the following:

1. Get the node's chain head in order to get its miner worker address.

2. Look in its piece store to determine if it can serve the given payload CID.

3. Combine these results with its existing parameters for retrieval deals to construct a `retrievalmarket.QueryResponse` struct.

4. Writes this response to the `Query` stream.

The connection is kept open only as long as the query-response exchange.

func (*Provider) ListDeals added in v0.3.0 

func (p *Provider) ListDeals() map[retrievalmarket.ProviderDealIdentifier]retrievalmarket.ProviderDealState

ListDeals lists all known retrieval deals

func (*Provider) OnReady added in v0.7.0 

func (p *Provider) OnReady(ready shared.ReadyFunc)

OnReady registers a listener for when the provider has finished starting up

func (*Provider) SetAsk added in v0.5.3 

func (p *Provider) SetAsk(ask *retrievalmarket.Ask)

SetAsk sets the deal parameters this provider accepts

func (*Provider) Start added in v0.3.0 

func (p *Provider) Start(ctx context.Context) error

Start begins listening for deals on the given host. Start must be called in order to accept incoming deals.

func (*Provider) Stop added in v0.3.0 

func (p *Provider) Stop() error

Stop stops handling incoming requests.

func (*Provider) SubscribeToEvents added in v0.3.0 

func (p *Provider) SubscribeToEvents(subscriber retrievalmarket.ProviderSubscriber) retrievalmarket.Unsubscribe

SubscribeToEvents listens for events that happen related to client retrievals

func (*Provider) SubscribeToQueryEvents added in v1.25.1 

func (p *Provider) SubscribeToQueryEvents(subscriber retrievalmarket.ProviderQueryEventSubscriber) retrievalmarket.Unsubscribe

SubscribeToQueryEvents subscribes to an event that is fired when a message is received on the query protocol

func (*Provider) SubscribeToValidationEvents added in v1.25.1 

func (p *Provider) SubscribeToValidationEvents(subscriber retrievalmarket.ProviderValidationSubscriber) retrievalmarket.Unsubscribe

SubscribeToValidationEvents subscribes to an event that is fired when the provider validates a request for data

type RetrievalPricingFunc added in v1.5.0 

type RetrievalPricingFunc func(ctx context.Context, dealPricingParams retrievalmarket.PricingInput) (retrievalmarket.Ask, error)

type RetrievalProviderOption added in v0.3.0 

type RetrievalProviderOption func(p *Provider)

RetrievalProviderOption is a function that configures a retrieval provider

func DealDeciderOpt added in v0.3.0 

func DealDeciderOpt(dd DealDecider) RetrievalProviderOption

DealDeciderOpt sets a custom protocol

Source Files

View all Source files

Directories

Path Synopsis
Package clientstates contains state machine logic relating to the `RetrievalClient`.
 Package clientstates contains state machine logic relating to the `RetrievalClient`.
Package dtutils provides event listeners for the client and provider to listen for events on the data transfer module and dispatch FSM events based on them
 Package dtutils provides event listeners for the client and provider to listen for events on the data transfer module and dispatch FSM events based on them
Package providerstates contains state machine logic relating to the `RetrievalProvider`.
 Package providerstates contains state machine logic relating to the `RetrievalProvider`.
Package testnodes contains stubbed implementations of the RetrievalProviderNode and RetrievalClientNode interface to simulate communications with a filecoin node
 Package testnodes contains stubbed implementations of the RetrievalProviderNode and RetrievalClientNode interface to simulate communications with a filecoin node

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.