legs

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 16, 2021 License: Apache-2.0, MIT Imports: 39 Imported by: 12

README

legs 🦵

Legs is a simplified interface for go-data-transfer, providing a 1:1 mechanism for maintaining a synchronized IPLD dag of data between a publisher and subscriber.

Usage

Creating a legs publisher is as simple as creating a LegTransport and starting a publisher. LegTransport handles all the transport machinery to enable the instantiation of several publishers and subscribers in the same libp2p host:

t, err := legs.MakeLegTransport(ctx, host, host, lsys, "legs/topic")
if err != nil {
	panic(err)
}
publisher, err := legs.Publish(ctx, t)
...
// Publish updated root.
if err := publisher.UpdateRoot(ctx, lnk.(cidlink.Link).Cid); err != nil {
	panic(err)
}

A subscriber to the same "legs/topic" would be created similarly

t, err := legs.MakeLegTransport(ctx, host, host, lsys, "legs/topic")
if err != nil {
	panic(err)
}
subscriber, err := legs.NewSubscriber(ctx, t, policy)

and can watch to changes via:

subscriber, err := legs.Subscribe(ctx, dataStore, lp2pHost, "legs/topic")
subscriptionWatcher, cncl := subscriber.OnChange()
go watch(subscriptionWatcher)

func watch(notifications chan cid.Cid) {
    for {
        newHead := <-notifications
        // newHead is now available in the local dataStore
    }
}

Subscribers can be created with a PolicyHandler. This policy is used to filter the exchanges and updates a subscriber will process. Subscribers started with FilterPeerPolicy, for instance, will only process (and thus exchange) updates from the specific peer specified in the policy.

Subscribers keep track of the latest head they've already synced to prevent from exchanging all the DAG from scratch in every update, downloading exclusively the part they're missing. This value is not persisted as part of the library. If you want to start a subscriber which has already partially synced with a provider you can use:

subscriber, err := legs.NewSubscriberPartiallySynced(ctx, t, policy, latestSync)

License

Legs is dual-licensed under Apache 2.0 and MIT terms:

Apache License, Version 2.0, (LICENSE or http://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ExploreRecursiveWithStop

func ExploreRecursiveWithStop(limit selector.RecursionLimit, sequence selectorbuilder.SelectorSpec, stopLnk ipld.Link) ipld.Node

ExploreRecursiveWithStop builds a selector that recursively syncs a DAG until the link stopLnk is seen. It prevents from having to sync DAGs from scratch with every update.

func ExploreRecursiveWithStopNode

func ExploreRecursiveWithStopNode(limit selector.RecursionLimit, sequence ipld.Node, stopLnk ipld.Link) ipld.Node

ExploreRecursiveWithStopNode builds a selector that recursively syncs a DAG until the link stopLnk is seen. It prevents from having to sync DAGs from scratch with every update.

func LegSelector

func LegSelector(stopLnk ipld.Link) ipld.Node

LegSelector is a convenient function that returns the selector used by leg subscribers

LegSelector is a "recurse all" selector that provides conditions to stop the traversal at a specific link (stopAt).

Types

type LegMultiSubscriber

type LegMultiSubscriber interface {
	NewSubscriber(PolicyHandler) (LegSubscriber, error)
	NewSubscriberPartiallySynced(PolicyHandler, cid.Cid) (LegSubscriber, error)
	GraphSync() graphsync.GraphExchange
	DataTransfer() dt.Manager
	Close(ctx context.Context) error
}

LegMultiSubscriber allows you to setup multiple subscriptions over single graphsync/datatransfer instance with difference policy filters

func NewMultiSubscriber

func NewMultiSubscriber(ctx context.Context, host host.Host, ds datastore.Batching, lsys ipld.LinkSystem, topic string, dss ipld.Node) (LegMultiSubscriber, error)

NewMultiSubscriber sets up a new instance of a multi subscriber.

A default selector sequence, dss, may optionally be specified. The selector sequence is used during sync traversal to define the extent by which a node is explored. If unspecified, all edges of nodes are recursively explored.

Note that the default selector sequence is wrapped with a selector logic that will stop the traversal when the latest synced link is reached. Therefore, it must only specify the selection sequence itself.

See: ExploreRecursiveWithStopNode.

type LegPublisher

type LegPublisher interface {
	// Publishes and update for the DAG in the pubsub channel.
	UpdateRoot(context.Context, cid.Cid) error
	// Publishes and update for the DAG in the pubsub channel using custom multiaddrs.
	UpdateRootWithAddrs(context.Context, cid.Cid, []ma.Multiaddr) error
	// Close publisher
	Close() error
}

LegPublisher is an interface for updating the published dag.

func NewPublisher

func NewPublisher(ctx context.Context,
	host host.Host,
	ds datastore.Batching,
	lsys ipld.LinkSystem,
	topic string) (LegPublisher, error)

NewPublisher creates a new legs publisher

func NewPublisherFromExisting

func NewPublisherFromExisting(ctx context.Context,
	dt dt.Manager,
	host host.Host,
	topic string,
	lsys ipld.LinkSystem) (LegPublisher, error)

NewPublisherFromExisting instantiates go-legs publishing on an existing data transfer instance

type LegSubscriber

type LegSubscriber interface {
	// OnChange returns a listener and cancel func for subscriber.  When cancel
	// is called the listener channel is closed.
	OnChange() (chan cid.Cid, context.CancelFunc)
	// SetPolicyHandler triggered to know if an exchange needs to be made.
	SetPolicyHandler(PolicyHandler) error
	// SetLatestSync updates the latest sync of a subcriber to account for
	// out of band updates.
	SetLatestSync(c cid.Cid) error
	// Sync to a specific Cid of a peer's DAG without having to wait for a
	// publication.
	// Returns a channel that will resolve with the same cid, c that was passed as input when
	// the dag from 'c' is available, a function to cancel, or a synchronous error if
	// the subscriber is not in a state where it can sync from the specified peer.
	Sync(ctx context.Context, p peer.ID, c cid.Cid, selector ipld.Node) (<-chan cid.Cid, context.CancelFunc, error)
	// Close subscriber
	Close() error
}

LegSubscriber is an interface for watching a published dag.

func NewSubscriber

func NewSubscriber(ctx context.Context,
	host host.Host,
	ds datastore.Batching,
	lsys ipld.LinkSystem,
	topic string,
	selector ipld.Node) (LegSubscriber, error)

NewSubscriber creates a new leg subscriber listening to a specific pubsub topic with a specific policyHandle to determine when to perform exchanges

func NewSubscriberPartiallySynced

func NewSubscriberPartiallySynced(
	ctx context.Context,
	host host.Host,
	ds datastore.Batching,
	lsys ipld.LinkSystem,
	topic string,
	latestSync cid.Cid,
	selector ipld.Node) (LegSubscriber, error)

NewSubscriberPartiallySynced creates a new leg subscriber with a specific latestSync. LegSubscribers don't persist their latestSync. With this, users are able to start a new subscriber with knowledge about what has already been synced.

type PolicyHandler

type PolicyHandler func(*pubsub.Message) (bool, error)

PolicyHandler make some preliminary checks before running the exchange

func FilterPeerPolicy

func FilterPeerPolicy(p peer.ID) PolicyHandler

FilterPeerPolicy is a sample policy that only triggers exchanges if the update is generated from a specific peer.

type Voucher

type Voucher struct {
	Head *cid.Cid
}

A Voucher is used to communicate a new DAG head

func (*Voucher) MarshalCBOR

func (t *Voucher) MarshalCBOR(w io.Writer) error

func (*Voucher) Type

Type provides an identifier for the voucher to go-data-transfer

func (*Voucher) UnmarshalCBOR

func (t *Voucher) UnmarshalCBOR(r io.Reader) error

type VoucherResult

type VoucherResult struct {
	Code uint64
}

A VoucherResult responds to a voucher

func (*VoucherResult) MarshalCBOR

func (t *VoucherResult) MarshalCBOR(w io.Writer) error

func (*VoucherResult) Type

Type provides an identifier for the voucher result to go-data-transfer

func (*VoucherResult) UnmarshalCBOR

func (t *VoucherResult) UnmarshalCBOR(r io.Reader) error

Directories

Path Synopsis
p2p

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL