fanout

Documentation

Overview

Package fanout is useful for invoking RPCs across a bunch of servers. It knows how to collect items together into fewer requests, how to retry failed requests, and how to hedge requests to mask slow servers.

Index

• func Call(ctx context.Context, points []space.Point, views Views, rpc RPC, ...) error
• func Partition(overallRange space.Range, views Views) space.PartitionedRange
• type Chunk
• type RPC
• type Result
• type View
• type Views
  • func NewViews(v []View) Views

Functions

func Call

func Call(
	ctx context.Context,
	points []space.Point,
	views Views,
	rpc RPC,
	results chan<- Chunk,
) error

Call sends out RPCs for a particular set of items.

'points' describe where each item fits in the key-space/hash-space. 'views' is a set of servers in the same key-space/hash-space. Note that Call() may modify the 'views' slice. 'rpc' will be called many times to fetch results.

RPC results are sent to 'results' in no particular order. This channel is closed before this function returns. The caller may apply backpressure by reading slowly from this channel. Note that the caller must be prepared to receive duplicate results for some items; this can happen when servers are slow to respond or return errors after producing some results.

Call returns an error if it could not successfully complete an RPC for every item. It will not return until all invoked RPCs have returned.

func Partition

func Partition(overallRange space.Range, views Views) space.PartitionedRange

Partition splits the given range into subranges, where each view serves one or more complete subranges. If there are insufficient views, some subranges will be served by no views.

This function is useful when you need to gather results covering an entire range. To do so, split up the range using Partition, then pass the resulting PartitionedRange's StartPoints into Call. Call will collect results for each of the subranges. TODO: an example would help here.

Types

type Chunk

type Chunk struct {
	// Same as passed to the RPC.
	View View
	// Same as passed to the RPC.
	Offsets []int
	// A result produced by the RPC.
	Result Result
}

Chunk contains a single result from an RPC. Streaming RPCs may return multiple results, corresponding to multiple chunks.

type RPC

type RPC func(ctx context.Context, view View, offsets []int, results func(Result)) error

RPC synchronously invokes an RPC for a set of items.

'offsets' identify which items to include in the request. They correspond to the indices of the 'points' slice given to Call(). The function must not modify offsets.

Any responses should be returned by invoking the results callback. It may be invoked multiple times to support streaming RPCs where one request can have many responses. Note that the callback may block to apply backpressure.

type Result

type Result interface{}

A Result is a reply from an RPC. The caller will need to cast this appropriately.

type View

type View interface {
	// Serves returns what is hosted on the server.
	Serves() space.Range
}

A View is a server that hosts some continuous range of a space.

type Views

type Views interface {
	// Len returns the number of Views in the list
	Len() int
	// View returns the View at supplied index into the list.
	// the Index -> View relationship should be stable for the lifetime
	// of the Views instance
	View(index int) View
}

Views is a list of View, its abstracted so that callers don't have to keep converting their types into []View

func NewViews

func NewViews(v []View) Views

NewViews will return an instance of the Views interface backed by the provided list of View.