broker

package
v0.87.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 10, 2020 License: MIT Imports: 31 Imported by: 1

Documentation

Overview

Package broker implements the broker runtime and protocol.JournalServer APIs (Read, Append, Replicate, List, Apply). Its `pipeline` type manages the coordination of write transactions, and `resolver` the mapping of journal names to Routes of responsible brokers. `replica` is a top-level collection of runtime state and maintenance tasks associated with the processing of a journal. gRPC proxy support is also implemented by this package.

Index

Constants

This section is empty.

Variables

View Source
var (
	// MaxAppendRate is the maximum rate at which any journal may be appended to,
	// in bytes per second. It upper-bounds the MaxAppendRate of any particular
	// JournalSpec. If zero, there is no maximum rate.
	MaxAppendRate int64 = 0 // No limit.
	// MinAppendRate is the minimum rate at which any Append RPC client may
	// stream content chunks, in bytes per second. Client RPCs that are unable
	// to sustain this flow rate in any given second are aborted, allowing other
	// blocked RPCs to proceed. MinAppendRate provides baseline protection to
	// limit the impact of slow or faulted clients over the pipeline, which is
	// an exclusively owned and highly contended resource.
	MinAppendRate int64 = 1 << 16 // 64K per second.
	// ErrFlowControlUnderflow is returned if an Append RPC was terminated due to
	// flow control policing. Specifically, the client failed to sustain the
	// MinAppendRate when sending content chunks of the stream.
	ErrFlowControlUnderflow = errors.New(
		"client stream didn't sustain the minimum append data rate")
)

Functions

func JournalIsConsistent added in v0.86.1

func JournalIsConsistent(item allocator.Item, _ keyspace.KeyValue, assignments keyspace.KeyValues) bool

JournalIsConsistent returns true if all allocator.Assignments of the JournalSpec identified by Item advertise the same Route, denoting that all replicas of the journal have synchronized.

func JournalRouteMatchesAssignments added in v0.86.1

func JournalRouteMatchesAssignments(rt pb.Route, assignments keyspace.KeyValues) bool

JournalRouteMatchesAssignments returns true iff the Route is equivalent to the Route marshaled with each of the journal's |assignments|.

func NewKeySpace

func NewKeySpace(prefix string) *keyspace.KeySpace

NewKeySpace returns a KeySpace suitable for use with an Allocator. It decodes allocator Items as JournalSpec messages, Members as BrokerSpecs, and Assignments as Routes.

func SetSharedPersister

func SetSharedPersister(p *fragment.Persister)

SetSharedPersister sets the Persister instance used by the `broker` package.

Types

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service is the top-level runtime concern of a Gazette Broker process. It drives local journal handling in response to allocator.State, powers journal resolution, and is also an implementation of protocol.JournalServer.

func NewService

func NewService(state *allocator.State, jc pb.JournalClient, etcd *clientv3.Client) *Service

NewService constructs a new broker Service, driven by allocator.State.

func (*Service) Append

func (svc *Service) Append(stream pb.Journal_AppendServer) (err error)

Append dispatches the JournalServer.Append API.

func (*Service) Apply

func (svc *Service) Apply(ctx context.Context, req *pb.ApplyRequest) (resp *pb.ApplyResponse, err error)

Apply dispatches the JournalServer.Apply API.

func (*Service) IsNoopRouter

func (svc *Service) IsNoopRouter() bool

IsNoopRouter returns false.

func (*Service) List

func (svc *Service) List(ctx context.Context, req *pb.ListRequest) (resp *pb.ListResponse, err error)

List dispatches the JournalServer.List API.

func (*Service) ListFragments

func (svc *Service) ListFragments(ctx context.Context, req *pb.FragmentsRequest) (resp *pb.FragmentsResponse, err error)

ListFragments dispatches the JournalServer.ListFragments API.

func (*Service) QueueTasks

func (svc *Service) QueueTasks(tasks *task.Group, server *server.Server, finishFn func())

QueueTasks of the Service to watch its KeySpace and serve local replicas.

func (*Service) Read

func (svc *Service) Read(req *pb.ReadRequest, stream pb.Journal_ReadServer) (err error)

Read dispatches the JournalServer.Read API.

func (*Service) Replicate

func (svc *Service) Replicate(stream pb.Journal_ReplicateServer) (err error)

Replicate dispatches the JournalServer.Replicate API.

func (*Service) Route

func (svc *Service) Route(ctx context.Context, item string) pb.Route

Route an item using the Service resolver. Route implements the protocol.DispatchRouter interface, and enables usages of protocol.WithDispatchItemRoute (eg, `client` & `http_gateway` packages) to resolve items via the Service resolver.

func (*Service) UpdateRoute

func (svc *Service) UpdateRoute(string, *pb.Route)

UpdateRoute is a no-op implementation of protocol.DispatchRouter.

Directories

Path Synopsis
Package client implements a Go client for interacting with the gRPC Journal service of Gazette brokers.
Package client implements a Go client for interacting with the gRPC Journal service of Gazette brokers.
Package fragment is a broker-only package concerned with the mapping of journal offsets to protocol.Fragments, and from there to corresponding local or remote journal content.
Package fragment is a broker-only package concerned with the mapping of journal offsets to protocol.Fragments, and from there to corresponding local or remote journal content.
Package journalspace provides mechanisms for mapping a collection of JournalSpecs into a minimally-described hierarchical structure, and for mapping back again.
Package journalspace provides mechanisms for mapping a collection of JournalSpecs into a minimally-described hierarchical structure, and for mapping back again.
Package protocol defines the core broker datamodel, validation behaviors, and gRPC APIs which are shared across clients and broker servers.
Package protocol defines the core broker datamodel, validation behaviors, and gRPC APIs which are shared across clients and broker servers.
ext
Package ext defines extensions to broker/protocol that depend on keyspace and allocator (which in turn depend on etcd).
Package ext defines extensions to broker/protocol that depend on keyspace and allocator (which in turn depend on etcd).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL