Documentation

Overview

    Package butler is the main engine for the Butler executable. It is an internal package to support go.chromium.org/luci/logdog/client/cmd/logdog_butler.

    Index

    Constants

    View Source
    const (
    	// DefaultMaxBufferAge is the default amount of time that a log entry may
    	// be buffered before being dispatched.
    	DefaultMaxBufferAge = time.Duration(5 * time.Second)
    )

    Variables

    This section is empty.

    Functions

    This section is empty.

    Types

    type Butler

    type Butler struct {
    	// contains filtered or unexported fields
    }

      Butler is the Butler application structure. The Butler runs until closed. During operation, it acts as a service manager and data router, routing: - Messages from Streams to the attached Output. - Streams from a StreamServer to the Stream list (AddStream).

      func New

      func New(ctx context.Context, config Config) (*Butler, error)

        New instantiates a new Butler instance and starts its processing.

        func (*Butler) Activate

        func (b *Butler) Activate()

          Activate notifies the Butler that its current stream load is sufficient. This enables it to exit Run when it reaches a stream count of zero. Prior to activation, the Butler would block in Run regardless of stream count.

          func (*Butler) AddStream

          func (b *Butler) AddStream(rc io.ReadCloser, d *logpb.LogStreamDescriptor) error

            AddStream adds a Stream to the Butler. This is goroutine-safe.

            If no error is returned, the Butler assumes ownership of the supplied stream. The stream will be closed when processing is finished.

            If an error is occurred, the caller is still the owner of the stream and is responsible for closing it.

            func (*Butler) AddStreamRegistrationCallback

            func (b *Butler) AddStreamRegistrationCallback(cb StreamRegistrationCallback, wrap bool)

            AddStreamRegistrationCallback adds a new callback to this Butler.

            The callback is called on new streams and returns a chunk callback to attach to the stream or nil if you don't want to monitor the stream.

            If multiple callbacks are all interested in the same stream, the first one wins.

            Wrapping

            If `wrap` is true, the callback is wrapped internally to buffer LogEntries until they're complete.

            In wrapped callbacks for text and datagram streams, LogEntry .TimeOffset, and .PrefixIndex will be 0. .StreamIndex and .Sequence WILL NOT correspond to the values that the logdog service sees. They will, however, be internally consistent within the stream.

            Wrapped datagram streams never send a partial datagram; If the logdog server or stream is shut down while we have a partial datagram buffered, the partially buffered datagram will not be observed by the buffered callback.

            Wrapping a binary stream is a noop (i.e. your callback will see the exact same values wrapped and unwrapped).

            When the stream ends (either due to EOF from the user, or when the butler is stopped), your callback will be invoked exactly once with `nil`.

            func (*Butler) AddStreamServer

            func (b *Butler) AddStreamServer(streamServer StreamServer)

              AddStreamServer adds a StreamServer to the Butler. This is goroutine-safe and may be called anytime before or during Butler execution.

              After this call completes, the Butler assumes ownership of the StreamServer.

              func (*Butler) DrainNamespace

              func (b *Butler) DrainNamespace(ctx context.Context, namespace types.StreamName) []types.StreamName

                DrainNamespace prevents any new streams from being registered in the given namespace, and waits for all existing streams in that namespace to drain.

                If this exits due to context cancelation, it returns the list of stream names in the namespace which are still open.

                func (*Butler) Streams

                func (b *Butler) Streams() []types.StreamName

                  Streams returns a sorted list of stream names that have been registered to the Butler.

                  func (*Butler) Wait

                  func (b *Butler) Wait() error

                    Wait blocks until the Butler instance has completed, returning with the Butler's return code.

                    type Config

                    type Config struct {
                    	// Output is the output instance to use for log dispatch.
                    	Output output.Output
                    
                    	// GlobalTags are a set of global log stream tags to apply to individual
                    	// streams on registration. Individual stream tags will override tags with
                    	// the same key.
                    	GlobalTags streamproto.TagMap
                    
                    	// BufferLogs, if true, instructs the butler to buffer collected log data
                    	// before sending it to Output.
                    	BufferLogs bool
                    
                    	// If buffering logs, this is the maximum amount of time that a log will
                    	// be buffered before being marked for dispatch. If this is zero,
                    	// DefaultMaxBufferAge will be used.
                    	MaxBufferAge time.Duration
                    }

                      Config is the set of Butler configuration parameters.

                      func (*Config) Validate

                      func (c *Config) Validate() error

                        Validate validates that the configuration is sufficient to instantiate a Butler instance.

                        type StreamChunkCallback

                        type StreamChunkCallback func(*logpb.LogEntry)

                          StreamChunkCallback is a callback to invoke on a complete LogEntry.

                          Called once with nil when this stream has come to an end.

                          See streamConfig.callback.

                          type StreamRegistrationCallback

                          type StreamRegistrationCallback func(*logpb.LogStreamDescriptor) StreamChunkCallback

                            StreamRegistrationCallback is a callback to invoke when a new stream is registered with this butler.

                            Expects passed *logpb.LogStreamDescriptor reference to be safe to keep, and should treat it as read-only.

                            See streamConfig.callback.

                            type StreamServer

                            type StreamServer interface {
                            	Next() (io.ReadCloser, *logpb.LogStreamDescriptor)
                            	Close()
                            }

                              StreamServer is butler's interface for go.chromium.org/luci/logdog/client/butler/streamserver

                              TODO(iannucci): internalize streamserver to butler; there's no need for it to be managed by the user.

                              Directories

                              Path Synopsis
                              Package bootstrap handles Butler-side bootstrapping functionality.
                              Package bootstrap handles Butler-side bootstrapping functionality.
                              Package bundler is responsible for efficiently transforming aggregate stream data into Butler messages for export.
                              Package bundler is responsible for efficiently transforming aggregate stream data into Butler messages for export.
                              Package output contains interfaces and implementations for Butler Outputs, which are responsible for delivering Butler protobufs to LogDog collection endpoints.
                              Package output contains interfaces and implementations for Butler Outputs, which are responsible for delivering Butler protobufs to LogDog collection endpoints.
                              log
                              Package log implements the "log" Output.
                              Package log implements the "log" Output.
                              logdog
                              Package logdog implements output to a Logdog server via PubSub.
                              Package logdog implements output to a Logdog server via PubSub.
                              memory
                              Package memory implements an in-memory sink for the logdog Butler.
                              Package memory implements an in-memory sink for the logdog Butler.