View Source
const (
	// DefaultDelay is the default Delay value.
	DefaultDelay = 5 * time.Second

	// DefaultBufferBytes is the default number of bytes to buffer.
	DefaultBufferBytes = int64(1024 * 1024) // 1MB


View Source
var ErrIncompleteStream = errors.New("stream has not yet terminated")

    ErrIncompleteStream is returned by Fetcher if Options.RequireCompleteStream was true and the underlying Stream is still incomplete (i.e. has not yet been terminated by the client, or archived).


    This section is empty.


    type Fetcher

    type Fetcher struct {
    	// contains filtered or unexported fields

      A Fetcher buffers LogEntry records by querying the Source for log data. It attmepts to maintain a steady stream of records by prefetching available records in advance of consumption.

      A Fetcher is not goroutine-safe.

      func New

      func New(c context.Context, o Options) *Fetcher

        New instantiates a new Fetcher instance.

        The Fetcher can be cancelled by cancelling the supplied context.

        func (*Fetcher) Descriptor

        func (f *Fetcher) Descriptor() *logpb.LogStreamDescriptor

          Descriptor returns the last-known Descriptor from the underlying Stream, if known.

          If the underlying stream doesn't have this information, this returns nil.

          func (*Fetcher) NextLogEntry

          func (f *Fetcher) NextLogEntry() (*logpb.LogEntry, error)

            NextLogEntry returns the next buffered LogEntry, blocking until it becomes available.

            If the end of the log stream is encountered, NextLogEntry will return io.EOF.

            If the Fetcher is cancelled, a context.Canceled error will be returned.

            func (*Fetcher) Reader

            func (f *Fetcher) Reader() io.Reader

              Reader returns an io.Reader (a *renderer.Renderer) for this Fetcher.

              type LogRequest

              type LogRequest struct {
              	// Index is the starting log index to request.
              	Index types.MessageIndex
              	// Count, if >0, is the maximum number of log entries to request.
              	Count int
              	// Bytes, if >0, is the maximum number of log bytes to request. At least one
              	// log must be returned regardless of the byte limit.
              	Bytes int64

                LogRequest is a structure used by the Fetcher to request a range of logs from its Source.

                type Options

                type Options struct {
                	// Source is the log stream source.
                	Source Source
                	// Index is the starting stream index to retrieve.
                	Index types.MessageIndex
                	// Count is the total number of logs to retrieve. If zero, the full stream
                	// will be fetched.
                	Count int64
                	// Count is the minimum amount of LogEntry records to buffer. If the buffered
                	// amount dips below Count, more logs will be fetched.
                	// If zero, no count target will be applied.
                	BufferCount int
                	// BufferBytes is the target number of LogEntry bytes to buffer. If the
                	// buffered amount dips below Bytes, more logs will be fetched.
                	// If zero, no byte target will be applied unless Count is also zero, in which
                	// case DefaultBufferBytes byte constraint will be applied.
                	BufferBytes int64
                	// PrefetchFactor constrains the amount of additional data to fetch when
                	// refilling the buffer. Effective Count and Bytes values are multiplied
                	// by PrefetchFactor to determine the amount of logs to request.
                	PrefetchFactor int
                	// Delay is the amount of time to wait in between unsuccessful log requests.
                	Delay time.Duration
                	// Set this to immediately bail out with ErrIncompleteStream if the stream
                	// isn't complete yet. This can be useful when you believe the stream to
                	// already be terminal, but haven't done an RPC with LogDog yet to actually
                	// confirm this.
                	RequireCompleteStream bool
                	// contains filtered or unexported fields

                  Options is the set of configuration parameters for a Fetcher.

                  type Source

                  type Source interface {
                  	// LogEntries populates the supplied LogRequest with available sequential
                  	// log entries as available.
                  	// This may optionally block pending new log entries, but may also return zero
                  	// log entries if none are available yet.
                  	// Upon success, the requested logs and terminal message index is returned. If
                  	// no terminal index is known, a value <0 will be returned.
                  	LogEntries(context.Context, *LogRequest) ([]*logpb.LogEntry, types.MessageIndex, error)
                  	// Descriptor returns the stream's descriptor, if the source knows it.
                  	// If the source doesn't have this information, this should return nil.
                  	Descriptor() *logpb.LogStreamDescriptor

                    Source is the source of log stream and log information.

                    The Source is responsible for handling retries, backoff, and transient errors. An error from the Source will shut down the Fetcher.