archivist

Documentation

Index

• type Archivist
  • func (a *Archivist) ArchiveTask(c context.Context, task Task)
• type Settings
• type SettingsLoader
• type Task

Types

type Archivist

type Archivist struct {
	// Service is the client to use to communicate with Coordinator's Services
	// endpoint.
	Service logdog.ServicesClient

	// SettingsLoader loads archival settings for a specific project.
	SettingsLoader SettingsLoader

	// Storage is the archival source Storage instance.
	Storage storage.Storage
	// GSClient is the Google Storage client to for archive generation.
	GSClient gs.Client
}

Archivist is a stateless configuration capable of archiving individual log streams.

func (*Archivist) ArchiveTask

func (a *Archivist) ArchiveTask(c context.Context, task Task)

ArchiveTask processes and executes a single log stream archive task.

During processing, the Task's Consume method may be called to indicate that it should be consumed.

If the supplied Context is Done, operation may terminate before completion, returning the Context's error.

type Settings

type Settings struct {
	// GSBase is the base Google Storage path. This includes the bucket name
	// and any associated path.
	//
	// This must be unique to this archival project. In practice, it will be
	// composed of the project's archival bucket and project ID.
	GSBase gs.Path
	// GSStagingBase is the base Google Storage path for archive staging. This
	// includes the bucket name and any associated path.
	//
	// This must be unique to this archival project. In practice, it will be
	// composed of the project's staging archival bucket and project ID.
	GSStagingBase gs.Path

	// AlwaysRender, if true, means that a binary should be archived
	// regardless of whether a specific binary file extension has been supplied
	// with the log stream.
	AlwaysRender bool

	// IndexStreamRange is the maximum number of stream indexes in between index
	// entries. See archive.Manifest for more information.
	IndexStreamRange int
	// IndexPrefixRange is the maximum number of prefix indexes in between index
	// entries. See archive.Manifest for more information.
	IndexPrefixRange int
	// IndexByteRange is the maximum number of stream data bytes in between index
	// entries. See archive.Manifest for more information.
	IndexByteRange int
}

Settings defines the archival parameters for a specific archival operation.

In practice, this will be formed from service and project settings.

type SettingsLoader

type SettingsLoader func(context.Context, cfgtypes.ProjectName) (*Settings, error)

SettingsLoader returns archival Settings for a given project.

type Task

type Task interface {
	// UniqueID returns a task-unique value. Other tasks, and other retries of
	// this task, should (try to) not reuse this ID.
	UniqueID() string

	// Task is the archive task to execute.
	Task() *logdog.ArchiveTask

	// Consume marks that this task's processing is complete and that it should be
	// consumed. This may be called multiple times with no additional effect.
	Consume()

	// AssertLease asserts that the lease for this Task is still held.
	//
	// On failure, it will return an error. If successful, the Archivist may
	// assume that it holds the lease longer.
	AssertLease(context.Context) error
}

Task is a single archive task.