Documentation

Overview

    Package prjmanager implements public API for Project Manager.

    The Project Manager keeps track of the project config, incomplete runs, and CLs in flight, and notifies runs when the project config changes.

    Index

    Constants

    View Source 
    const ProjectKind = "Project"

      ProjectKind is the Datastore entity kind for Project.

      Variables

      View Source 
      var StateChangedTag = errors.BoolTag{Key: errors.NewTagKey("the task should be dropped")}

        StateChangedTag is an error tag used to indicate that state read from Datastore differs from the expected state.

        Functions

        func NotifyCLUpdated 

        func NotifyCLUpdated(ctx context.Context, luciProject string, clid common.CLID, eversion int) error

          NotifyCLUpdated tells ProjectManager to check latest version of a given CL.

          func NotifyCLsUpdated 

          func NotifyCLsUpdated(ctx context.Context, luciProject string, cls []*changelist.CL) error

            NotifyCLsUpdated is a batch of NotifyCLUpdated for the same ProjectManager.

            In each given CL, .ID and .EVersion must be set.

            func NotifyRunCreated 

            func NotifyRunCreated(ctx context.Context, runID common.RunID) error

              NotifyRunCreated is sent by ProjectManager to itself within a Run creation transaction.

              Unlike other event-sending funcs, this only creates an event and doesn't create a task. This is fine because: 

              * if Run creation transaction fails, then this event isn't actually
  created anyways.
* if ProjectManager observes the Run creation success, then it'll act as if
  this event was received in the upcoming state transition. Yes, it won't
  process this event immediately, but at this point the event is a noop,
  so it'll be cleared out from the eventbox upon next invocation of
  ProjectManager. So there is no need to create a TQ task.
* else, namely Run creation succeeds but ProjectManager sees it as a
  failure OR ProjectManager fails at any point before it can act on
  RunCreation, then the existing TQ task running ProjectManager will be
  retried. So once again there is no need to create a TQ task.

              func NotifyRunFinished 

              func NotifyRunFinished(ctx context.Context, runID common.RunID) error

                NotifyRunFinished tells ProjectManager that a run has finalized its state.

                func Poke 

                func Poke(ctx context.Context, luciProject string) error

                  Poke tells ProjectManager to poke all downstream actors and check its own state.

                  func UpdateConfig 

                  func UpdateConfig(ctx context.Context, luciProject string) error

                    UpdateConfig tells ProjectManager to read and update to newest ProjectConfig by fetching it from Datatstore.

                    Results in stopping ProjectManager if ProjectConfig got disabled or deleted.

                    Types

                    type Project 

                    type Project struct {

	// ID is LUCI project name.
	ID string `gae:"$id"`

	// EVersion is entity version. Every update should increment it by 1.
	EVersion int `gae:",noindex"`
	// UpdateTime is exact time of when this entity was last updated.
	//
	// It's not indexed to avoid hot areas in the index.
	UpdateTime time.Time `gae:",noindex"`

	// State serializes internal Project Manager state.
	//
	// The `LuciProject` field isn't set as it duplicates Project.ID.
	State *prjpb.PState
	// contains filtered or unexported fields
}

                      Project is an entity per LUCI Project in Datastore.

                      func Load 

                      func Load(ctx context.Context, luciProject string) (*Project, error)

                        Load loads LUCI project state from Datastore.

                        If project doesn't exist in Datastore, returns nil, nil, nil.

                        func (*Project) ConfigHash 

                        func (p *Project) ConfigHash() string

                          ConfigHash returns Project's Config hash.

                          func (*Project) IncompleteRuns 

                          func (p *Project) IncompleteRuns() (ids common.RunIDs)

                            IncompleteRuns are IDs of Runs which aren't yet completed.

                            func (*Project) Status 

                            func (p *Project) Status() prjpb.Status

                              Status returns Project Manager status.

                              type ProjectStateOffload 

                              type ProjectStateOffload struct {

	// ID is alaways the same, set/read only by the datastore ORM.
	ID      string         `gae:"$id,const"`
	Project *datastore.Key `gae:"$parent"`

	// Status of project manager {STARTED, STOPPING, STOPPED (disabled)}.
	Status prjpb.Status `gae:",noindex"`
	// ConfigHash is the latest processed Project Config hash.
	ConfigHash string `gae:",noindex"`
	// contains filtered or unexported fields
}

                                ProjectStateOffload stores rarely-changed project state duplicated from the main Project entity for use in transactions creating Runs.

                                Although this is already stored in the main Project entity, doing so would result in retries of Run creation transactions, since Project entity is frequently modified in busy projects.

                                On the other hand, ProjectStateOffload is highly likely to remain unchanged by the time Run creation transaction commits, thus avoiding needless retries.

                                type RunBuilder 

                                type RunBuilder struct {

	// LUCIProject. Required.
	LUCIProject string
	// ConfigGroupID for the Run. Required.
	//
	// TODO(tandrii): support triggering via API calls by validating Run creation
	// request against the latest config *before* transaction.
	ConfigGroupID config.ConfigGroupID
	// InputCLs will reference the newly created Run via their IncompleteRuns
	// field, and Run's RunCL entities will reference these InputCLs back.
	// Required.
	InputCLs []RunBuilderCL
	// Mode is the Run's mode. Required.
	Mode run.Mode
	// Owner is the Run Owner. Required.
	Owner identity.Identity
	// ExpectedIncompleteRunIDs are a sorted slice of Run IDs which may be associated with
	// CLs.
	//
	// If CLs.IncompleteRuns reference any other Run ID, the creation will be
	// aborted and error tagged with StateChangedTag.
	//
	// Nil by default, which doesn't permit any incomplete Run.
	ExpectedIncompleteRunIDs common.RunIDs
	// OperationID is an arbitrary string uniquely identifying this creation
	// attempt.
	//
	// TODO(tandrii): for CV API, record this ID in a separate entity index by
	// this ID for full idempotency of CV API.
	OperationID string
	// contains filtered or unexported fields
}

                                  RunBuilder creates a new Run.

                                  If Expected<...> parameters differ from what's read from Datastore during transaction, the creation is aborted with error tagged with StateChangedTag. See RunBuilder.Create doc.

                                  func (*RunBuilder) Create 

                                  func (rb *RunBuilder) Create(ctx context.Context) (*run.Run, error)

                                    Create atomically creates a new Run.

                                    Returns the newly created Run.

                                    Returns 3 kinds of errors: 

                                    * tagged with transient.Tag, meaning it's reasonable to retry.
  Typically due to contention on simultaneously updated CL entity or
  transient Datastore Get/Put problem.

* tagged with StateChangedTag.
  This means the desired Run might still be possible to create,
  but it must be re-validated against updated CLs or Project config
  version.

* all other errors are non retryable and typically indicate a bug or severe
  misconfiguration. For example, lack of ProjectStateOffload entity.

                                    type RunBuilderCL 

                                    type RunBuilderCL struct {
	ID               common.CLID
	ExpectedEVersion int
	TriggerInfo      *run.Trigger
	Snapshot         *changelist.Snapshot // Only needed for compat with CQDaemon.
}

                                      RunBuilderCL is a helper struct for per-CL input for run creation.

                                      Source Files

                                      View all

                                      Directories

                                      Path Synopsis
                                      Package copyonwrite providers helpers for modifying slices in Copy-on-Write way.
                                      		 Package copyonwrite providers helpers for modifying slices in Copy-on-Write way.
                                      Package impl implements a ProjectManager.
                                      		 Package impl implements a ProjectManager.
                                      state
                                      Package state implements state machine of a Project Manager.
                                      		 Package state implements state machine of a Project Manager.
                                      state/componentactor
                                      Package componentactor acts on a specific component by implementing state.componentActor.
                                      		 Package componentactor acts on a specific component by implementing state.componentActor.
                                      Package pmtest implements tests for working with Project Manager.
                                      		 Package pmtest implements tests for working with Project Manager.
                                      package prjinternal stores protos for event processing of ProjectManager.
                                      		 package prjinternal stores protos for event processing of ProjectManager.