Documentation

Index

Constants

View Source
const BotPoolKind = "BotPool"

    BotPoolKind is the name of the datastore entity kind of BotPool.

    View Source
    const BuilderSummaryKind = "BuilderSummary"

      BuilderSummaryKind is the name of the datastore entity kind of the entity represented by the BuilderSummary struct.

      View Source
      const InvalidBuildIDURL = "#invalid-build-id"

        InvalidBuildIDURL is returned if a BuildID cannot be parsed and a URL generated.

        View Source
        const InvalidBuilderIDURL = "#invalid-builder-id"

          InvalidBuilderIDURL is returned if a BuilderID cannot be parsed and a URL generated.

          View Source
          const SwarmingHostnameKey = "swarming_hostname"

            SwarmingHostnameKey is a magic PoolDescriptor tag containing the Swarming Hostname.

            Variables

            This section is empty.

            Functions

            func BuilderIDLink(b, project string) string

              BuilderIDLink gets a builder link from builder ID and project. Depends on routes.go.

              func UpdateBuilderForBuild

              func UpdateBuilderForBuild(c context.Context, build *BuildSummary) error

                UpdateBuilderForBuild updates the appropriate BuilderSummary for the provided BuildSummary, if needed. In particular, a BuilderSummary is updated with a BuildSummary if the latter is marked complete and has a more recent creation time than the one stored in the BuilderSummary. If there is no existing BuilderSummary for the BuildSummary provided, one is created. Idempotent. c must have a current datastore transaction.

                Types

                type Bot

                type Bot struct {
                	// Name is an identifier for the Bot.  This is usually a short hostname.
                	Name string
                	// URL is a link to a bot page, if available.
                	URL string
                	// Status is the current status of the Bot.
                	Status BotStatus
                	// LastSeen denotes when the Bot was last seen.
                	LastSeen time.Time
                }

                  Bot represents a single job executor.

                  type BotPool

                  type BotPool struct {
                  	// PoolID is an identifier for the BotPool.
                  	// PoolID must match the Descriptor.
                  	// This field should always be constructed with PoolID().
                  	PoolID string `gae:"$id"`
                  
                  	// Descriptor uniquely describes this BotPool.
                  	// It is defined as Swarming Dimensions + Hostname.
                  	Descriptor PoolDescriptor
                  
                  	// Bots is a slice of bots in the pool, along with their statuses.
                  	Bots []Bot `gae:",noindex"`
                  
                  	// LastUpdate is when this entity was last updated.
                  	LastUpdate time.Time
                  }

                    BotPool is the execution pool information associated with a set of Swarming bots. This information is periodically refreshed from Swarming.

                    type BotStatus

                    type BotStatus int

                      BotStatus indicates the status of a machine.

                      const (
                      	// Idle means the bot is ready to accept a job.
                      	Idle BotStatus = iota
                      	// Busy means the bot is currently running a job, or recently finished
                      	// a job and may not be ready to accept a new job yet.
                      	Busy
                      	// Offline means the bot is neither ready to accept a job nor
                      	// currently running a job.  The bot could be dead or quarantined.
                      	Offline
                      )

                      func (BotStatus) String

                      func (i BotStatus) String() string

                      type BuildSummary

                      type BuildSummary struct {
                      
                      	// BuildKey will always point to the "real" build, i.e. a buildbotBuild or
                      	// a buildbucketBuild. It is always the parent key for the BuildSummary.
                      	// The parent build may or may not actually exist.
                      	BuildKey *datastore.Key `gae:"$parent"`
                      
                      	// Global identifier for the builder that this Build belongs to, i.e.:
                      	//   "buildbot/<mastername>/<buildername>"
                      	//   "buildbucket/<bucketname>/<buildername>"
                      	BuilderID string
                      
                      	// Global identifier for this Build.
                      	// Buildbot: "buildbot/<mastername>/<buildername>/<buildnumber>"
                      	// Buildbucket: "buildbucket/<buildaddr>"
                      	// For buildbucket, <buildaddr> looks like <bucketname>/<buildername>/<buildnumber> if available
                      	// and <buildid> otherwise.
                      	BuildID string
                      
                      	// The LUCI project ID associated with this build. This is used for ACL checks
                      	// when presenting this build to end users.
                      	ProjectID string
                      
                      	// This contains URI to any contextually-relevant underlying tasks/systems
                      	// associated with this build, in the form of:
                      	//
                      	//   * swarming://<host>/task/<taskID>
                      	//   * swarming://<host>/bot/<botID>
                      	//   * buildbot://<master>/build/<builder>/<number>
                      	//   * buildbot://<master>/bot/<bot>
                      	//   * buildbucket://<host>/build/<buildID>
                      	//
                      	// This will be used for queries, and can be used to store semantically-sound
                      	// clues about this Build (e.g. to identify the underlying swarming task so
                      	// that we don't need to RPC back to the build source to find that out). This
                      	// can also be used for link generation in the UI, since the schema for these
                      	// URIs should be stable within Milo (so if swarming changes its URL format we
                      	// can change the links in the UI code without map-reducing these
                      	// ContextURIs).
                      	ContextURI []string
                      
                      	// The buildbucket buildsets associated with this Build, if any.
                      	//
                      	// Example:
                      	//   commit/gitiles/<host>/<project/path>/+/<commit>
                      	//
                      	// See https://chromium.googlesource.com/infra/infra/+/master/appengine/cr-buildbucket/doc/index.md#buildset-tag
                      	BuildSet []string
                      
                      	// Created is the time when the Build was first created. Due to pending
                      	// queues, this may be substantially before Summary.Start.
                      	Created time.Time
                      
                      	// Summary summarizes relevant bits about the overall build.
                      	Summary Summary
                      
                      	// Manifests is a list of links to source manifests that this build reported.
                      	Manifests []ManifestLink
                      
                      	// ManifestKeys is the list of ManifestKey entries for this BuildSummary.
                      	ManifestKeys []ManifestKey
                      
                      	// AnnotationURL is the URL to the logdog annotation location. This will be in
                      	// the form of:
                      	//   logdog://service.host.example.com/project_id/prefix/+/stream/name
                      	// (Deprecated)
                      	AnnotationURL string
                      
                      	// Version can be used by buildsource implementations to compare with an
                      	// externally provided version number/timestamp to ensure that BuildSummary
                      	// objects are only updated forwards in time.
                      	//
                      	// Known uses:
                      	//   * Buildbucket populates this with Build.UpdatedTs, which is guaranteed to
                      	//     be monotonically increasing. Used to ignore out-of-order pubsub
                      	//     messages.
                      	Version int64
                      
                      	// Experimental indicates that even though this build belongs to the indicated
                      	// Builder, it's considered experimental (e.g. "not in production"). The meaning
                      	// of this varies by builder, but generally it indicates that this build did
                      	// not write to production services and/or was not running the production
                      	// version of this Builder's job definition (e.g. a modified not-committed
                      	// recipe).
                      	Experimental bool
                      
                      	// If NO, then the build status SHOULD NOT be used to assess correctness of
                      	// the input gitiles_commit or gerrit_changes.
                      	// For example, if a pre-submit build has failed, CQ MAY still land the CL.
                      	// For example, if a post-submit build has failed, CLs MAY continue landing.
                      	Critical buildbucketpb.Trinary
                      	// contains filtered or unexported fields
                      }

                        BuildSummary is a datastore model which is used for storing staandardized summarized build data, and is used for backend-agnostic views (i.e. builders, console). It contains only data that:

                        * is necessary to render these simplified views
                        * is present in all implementations (buildbot, buildbucket)
                        

                        This entity will live as a child of the various implementation's representations of a build (e.g. buildbotBuild). It has various 'tag' fields so that it can be queried by the various backend-agnostic views.

                        func (*BuildSummary) AddManifestKey

                        func (bs *BuildSummary) AddManifestKey(project, console, manifest, repoURL string, revision []byte)

                          AddManifestKey adds a new entry to ManifestKey.

                          `revision` should be the hex-decoded git revision.

                          It's up to the caller to ensure that entries in ManifestKey aren't duplicated.

                          func (*BuildSummary) AddManifestKeysFromBuildSets

                          func (bs *BuildSummary) AddManifestKeysFromBuildSets(c context.Context) error

                            AddManifestKeysFromBuildSets potentially adds one or more ManifestKey's to the BuildSummary for any defined BuildSets.

                            This assumes that bs.BuilderID and bs.BuildSet have already been populated. If BuilderID is not populated, this will return an error.

                            func (*BuildSummary) GetBuildName

                            func (bs *BuildSummary) GetBuildName() string

                              GetBuildName returns an abridged version of the BuildID meant for human consumption. Currently, this is always the last token of the BuildID.

                              func (*BuildSummary) GetConsoleNames

                              func (bs *BuildSummary) GetConsoleNames() ([]string, error)

                                GetConsoleNames extracts the "<project>/<console>" names from the BuildSummary's current ManifestKeys.

                                func (*BuildSummary) GitilesCommit

                                func (bs *BuildSummary) GitilesCommit() *buildbucketpb.GitilesCommit

                                  GitilesCommit extracts the first BuildSet which is a valid GitilesCommit.

                                  If no such BuildSet is found, this returns nil.

                                  func (bs *BuildSummary) SelfLink() string

                                    SelfLink returns a link to the build represented by the BuildSummary via BuildID.

                                    BuildID is used for indexing BuildSummary and BuilderSummary entities, so this lets us get links given BuildSummaries and BuilderSummaries in the console, console header, and console lists.

                                    Returns bogus URL in case of error (just "/" + BuildID). Depends on buildbucket.ParseBuildAddress to get project Depends on frontend/routes.go for link structures.

                                    Buildbot: "buildbot/<mastername>/<buildername>/<buildnumber>" Buildbucket: "buildbucket/<buildaddr>" For buildbucket, <buildaddr> looks like <bucketname>/<buildername>/<buildnumber> if available and <buildid> otherwise.

                                    type BuilderPool

                                    type BuilderPool struct {
                                    
                                    	// BuilderID is a datastore key pointing to the parent BuilderSummary entity.
                                    	BuilderID *datastore.Key `gae:"$parent"`
                                    
                                    	// PoolKey is a datastore key pointing to the BotPool associated with this Builder.
                                    	PoolKey *datastore.Key
                                    	// contains filtered or unexported fields
                                    }

                                      BuilderPool is a datastore entity that associates a BuilderSummary entity with a BotPool entity. BuilderPool is a separate entity from BuilderSummary to allow for transaction-less updates.

                                      type BuilderSummary

                                      type BuilderSummary struct {
                                      	// BuilderID is the global identifier for the builder that this Build belongs to, i.e.:
                                      	//   "buildbucket/<bucketname>/<buildername>"
                                      	// Matches field in BuildSummary.
                                      	BuilderID string `gae:"$id"`
                                      
                                      	// The LUCI project ID associated with this build. This is used for generating links.
                                      	ProjectID string
                                      
                                      	// LastFinishedCreated is the time the last finished build was created.
                                      	LastFinishedCreated time.Time
                                      
                                      	// LastFinishedStatus is the status of last finished build on builder.
                                      	LastFinishedStatus Status
                                      
                                      	// LastFinishedCritical is the criticality of last finished build on builder.
                                      	LastFinishedCritical buildbucketpb.Trinary
                                      
                                      	// LastFinishedBuildID is the BuildID of the BuildSummary associated with last finished build on
                                      	// the builder.
                                      	LastFinishedBuildID string
                                      
                                      	// LastFinishedExperimental indicates if the last build on this builder was
                                      	// marked as experimental.
                                      	LastFinishedExperimental bool
                                      	// contains filtered or unexported fields
                                      }

                                        BuilderSummary holds builder state for the purpose of representing e.g. header consoles.

                                        func (b *BuilderSummary) LastFinishedBuildIDLink() string

                                          LastFinishedBuildIDLink returns a link to the last finished build.

                                          func (b *BuilderSummary) SelfLink() string

                                            SelfLink returns a link to the associated builder.

                                            type Link struct {
                                            	// Title (text) of the link.
                                            	Label string `json:"label,omitempty"`
                                            
                                            	// The destination for the link.
                                            	URL string `json:"url,omitempty"`
                                            }

                                              Link denotes a single labeled link.

                                              type ManifestKey

                                              type ManifestKey []byte

                                                ManifestKey is an index entry for BuildSummary, which looks like

                                                0 ++ project ++ console ++ manifest_name ++ url ++ revision.decode('hex')
                                                

                                                This is used to index this BuildSummary as the row for any consoles that it shows up in that use the Manifest/RepoURL/Revision indexing scheme.

                                                (++ is cmpbin concatenation)

                                                Example:

                                                0 ++ "chromium" ++ "main" ++ "UNPATCHED" ++ "https://.../src.git" ++ deadbeef
                                                

                                                The list of interested consoles is compiled at build summarization time.

                                                type ManifestLink struct {
                                                	// The name of the manifest as the build annotated it.
                                                	Name string
                                                
                                                	// The manifest ID (sha256).
                                                	ID []byte
                                                }

                                                  ManifestLink is an in-MILO link to a named source manifest.

                                                  func (*ManifestLink) FromProperty

                                                  func (m *ManifestLink) FromProperty(p ds.Property) (err error)

                                                    FromProperty implements ds.PropertyConverter

                                                    func (*ManifestLink) ToProperty

                                                    func (m *ManifestLink) ToProperty() (ds.Property, error)

                                                      ToProperty implements ds.PropertyConverter

                                                      type PartialManifestKey

                                                      type PartialManifestKey []byte

                                                        PartialManifestKey is an incomplete ManifestKey key which can be made complete by calling AddRevision.

                                                        func NewPartialManifestKey

                                                        func NewPartialManifestKey(project, console, manifest, repoURL string) PartialManifestKey

                                                          NewPartialManifestKey generates a ManifestKey prefix corresponding to the given parameters.

                                                          func (PartialManifestKey) AddRevision

                                                          func (p PartialManifestKey) AddRevision(revision []byte) ManifestKey

                                                            AddRevision appends a git revision (as bytes) to the PartialManifestKey, returning a full index value for BuildSummary.ManifestKey.

                                                            type PatchInfo

                                                            type PatchInfo struct {
                                                            	// A link to the patch page (i.e. the gerrit/rietveld review page).
                                                            	Link Link
                                                            
                                                            	// The email of the author of the Patch.
                                                            	AuthorEmail string
                                                            }

                                                              PatchInfo provides information about a patch included in a build.

                                                              type PoolDescriptor

                                                              type PoolDescriptor []string

                                                                PoolDescriptor describes the attributes of a pool in "tag:value" format. This is defined as Swarming Dimensions + Hostname.

                                                                func NewPoolDescriptor

                                                                func NewPoolDescriptor(host string, dimensions []string) PoolDescriptor

                                                                  NewPoolDescriptor creates a new PoolDescriptor given a swarming hostname and a slice of dimensions. The dimensions are expected to be in "key:value" format, where the key is the tag name. This is the same format returned by the Swarming API.

                                                                  func (PoolDescriptor) Dimensions

                                                                  func (d PoolDescriptor) Dimensions() strpair.Map

                                                                    Dimensions returns a copy of the PoolDescriptor without the swarming_hostname item in strpair.Map format.

                                                                    func (PoolDescriptor) Host

                                                                    func (d PoolDescriptor) Host() string

                                                                      Host returns the swarming_hostname item of the PoolDescriptor.

                                                                      func (PoolDescriptor) PoolID

                                                                      func (d PoolDescriptor) PoolID() string

                                                                        PoolID returns they ID for a BotPool with the specified PoolDescriptor. The ID is a SHA256 hash in hex form. descriptor must be pre-sorted (I.E. Created with NewPoolDescriptor).

                                                                        type Status

                                                                        type Status int

                                                                          Status indicates the status of some piece of the CI; builds, steps, builders, etc. The UI maps this to a color, and some statuses may map to the same color; however, we attempt to preserve full informational fidelity.

                                                                          const (
                                                                          	// NotRun if the component has not yet been run.  E.g. if the component has
                                                                          	// been scheduled, but is pending execution.
                                                                          	NotRun Status = iota
                                                                          
                                                                          	// Running if the component is currently running.
                                                                          	Running
                                                                          
                                                                          	// Success if the component has finished executing and accomplished what it
                                                                          	// was supposed to.
                                                                          	Success
                                                                          
                                                                          	// Failure if the component has finished executing and failed in
                                                                          	// a non-exceptional way. e.g. if a test completed execution, but determined
                                                                          	// that the code was bad.
                                                                          	Failure
                                                                          
                                                                          	// Warning if the component has finished executing, but encountered
                                                                          	// non-stoppage problems. e.g. if a test completed execution, but determined
                                                                          	// that the code was slow (but not slow enough to be a failure).
                                                                          	Warning
                                                                          
                                                                          	// InfraFailure if the component has finished incompletely due to an
                                                                          	// infrastructure layer.
                                                                          	//
                                                                          	// This is used to categorize all unknown errors.
                                                                          	InfraFailure
                                                                          
                                                                          	// Exception if the component has finished incompletely due to an exceptional
                                                                          	// error in the task. That means the infrastructure layers executed the task
                                                                          	// completely, but the task self-reported that it failed in an exceptional
                                                                          	// way.
                                                                          	//
                                                                          	// DON'T USE THIS IN ANY NEW CODE. Instead, prefer InfraFailure.
                                                                          	Exception
                                                                          
                                                                          	// Expired if the component was never scheduled due to resource exhaustion.
                                                                          	Expired
                                                                          
                                                                          	// Canceled if the component had external intervention to stop it after it
                                                                          	// was scheduled, but before it completed on its own.
                                                                          	Canceled
                                                                          )

                                                                          func (Status) MarshalJSON

                                                                          func (s Status) MarshalJSON() ([]byte, error)

                                                                            MarshalJSON renders enums into String rather than an int when marshalling.

                                                                            func (Status) String

                                                                            func (i Status) String() string

                                                                            func (Status) Terminal

                                                                            func (s Status) Terminal() bool

                                                                              Terminal returns true if the step status won't change.

                                                                              type Summary

                                                                              type Summary struct {
                                                                              	// Status indicates the 'goodness' and lifetime of the thing. This usually
                                                                              	// translates directly to a status color.
                                                                              	Status Status
                                                                              
                                                                              	// Start indicates when this thing started doing its action.
                                                                              	Start time.Time
                                                                              
                                                                              	// End indicates when this thing completed doing its action.
                                                                              	End time.Time
                                                                              
                                                                              	// Text is a possibly-multi-line summary of what happened.
                                                                              	Text []string
                                                                              }

                                                                                Summary summarizes a thing (step, build, group of builds, whatever).