Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func HandleReverse

func HandleReverse(w http.ResponseWriter, r *http.Request)

    HandleReverse handles reverse buildlet connections.

    func HasScope

    func HasScope(want string) bool

      HasScope returns true if the GCE metadata contains the default scopes.

      func InitGCE

      func InitGCE(sc *secret.Client, vmDeleteTimeout time.Duration, tFiles map[string]string, basePin *atomic.Value, fn IsRemoteBuildletFunc, buildEnvName, mode string) error

        InitGCE initializes the GCE buildlet pool.

        func InitKube

        func InitKube(monitorGitMirror MonitorGitMirrorFunc) error

          InitGCE must be called before initKube

          func KubeErr

          func KubeErr() error

            KubeErr retrieves the kube error value.

            func KubeGoClient

            func KubeGoClient() *kubernetes.Client

              KubeGoClient retrieves a kube client for the go cluster.

              func KubePool

              func KubePool() *kubeBuildletPool

                KubePool returns the kube buildlet pool.

                func KubeSetErr

                func KubeSetErr(err error)

                  KubeSetErr sets the kube error to passed in value.

                  func ReadGCSFile

                  func ReadGCSFile(name string) ([]byte, error)

                    ReadGCSFile reads the named file from the GCS bucket.

                    func SetBuilderMasterKey

                    func SetBuilderMasterKey(masterKey []byte)

                      SetBuilderMasterKey sets the builder master key used to generate keys used by the builders.

                      Types

                      type Buildlet

                      type Buildlet interface {
                      	// GetBuildlet returns a new buildlet client.
                      	//
                      	// The hostType is the key into the dashboard.Hosts
                      	// map (such as "host-linux-jessie"), NOT the buidler type
                      	// ("linux-386").
                      	//
                      	// Users of GetBuildlet must both call Client.Close when done
                      	// with the client as well as cancel the provided Context.
                      	//
                      	// The ctx may have context values of type buildletTimeoutOpt
                      	// and highPriorityOpt.
                      	GetBuildlet(ctx context.Context, hostType string, lg Logger) (*buildlet.Client, error)
                      
                      	String() string // TODO(bradfitz): more status stuff
                      }

                        Buildlet defines an interface for a pool of buildlets.

                        type BuildletTimeoutOpt

                        type BuildletTimeoutOpt struct{} // context Value key; value is time.Duration
                        

                          BuildletTimeoutOpt is a context.Value key for BuildletPool.GetBuildlet.

                          type ByCreationTime

                          type ByCreationTime []ResourceTime

                            ByCreationTime provides the functionality to sort resource times by the time of creation.

                            func (ByCreationTime) Len

                            func (s ByCreationTime) Len() int

                            func (ByCreationTime) Less

                            func (s ByCreationTime) Less(i, j int) bool

                            func (ByCreationTime) Swap

                            func (s ByCreationTime) Swap(i, j int)

                            type EC2Buildlet

                            type EC2Buildlet struct {
                            	// contains filtered or unexported fields
                            }

                              EC2Buildlet manages a pool of AWS EC2 buildlets.

                              func EC2BuildetPool

                              func EC2BuildetPool() *EC2Buildlet

                                EC2BuildetPool retrieves the package level EC2Buildlet pool set by the constructor.

                                TODO(golang.org/issues/38337) remove once a package level variable is no longer required by the main package.

                                func NewEC2Buildlet

                                func NewEC2Buildlet(client *cloud.AWSClient, buildEnv *buildenv.Environment, hosts map[string]*dashboard.HostConfig, fn IsRemoteBuildletFunc, opts ...EC2Opt) (*EC2Buildlet, error)

                                  NewEC2Buildlet creates a new EC2 buildlet pool used to create and manage the lifecycle of EC2 buildlets. Information about ARM64 instance types is retrieved before starting the pool. EC2 quota types are also retrieved before starting the pool. The pool will continuously poll for quotas which limit the resources that can be consumed by the pool. It will also periodically search for VMs which are no longer in use or are untracked by the pool in order to delete them.

                                  func (*EC2Buildlet) Close

                                  func (eb *EC2Buildlet) Close()

                                    Close stops the pollers used by the EC2Buildlet pool from running.

                                    func (*EC2Buildlet) GetBuildlet

                                    func (eb *EC2Buildlet) GetBuildlet(ctx context.Context, hostType string, lg Logger) (*buildlet.Client, error)

                                      GetBuildlet retrieves a buildlet client for a newly created buildlet.

                                      func (*EC2Buildlet) String

                                      func (eb *EC2Buildlet) String() string

                                        String gives a report of capacity usage for the EC2 buildlet pool.

                                        func (*EC2Buildlet) WriteHTMLStatus

                                        func (eb *EC2Buildlet) WriteHTMLStatus(w io.Writer)

                                          WriteHTMLStatus writes the status of the EC2 buildlet pool to an io.Writer.

                                          type EC2Opt

                                          type EC2Opt func(*EC2Buildlet)

                                            EC2Opt is optional configuration for the the buildlet.

                                            func WithVMDeleteTimeout

                                            func WithVMDeleteTimeout(timeout time.Duration) EC2Opt

                                              WithVMDeleteTimeout sets the VM deletion timeout for all EC2 VMs.

                                              type EventTimeLogger

                                              type EventTimeLogger interface {
                                              	LogEventTime(event string, optText ...string)
                                              }

                                                EventTimeLogger is the logging interface used to log an event at a point in time.

                                                type GCEBuildlet

                                                type GCEBuildlet struct {
                                                	// contains filtered or unexported fields
                                                }

                                                  GCEBuildlet manages a pool of GCE buildlets.

                                                  func (*GCEBuildlet) CleanUpOldVMs

                                                  func (p *GCEBuildlet) CleanUpOldVMs()

                                                    CleanUpOldVMs loops forever and periodically enumerates virtual machines and deletes those which have expired.

                                                    A VM is considered expired if it has a "delete-at" metadata attribute having a unix timestamp before the current time.

                                                    This is the safety mechanism to delete VMs which stray from the normal deleting process. VMs are created to run a single build and should be shut down by a controlling process. Due to various types of failures, they might get stranded. To prevent them from getting stranded and wasting resources forever, we instead set the "delete-at" metadata attribute on them when created to some time that's well beyond their expected lifetime.

                                                    func (*GCEBuildlet) GetBuildlet

                                                    func (p *GCEBuildlet) GetBuildlet(ctx context.Context, hostType string, lg Logger) (bc *buildlet.Client, err error)

                                                      GetBuildlet retrieves a buildlet client for an available buildlet.

                                                      func (*GCEBuildlet) SetEnabled

                                                      func (p *GCEBuildlet) SetEnabled(enabled bool)

                                                        SetEnabled marks the buildlet pool as enabled.

                                                        func (*GCEBuildlet) String

                                                        func (p *GCEBuildlet) String() string

                                                        func (*GCEBuildlet) WriteHTMLStatus

                                                        func (p *GCEBuildlet) WriteHTMLStatus(w io.Writer)

                                                          WriteHTMLStatus writes the status of the buildlet pool to an io.Writer.

                                                          type GCEConfiguration

                                                          type GCEConfiguration struct{}

                                                            GCEConfiguration manages and contains all of the GCE configuration.

                                                            func NewGCEConfiguration

                                                            func NewGCEConfiguration() *GCEConfiguration

                                                              NewGCEConfiguration creates a new GCEConfiguration.

                                                              func (*GCEConfiguration) BuildEnv

                                                              func (c *GCEConfiguration) BuildEnv() *buildenv.Environment

                                                                BuildEnv retrieves the GCE build env.

                                                                func (*GCEConfiguration) BuildletPool

                                                                func (c *GCEConfiguration) BuildletPool() *GCEBuildlet

                                                                  BuildletPool retrieves the GCE buildlet pool.

                                                                  func (*GCEConfiguration) DSClient

                                                                  func (c *GCEConfiguration) DSClient() *datastore.Client

                                                                    DSClient retrieves the datastore client.

                                                                    func (*GCEConfiguration) ErrorsClient

                                                                    func (c *GCEConfiguration) ErrorsClient() *errorreporting.Client

                                                                      ErrorsClient retrieves the stackdriver errors client.

                                                                      func (*GCEConfiguration) GCPCredentials

                                                                      func (c *GCEConfiguration) GCPCredentials() *google.Credentials

                                                                        GCPCredentials retrieves the GCP credentials.

                                                                        func (*GCEConfiguration) GKENodeIP

                                                                        func (c *GCEConfiguration) GKENodeIP() string

                                                                          GKENodeIP retrieves the GKE node IP.

                                                                          func (*GCEConfiguration) GerritClient

                                                                          func (c *GCEConfiguration) GerritClient() *gerrit.Client

                                                                            GerritClient retrieves a gerrit client.

                                                                            func (*GCEConfiguration) GoDSClient

                                                                            func (c *GCEConfiguration) GoDSClient() *datastore.Client

                                                                              GoDSClient retrieves the datastore client for golang.org project.

                                                                              func (*GCEConfiguration) InStaging

                                                                              func (c *GCEConfiguration) InStaging() bool

                                                                                InStaging returns a boolean denoting if the enviornment is stageing.

                                                                                func (*GCEConfiguration) MetricsClient

                                                                                func (c *GCEConfiguration) MetricsClient() *monapi.MetricClient

                                                                                  MetricsClient retrieves a metrics client.

                                                                                  func (*GCEConfiguration) OAuthHTTPClient

                                                                                  func (c *GCEConfiguration) OAuthHTTPClient() *http.Client

                                                                                    OAuthHTTPClient retrieves an OAuth2 HTTP client used to make API calls to GCP.

                                                                                    func (*GCEConfiguration) SetBuildEnv

                                                                                    func (c *GCEConfiguration) SetBuildEnv(b *buildenv.Environment)

                                                                                      SetBuildEnv sets the GCE build env. This is primarily reserved for testing purposes.

                                                                                      func (*GCEConfiguration) StorageClient

                                                                                      func (c *GCEConfiguration) StorageClient() *storage.Client

                                                                                        StorageClient retrieves the GCE storage client.

                                                                                        func (*GCEConfiguration) TryDepsErr

                                                                                        func (c *GCEConfiguration) TryDepsErr() error

                                                                                          TryDepsErr retrives any Trybot dependency error.

                                                                                          type IsRemoteBuildletFunc

                                                                                          type IsRemoteBuildletFunc func(instanceName string) bool

                                                                                            IsRemoteBuildletFunc should report whether the buildlet instance name is is a remote buildlet. This is applicable to GCE and EC2 instances.

                                                                                            TODO(golang.org/issue/38337): should be removed once remote buildlet management functions are moved into a package.

                                                                                            type Logger

                                                                                            type Logger interface {
                                                                                            	EventTimeLogger // point in time
                                                                                            	spanlog.Logger  // action spanning time
                                                                                            }

                                                                                              Logger is the logging interface used within the coordinator. It can both log a message at a point in time, as well as log a span (something having a start and end time, as well as a final success status).

                                                                                              type MonitorGitMirrorFunc

                                                                                              type MonitorGitMirrorFunc func()

                                                                                                MonitorGitMirrorFunc defines a function used to monitor gitmirror.

                                                                                                type ResourceTime

                                                                                                type ResourceTime struct {
                                                                                                	Name     string
                                                                                                	Creation time.Time
                                                                                                }

                                                                                                  ResourceTime is a GCE instance or Kube pod name and its creation time.

                                                                                                  type ReverseBuildletPool

                                                                                                  type ReverseBuildletPool struct {
                                                                                                  	// contains filtered or unexported fields
                                                                                                  }

                                                                                                    ReverseBuildletPool manages the pool of reverse buildlet pools.

                                                                                                    func ReversePool

                                                                                                    func ReversePool() *ReverseBuildletPool

                                                                                                      ReversePool retrieves the reverse buildlet pool.

                                                                                                      func (*ReverseBuildletPool) BuildReverseStatusJSON

                                                                                                      func (p *ReverseBuildletPool) BuildReverseStatusJSON() *types.ReverseBuilderStatus

                                                                                                        BuildReverseStatusJSON is an HTTP handler implementation which builds the reverse status reverse buildlets.

                                                                                                        func (*ReverseBuildletPool) BuildletHostnames

                                                                                                        func (p *ReverseBuildletPool) BuildletHostnames() []string

                                                                                                          BuildletHostnames returns a slice of reverse buildlet hostnames.

                                                                                                          func (*ReverseBuildletPool) BuildletLastSeen

                                                                                                          func (p *ReverseBuildletPool) BuildletLastSeen(host string) (time.Time, bool)

                                                                                                            BuildletLastSeen gives the last time a buildlet was connected to the pool. If the buildlet has not been seen a false is returned by the boolean.

                                                                                                            func (*ReverseBuildletPool) CanBuild

                                                                                                            func (p *ReverseBuildletPool) CanBuild(hostType string) bool

                                                                                                              CanBuild reports whether the pool has a machine capable of building mode, even if said machine isn't currently idle.

                                                                                                              func (*ReverseBuildletPool) GetBuildlet

                                                                                                              func (p *ReverseBuildletPool) GetBuildlet(ctx context.Context, hostType string, lg Logger) (*buildlet.Client, error)

                                                                                                                GetBuildlet builds a buildlet client for the passed in host.

                                                                                                                func (*ReverseBuildletPool) HostTypeCount

                                                                                                                func (p *ReverseBuildletPool) HostTypeCount() map[string]int

                                                                                                                  HostTypeCount iterates through the running reverse buildlets, and constructs a count of running buildlets per hostType.

                                                                                                                  func (*ReverseBuildletPool) HostTypes

                                                                                                                  func (p *ReverseBuildletPool) HostTypes() (types []string)

                                                                                                                    HostTypes returns the a deduplicated list of buildlet types curently supported by the pool.

                                                                                                                    func (*ReverseBuildletPool) ServeReverseStatusJSON

                                                                                                                    func (p *ReverseBuildletPool) ServeReverseStatusJSON(w http.ResponseWriter, r *http.Request)

                                                                                                                      ServeReverseStatusJSON is an HTTP handler implementation which serves the status in JSON format.

                                                                                                                      func (*ReverseBuildletPool) SingleHostTypeCount

                                                                                                                      func (p *ReverseBuildletPool) SingleHostTypeCount(hostType string) int

                                                                                                                        SingleHostTypeCount iterates through the running reverse buildlets, and constructs a count of the running buildlet hostType requested.

                                                                                                                        func (*ReverseBuildletPool) String

                                                                                                                        func (p *ReverseBuildletPool) String() string

                                                                                                                        func (*ReverseBuildletPool) WriteHTMLStatus

                                                                                                                        func (p *ReverseBuildletPool) WriteHTMLStatus(w io.Writer)

                                                                                                                          WriteHTMLStatus writes a status of the reverse buildlet pool, in HTML format,

                                                                                                                          to the passed in io.Writer.
                                                                                                                          
                                                                                                                          GOOS=linux, GOARCH=amd64