gce/

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

chromium.googlesource.com/infra/luci/luci-go

Links

Open Source Insights

README ¶

GCE Provider Service

This service is responsible for managing the lifecycle of Google Compute Engine (GCE) Virtual Machines, primarily for use as Swarming bots.

Configuration

The service uses configuration files to define VM shapes, disk images, network settings, and swarming pool assignments.

Proto Definitions: The core configuration schema is defined in api/config/v1/config.proto. This includes messages for VM, Disk, NetworkInterface, etc.
User Configs: Current user configurations are located in the infra_superproject repository at data/config/configs/gce-provider/. Older configs can also be seen at go/gce-provider-config.

Regeneration After Proto Changes

When you modify api/config/v1/config.proto, you MUST regenerate the Go bindings by running:

go generate ./...

from within the go.chromium.org/luci/gce/api/config/v1 directory. This will update the *.pb.go and *.pb.discovery.go files.

Service Architecture

The service consists of several components:

appengine/frontend: Handles incoming API requests.
appengine/backend: Contains cron jobs and task queue handlers for:
- Creating new VM instances.
- Deleting terminated or orphaned instances.
- Auditing existing instances against the configuration.
- Managing instance deadlines and drainage.
Task Queues: The service utilizes Cloud Tasks to manage asynchronous operations like instance creation and deletion.

Deployment

The service is deployed on Google App Engine in the following GCP projects:

Production: gce-provider
Development: gce-provider-dev

Directories ¶

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

Path	Synopsis
api
affiliations/v1 Package affiliations contains VM affiliations config definitions.	Package affiliations contains VM affiliations config definitions.
bigquery/v1 Package bigquery contains BigQuery table schema definitions.	Package bigquery contains BigQuery table schema definitions.
config/v1 Package config contains VM config definitions.	Package config contains VM config definitions.
instances/v1 Package instances contains a GCE instances service.	Package instances contains a GCE instances service.
projects/v1 Package projects contains GCP project config definitions.	Package projects contains GCP project config definitions.
quotas/v1 Package quotas contains quota definitions.	Package quotas contains quota definitions.
tasks/v1 Package tasks contains task queue task definitions.	Package tasks contains task queue task definitions.
appengine
backend Package backend includes cron and task queue handlers.	Package backend includes cron and task queue handlers.
backend/internal/metrics Package metrics includes tsmon metric support.	Package metrics includes tsmon metric support.
config Package config knows how import configs from LUCI-config.	Package config knows how import configs from LUCI-config.
convert/toalpha toalpha contains functions that convert stable cloud protos to alpha cloud protos.	toalpha contains functions that convert stable cloud protos to alpha cloud protos.
frontend command Package main is the main entry point for the app.	Package main is the main entry point for the app.
model Package model contains datastore model definitions.	Package model contains datastore model definitions.
rpc Package rpc contains RPC server implementations.	Package rpc contains RPC server implementations.
rpc/internal/metrics Package metrics is tsmon metrics for rpc API	Package metrics is tsmon metrics for rpc API
rpc/memory Package memory contains a RPC servers backed by in-memory storage.	Package memory contains a RPC servers backed by in-memory storage.
testing/roundtripper Package roundtripper contains http.RoundTripper implementations suitable for testing.	Package roundtripper contains http.RoundTripper implementations suitable for testing.
cmd
agent command Package main is generated by go.chromium.org/luci/tools/cmd/assets.	Package main is generated by go.chromium.org/luci/tools/cmd/assets.
vmtoken Package vmtoken implements parsing and verification of signed GCE VM metadata tokens.	Package vmtoken implements parsing and verification of signed GCE VM metadata tokens.
client Package client implements client-side fetch and transmission of signed GCE VM metadata tokens.	Package client implements client-side fetch and transmission of signed GCE VM metadata tokens.