README
¶
QScheduler-Swarming
QScheduler-Swarming is an implementation of the ExternalScheduler API for swarming, using the quotascheduler algorithm.
This server runs on Kubernetes.
Code layout:
api/Definitions for the Admin API to QScheduler-Swarming.app/configDefinitions for QScheduler-Swarming configuration.app/eventlogBigQuery logging helper.app/frontendRequest handlers.app/stateCommon code for mutating scheduler state, and request-batcher implementation.app/state/metricsTime-series and analytics emission.app/state/nodestoreDatastore-backed store of scheduler state with in-memory cache, distributed over many write-only datastore entities.app/state/operationsState mutators for scheduling requests.cmd/qscheduler-swarmingEntry point for server.dockerContext directory for Docker (including Dockerfile)
Making changes
Submitted changes are automatically deployed to the staging instance by
Chrome Ops Kubernetes automation. It usually takes 5-10 min. To quickly check
what version is currently running, visit /healthz endpoint and look at the
image tag. It looks like ci-2020.04.10-30467-b307348. The last part is a
git revision of infra.git repository used to build the image from.
Once a change is verified on the staging, land a channels.json CL that makes
stable version equal to the current staging version there. It will result
in the production cluster update.
If you want to deploy an uncommitted change to the staging cluster, follow instructions in dev/k8s.star. Note that there's only one staging instance and you'll be fighting with the automation to deploy changes to it (unless the automation is disabled).
To make changes to Kubernetes configuration, modify files in projects/qscheduler in k8s.git repo. Changes are applied on commit. See Chrome Ops Kubernetes for all details.
Directories
¶
| Path | Synopsis |
|---|---|
|
api
|
|
|
qscheduler/v1
Package qscheduler contains service definitions for the qscheduler app.
|
Package qscheduler contains service definitions for the qscheduler app. |
|
app
|
|
|
config
Package config contains service configuration endpoints and data definition for qscheduler-swarming.
|
Package config contains service configuration endpoints and data definition for qscheduler-swarming. |
|
frontend
Package frontend exposes the primary pRPC API of qscheduler app.
|
Package frontend exposes the primary pRPC API of qscheduler app. |
|
state
Package state provides representations of the state of a quotascheduler app, as well as transformations that run on a state and ways to persist the state, and emits metrics about the state or its transitions.
|
Package state provides representations of the state of a quotascheduler app, as well as transformations that run on a state and ways to persist the state, and emits metrics about the state or its transitions. |
|
state/metrics
Package metrics provides functions to emit ts_mon and bq metrics.
|
Package metrics provides functions to emit ts_mon and bq metrics. |
|
state/nodestore
Package nodestore implements a datastore-backed persistent store of qscheduler state, that shards state over as many entities as necessary to stay under datastore's single-entity size limit, and uses an in-memory cache to avoid unnecessary datastore reads.
|
Package nodestore implements a datastore-backed persistent store of qscheduler state, that shards state over as many entities as necessary to stay under datastore's single-entity size limit, and uses an in-memory cache to avoid unnecessary datastore reads. |
|
state/nodestore/internal/blob
Package blob defines the proto format used by nodestore.
|
Package blob defines the proto format used by nodestore. |
|
cmd
|
|
|
qscheduler-swarming
Command qscheduler-swarming implements a qscheduler server process for GKE environment.
|
Command qscheduler-swarming implements a qscheduler server process for GKE environment. |