The Token Server

This directory contains an implementation of a service that generates and validates various tokens used in LUCI authentication protocol.

In particular, this service implements so called "machine tokens" used for authenticating Swarming bots:

  1. Each bot has a TLS private key and a certificate, signed by some trusted CA.
  2. luci_machine_tokend executable periodically runs and uses the private key and certificate when calling MintMachineToken gRPC method of the token server.
  3. The server verifies that the certificate is signed by a trusted CA, that it is not expired or revoked, and that the request was signed by the corresponding private key. If everything checks out, the server generates a short lived (1h by default) stateless machine token (basically, certificate Common Name and some additional data signed by the token server's own private key).
  4. The bot uses this token when sending requests to Swarming (by putting it into X-Luci-Machine-Token header).
  5. Swarming checks the signature of the token (using only local crypto) when authenticating requests from bots.


  • api: gRPC protocol definition and autogenerated Go code.
  • appengine: server implementation (runs on Standard GAE).
  • auth/machine: implementation of the token checking logic that can be used by backends that want to use machine tokens. Swarming service uses same logic (implemented in Python).
  • client: library that wraps TokenMinter gRPC API into a usable form. It implements logic for reading and using TLS certificate and private keys.
  • cmd/luci_machine_tokend: executable deployed on all bots. It knows how to generate machine tokens given a TLS certificate and private key.
  • testing: local integration test that checks interaction of luci_machine_tokend with the server (and some other things, such as certificate revocation list updates).
Expand ▾ Collapse ▴


Path Synopsis
api Package tokenserver contains common protobuf messages for the token server.
api/admin/v1 Package admin contains The Token Server Administrative and Config API.
api/bq Package bq contains BigQuery tables schemas.
api/minter/v1 Package minter contains the main API of the token server.
appengine/backend Binary backend implements HTTP server that handles requests to 'backend' module.
appengine/frontend Binary frontend implements HTTP server that handles requests to 'default' module.
appengine/impl Package impl holds code to initialize the server environment.
appengine/impl/certchecker Package certchecker contains implementation of CertChecker.
appengine/impl/certconfig Package certconfig contains code to work with imported CAs and their CRLs.
appengine/impl/machinetoken Package machinetoken implements generation of LUCI machine tokens.
appengine/impl/serviceaccountsv2 Package serviceaccountsv2 implements MintServiceAccountToken RPC.
appengine/impl/services/admin/adminsrv Package adminsrv implements Admin API.
appengine/impl/services/admin/certauthorities Package certauthorities implements CertificateAuthorities API.
appengine/impl/services/minter/tokenminter Package tokenminter implements TokenMinter API.
appengine/impl/utils Package utils contains a variety of small utility functions used by other tokenserver packages.
appengine/impl/utils/bq Package bq contains helpers for uploading rows to BigQuery.
appengine/impl/utils/identityset Package identityset implements a set-like structure for identity.Identity.
appengine/impl/utils/policy Package policy contains implementation of Policy parsing and querying.
appengine/impl/utils/revocation Package revocation contains utilities for implementing token revocation.
appengine/impl/utils/shards Package shards provides a low level support for implementing sharded set of []byte blobs.
appengine/impl/utils/tokensigning Package tokensigning implements utilities for RSA-signing of proto messages.
auth/machine Package machine implements authentication based on LUCI machine tokens.
client Package client implements pRPC client for The Token Server.
cmd/luci_machine_tokend Command luci_machine_tokend runs on all machines via cron.