armada

module
v0.3.95 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 9, 2023 License: Apache-2.0

README

CircleCI Go Report Card

Armada

Armada is a multi-Kubernetes cluster batch job scheduler.

Armada is designed to address the following issues:

  1. A single Kubernetes cluster can not be scaled indefinitely, and managing very large Kubernetes clusters is challenging. Hence, Armada is a multi-cluster scheduler built on top of several Kubernetes clusters.
  2. Acheiving very high throughput using the in-cluster storage backend, etcd, is challenging. Hence, queueing and scheduling is performed partly out-of-cluster using a specialized storage layer.

Armada is designed primarily for machine learning, AI, and data analytics workloads, and to:

  • Manage compute clusters composed of tens of thousands of nodes in total.
  • Schedule a thousand or more pods per second, on average.
  • Enqueue tens of thousands of jobs over a few seconds.
  • Divide resources fairly between users.
  • Provide visibility for users and admins.
  • Ensure near-constant uptime.

Armada is a CNCF Sandbox project used in production at G-Research.

For an overview of Armada, see these videos:

Armada adheres to the CNCF Code of Conduct.

Documentation

For an overview of the architecture and design of Armada, and instructions for submitting jobs, see:

For a full developer guide, see:

For API reference, see:

We expect readers of the documentation to have a basic understanding of Docker and Kubernetes; see, e.g., the following links:

Contributions

Thank you for considering contributing to Armada! We want everyone to feel that they can contribute to the Armada Project. Your contributions are valuable, whether it's fixing a bug, implementing a new feature, improving documentation, or suggesting enhancements. We appreciate your time and effort in helping make this project better for everyone. For more information about contributing to Armada see CONTRIBUTING.md and before proceeding to contributions see CODE_OF_CONDUCT.md

Discussion

If you are interested in discussing Armada you can find us on slack

Directories

Path Synopsis
cmd
internal
armadactl
Package armadactl contains all the business logic for armadactl.
Package armadactl contains all the business logic for armadactl.
common/armadaerrors
Package armadaerrors contains generic errors that should be returned by code handling gRPC requests.
Package armadaerrors contains generic errors that should be returned by code handling gRPC requests.
common/grpc/grpcpool
Package grpcpool provides a pool of grpc clients
Package grpcpool provides a pool of grpc clients
common/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
lookoutv2/gen/restapi
Package restapi Lookout v2 API
Package restapi Lookout v2 API
scheduler/mocks
Package schedulermocks is a generated GoMock package.
Package schedulermocks is a generated GoMock package.
testsuite/build
Package build info
Package build info
testsuite/eventwatcher
Utility for watching events.
Utility for watching events.
pkg
api
Package api is a reverse proxy.
Package api is a reverse proxy.
api/binoculars
Package binoculars is a reverse proxy.
Package binoculars is a reverse proxy.
api/lookout
Package lookout is a reverse proxy.
Package lookout is a reverse proxy.
scripts
testsuite

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL