README
¶
Orla is a runtime-adaptive execution layer for agentic workflows. Modern agentic applications combine many LLM calls, tool invocations, and heterogeneous backends, and developers usually wire these together by hand.
You point your agent at Orla as an OpenAI-compatible endpoint and tag each call with a stage, what the call is for rather than where to send it. Orla serves the call through the stage's current policy, records what happened, and lets an external optimizer update that policy as it learns. This loop tunes each stage's policy for cost and accuracy from production data, with no changes to your agent's code.
Three components participate. The agent issues stage-tagged calls and rates each result. Orla, the execution layer in the middle, applies each stage's policy to incoming calls and holds the shared record. The optimizer reads that record and revises a stage's policy whenever the data points to a better one, from a simple backend swap to a cheap-first rule that escalates to a stronger model only when a task fails.
Orla is a project of Dr. Minlan Yu's lab at Harvard SEAS.
Contributing
We welcome any and all open-source contributions to orla. Orla is designed to be a community-focused project and runs on individual contributions from amazing people around the world. This document provides guidelines and instructions for contributing to the project.
Getting Started
Install the orla daemon from source:
go install github.com/harvard-cns/orla/cmd/orla@latest
Or build it locally:
git clone https://github.com/harvard-cns/orla
cd orla
go build -o bin/orla ./cmd/orla
For your first stage and a tour of the runtime adaptation loop, see docs/quickstart.md.
Citation
If you use Orla for your research, we would greatly appreciate it if you cite our demo paper.
@misc{shahout2026orlalibraryservingllmbased,
title={Orla: A Library for Serving LLM-Based Multi-Agent Systems},
author={Rana Shahout and Hayder Tirmazi and Minlan Yu and Michael Mitzenmacher},
year={2026},
eprint={2603.13605},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2603.13605},
}
Contacting Us
- For technical questions and feature requests, please use GitHub Issues
- For security disclosures, please see SECURITY.md.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
orla
command
Command orla is the daemon entry point.
|
Command orla is the daemon entry point. |
|
orla/commands
Package commands defines the cobra command tree rooted at "orla".
|
Package commands defines the cobra command tree rooted at "orla". |
|
orlactl
command
Command orlactl is the control-plane CLI for the orla daemon.
|
Command orlactl is the control-plane CLI for the orla daemon. |
|
internal
|
|
|
api
Package api hosts the HTTP server, middleware stack, and request handlers.
|
Package api hosts the HTTP server, middleware stack, and request handlers. |
|
backends
Package backends owns the backend record and the registry that persists it.
|
Package backends owns the backend record and the registry that persists it. |
|
config
Package config loads the orla daemon configuration from environment variables prefixed with ORLA_.
|
Package config loads the orla daemon configuration from environment variables prefixed with ORLA_. |
|
costs
Package costs polls backend cost sources and holds the live prices.
|
Package costs polls backend cost sources and holds the live prices. |
|
mappings
Package mappings owns mapping variants and the registry that persists them.
|
Package mappings owns mapping variants and the registry that persists them. |
|
metrics
Package metrics owns the Prometheus metric definitions and the custom collectors that pull point-in-time gauges from the scheduler and the BatchWriter instances.
|
Package metrics owns the Prometheus metric definitions and the custom collectors that pull point-in-time gauges from the scheduler and the BatchWriter instances. |
|
orlaclient
Package orlaclient is the HTTP client and command tree for orlactl, the orla control-plane CLI.
|
Package orlaclient is the HTTP client and command tree for orlactl, the orla control-plane CLI. |
|
provider
Package provider integrates the backends orla dispatches to.
|
Package provider integrates the backends orla dispatches to. |
|
scheduler
Package scheduler owns one FCFS executor per registered backend.
|
Package scheduler owns one FCFS executor per registered backend. |
|
settings
Package settings stores daemon-wide operational settings that the control plane manages at runtime, as opposed to the bootstrap configuration in package config that is fixed at process start.
|
Package settings stores daemon-wide operational settings that the control plane manages at runtime, as opposed to the bootstrap configuration in package config that is fixed at process start. |
|
stages
Package stages owns the platform-engineer-facing stage record and the registry that persists it.
|
Package stages owns the platform-engineer-facing stage record and the registry that persists it. |
|
storage
Package storage opens the Postgres connection pool, runs goose migrations, and exposes both a pgx-native pool (for sqlc-generated queries) and a *sql.DB adapter (for migrations and any future code that prefers database/sql).
|
Package storage opens the Postgres connection pool, runs goose migrations, and exposes both a pgx-native pool (for sqlc-generated queries) and a *sql.DB adapter (for migrations and any future code that prefers database/sql). |
|
telemetry
Package telemetry owns the data-plane writers (completion records, feedback) and the aggregation queries the mapper reads.
|
Package telemetry owns the data-plane writers (completion records, feedback) and the aggregation queries the mapper reads. |
|
wire
Package wire holds the JSON request and response types for orla's control-plane HTTP API.
|
Package wire holds the JSON request and response types for orla's control-plane HTTP API. |