orla

module
v1.2.11 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 28, 2026 License: MIT

README

Orla

Go Report Card OpenSSF Best Practices Build pyorla CI pyorla on PyPI

Orla is a library for building and running LLM-based agentic systems. Modern agentic applications are workflows that combine multiple LLM calls, tool invocations, and heterogeneous infrastructure. Today, developers often stitch these pieces together manually using orchestration code, LLM serving engines, and tool execution logic.

Orla simplifies this process by separating workflow-level decisions from request execution. Developers define workflows as stages, while Orla handles how those stages are mapped to models and backends, scheduled and executed, and coordinated through shared inference state.

The system exposes three core components: a Stage Mapper for heterogeneous model routing, a Workflow Orchestrator for executing and scheduling stages, and a Memory Manager that manages KV cache across workflow stages.

Harvard University

Orla is a project of Dr. Minlan Yu's lab at Harvard SEAS.

Contributing

We welcome any and all open-source contributions to orla. Orla is designed to be a community-focused project and runs on individual contributions from amazing people around the world. This document provides guidelines and instructions for contributing to the project.

Getting Started

Orla CLI and API demo

Installing the orla daemon:

brew install --cask harvard-cns/orla/orla

Installing the orla client SDK:

pip install pyorla

Visit our website to learn more.

Citation

If you use Orla for your research, we would greatly appreciate it if you cite our demo paper.

@misc{shahout2026orlalibraryservingllmbased,
      title={Orla: A Library for Serving LLM-Based Multi-Agent Systems}, 
      author={Rana Shahout and Hayder Tirmazi and Minlan Yu and Michael Mitzenmacher},
      year={2026},
      eprint={2603.13605},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2603.13605}, 
}

Contacting Us

  • For technical questions and feature requests, please use GitHub Issues
  • For security disclosures, please see SECURITY.md.

Directories

Path Synopsis
cmd
orla command
examples
concurrent_stages_demo
Package concurrentstagesdemo demonstrates backend concurrency with parallel stages.
 Package concurrentstagesdemo demonstrates backend concurrency with parallel stages.
concurrent_stages_demo/cmd/concurrent_stages_demo command
Command concurrent_stages_demo runs the concurrent stages tutorial example.
 Command concurrent_stages_demo runs the concurrent stages tutorial example.
dag_math_eval/cmd/run command
Command dag_math_eval runs the DAG-Math memory management evaluation.
 Command dag_math_eval runs the DAG-Math memory management evaluation.
dag_math_eval/eval
Package eval runs the DAG-Math memory evaluation experiment.
 Package eval runs the DAG-Math memory evaluation experiment.
dag_math_eval/shared
Package shared provides common types and helpers for DAG-Math memory evaluation experiments.
 Package shared provides common types and helpers for DAG-Math memory evaluation experiments.
demo_video/cmd/part1 command
Command part1 runs Demo Part 1: Stage Mapping + FCFS + Cache Flushing.
 Command part1 runs Demo Part 1: Stage Mapping + FCFS + Cache Flushing.
demo_video/cmd/part2 command
Command part2 runs Demo Part 2: Stage Mapping + Priority + Cache Preserves.
 Command part2 runs Demo Part 2: Stage Mapping + Priority + Cache Preserves.
demo_video/cmd/part3 command
Command part3 runs Demo Part 3: Stage Mapping + Priority + Flush at Boundary (two workflows).
 Command part3 runs Demo Part 3: Stage Mapping + Priority + Flush at Boundary (two workflows).
demo_video/part1
Package part1 demonstrates the 5-min demo Part 1:
 Package part1 demonstrates the 5-min demo Part 1:
demo_video/part2
Package part2 demonstrates the 5-min demo Part 2:
 Package part2 demonstrates the 5-min demo Part 2:
demo_video/part3
Package part3 demonstrates the 5-min demo Part 3:
 Package part3 demonstrates the 5-min demo Part 3:
memory_manager_demo
Package memorymanagerdemo demonstrates the Memory Manager with a multi-stage workflow.
 Package memorymanagerdemo demonstrates the Memory Manager with a multi-stage workflow.
memory_manager_demo/cmd/memory_manager_demo command
Command memory_manager_demo runs the Memory Manager tutorial example.
 Command memory_manager_demo runs the Memory Manager tutorial example.
swe_bench_lite/cmd/single_shot command
Command single_shot runs the SWE-bench Lite single-shot experiment.
 Command single_shot runs the SWE-bench Lite single-shot experiment.
swe_bench_lite/shared
Package shared provides common types and helpers for SWE-bench Lite experiments.
 Package shared provides common types and helpers for SWE-bench Lite experiments.
swe_bench_lite/single_shot
Package singleshot runs the SWE-bench Lite single-shot experiment: each instance gets one inference call with oracle-provided source files.
 Package singleshot runs the SWE-bench Lite single-shot experiment: each instance gets one inference call with oracle-provided source files.
workflow_demo
Package workflowdemo demonstrates the full Orla abstraction stack: Workflow -> Stage DAG -> StageMapping -> Scheduling.
 Package workflowdemo demonstrates the full Orla abstraction stack: Workflow -> Stage DAG -> StageMapping -> Scheduling.
workflow_demo/cmd/workflow_demo command
Command workflow_demo runs the customer support workflow demo.
 Command workflow_demo runs the customer support workflow demo.
internal
agent
Package agent implements the agent loop and MCP client for Orla Agent Mode (RFC 4).
 Package agent implements the agent loop and MCP client for Orla Agent Mode (RFC 4).
config
Package config provides configuration management for Orla: load from a single config file path, or use defaults when path is empty.
 Package config provides configuration management for Orla: load from a single config file path, or use defaults when path is empty.
core
Package core implements the core functionality for orla that is shared across all components.
 Package core implements the core functionality for orla that is shared across all components.
model
Package model provides the Provider interface and implementations.
 Package model provides the Provider interface and implementations.
serving
Package serving implements a minimal programmatic serving layer.
 Package serving implements a minimal programmatic serving layer.
serving/api
Package api provides the HTTP API for the serving layer daemon.
 Package api provides the HTTP API for the serving layer daemon.
serving/cost
Package cost provides helpers for token-based cost estimation.
 Package cost provides helpers for token-based cost estimation.
serving/memory
Package memory implements the Memory Manager for Orla's agentic serving layer.
 Package memory implements the Memory Manager for Orla's agentic serving layer.
serving/metrics
Package metrics provides Prometheus metrics for the Orla serving layer.
 Package metrics provides Prometheus metrics for the Orla serving layer.
testing
Package testing provides default values and utilities for testing orla.
 Package testing provides default values and utilities for testing orla.
tui
Package tui provides terminal UI utilities using charmbracelet libraries.
 Package tui provides terminal UI utilities using charmbracelet libraries.
pkg
api
Package orla provides a basic public Go client library for Orla server.
 Package orla provides a basic public Go client library for Orla server.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.