throttle-proxy

command module
v0.1.28 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 21, 2025 License: MIT Imports: 15 Imported by: 0

README ยถ

Latest Release Go Report Card Go Code reference

Prometheus Backpressure Proxy

Adaptive Protection for Your Backend Services

๐Ÿ›ก๏ธ Dynamically shield your services from traffic overload using smart, metrics-driven congestion control.

See Netflix concurrency-limits for the inspiration.

Why Use This Project?

Throttle Proxy is the best solution for ensuring the safety and stability of your distributed systems during load spikes. Here's why:

  • Proven Algorithm: Utilizes the Additive Increase/Multiplicative Decrease (AIMD) algorithm, inspired by TCP congestion control, to dynamically adjust request concurrency.
  • Real-Time Metrics: Leverages Prometheus metrics to make real-time decisions, ensuring your system adapts quickly to changing loads.
  • Configurable and Flexible: Allows you to set custom thresholds and monitor multiple signals, providing fine-grained control over your traffic management.
  • Prevents Failures: Helps prevent cascading failures and maintains system stability under unpredictable load conditions.

Key Features

  • ๐Ÿ“Š Adaptive Traffic Management: Automatically adjusts request concurrency based on real-time Prometheus metrics
  • ๐Ÿ”€ Smart Scaling: Uses Additive Increase/Multiplicative Decrease (AIMD) algorithm
  • ๐Ÿšฆ Configurable Limits: Set min and max concurrent request thresholds
  • ๐Ÿ” Multi-Signal Monitoring: Track system health across multiple metrics simultaneously

Quick Example

config := proxymw.BackpressureConfig{
    EnableBackpressure: true,
    BackpressureQueries: []BackpressureQuery{
        {
            Query:              `sum(rate(http_server_errors_total[5m]))`,
            // Start to throttle when error rate reaches 50%
            WarningThreshold:   0.5,
            // Hard throttling up to 100 CongestionWindowMax when error rate is >80%
            EmergencyThreshold: 0.8,
        }
    },
    CongestionWindowMin: 10,
    CongestionWindowMax: 100,
}

How It Works

  1. ๐Ÿ”ญ Continuously monitor system metrics
  2. ๐Ÿ“ˆ Dynamically adjust request throughput
  3. ๐Ÿ›‘ Automatically throttle when system stress detected

When to Use

  • Protecting microservices from sudden traffic spikes
  • Preventing cascading failures
  • Maintaining system stability under unpredictable load

Quick Start

  1. Configure backpressure queries as Prometheus metrics
  2. Define min/max request windows
  3. Choose the server-side http proxy or client-side roundtripper
  4. Import the starter Grafana dashboard
  5. Let the proxy handle the rest!

Development

Installation

Build the docker-compose stack

make all
docker compose down
docker compose up --build
Lint and Test
make test
make lintfix
Upgrade dependencies
make deps
Contributing

CONTRIBUTING.md

Documentation ยถ

Overview ยถ

Package throttleproxy provides an adaptive backpressure proxy mechanism for dynamically managing traffic and protecting backend services using Prometheus metrics.

Usage Example:

	config := proxymw.BackpressureConfig{
	    EnableBackpressure: true,
	    BackpressureQueries: []BackpressureQuery{
	        {
	            Query:              `sum(rate(http_server_errors_total[5m]))`,
                Name:               "http_error_rate"
	            WarningThreshold:   0.5,
	            EmergencyThreshold: 0.8,
	        }
	    },
	    CongestionWindowMin: 10,
	    CongestionWindowMax: 100,
	}

Use Cases:

  • Protecting microservices from traffic spikes
  • Preventing cascading failures
  • Maintaining system stability under unpredictable load

The package supports both server-side HTTP proxy and client-side RoundTripper implementations, providing flexible integration options.

Directories ยถ

Path Synopsis
examples module
internal
util
Package util holds custom structs and functions to handle common operations
Package util holds custom structs and functions to handle common operations
Package proxymw holds interfaces and configuration to safeguard backend services from dynamic load
Package proxymw holds interfaces and configuration to safeguard backend services from dynamic load
Package proxyutil handles parsing logic for proxymw configs
Package proxyutil handles parsing logic for proxymw configs
proxyhttp
Package proxyhttp provides an HTTP mux handler for serving requests using httputil.ReverseProxy
Package proxyhttp provides an HTTP mux handler for serving requests using httputil.ReverseProxy

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL