lb

README

Load Balancer (lb)

The lb package provides high-performance Layer 4 (L4) and Layer 7 (L7) load balancers for both HTTP and gRPC traffic. It supports dynamic backend management, custom routing rules, health checks, and reflection service aggregation.

Features

• Layer 4 Load Balancing: Simple round-robin or random distribution across backends
• Layer 7 Load Balancing: Advanced routing based on paths, headers, methods, and gRPC services
• Dynamic Management: Add backends and routes at runtime via HTTP and gRPC APIs
• Health Checks: Built-in gRPC health checking support
• Reflection Aggregation: Combines reflection services from all backends (v1 and v1alpha)
• Protocol Support: HTTP and gRPC load balancing with protocol-specific optimizations

Architecture

lb/
├── backend/       # Backend server definitions
├── common/        # Shared utilities
├── grpc/          # gRPC load balancers
│   ├── l4/        # Layer 4 (connection-level) load balancing
│   ├── l7/        # Layer 7 (request-level) load balancing
│   └── reflection/# Aggregated reflection service
├── http/          # HTTP load balancers
│   ├── l4/        # Layer 4 HTTP load balancing
│   └── l7/        # Layer 7 HTTP load balancing
├── mux/           # Multiplexed load balancers
├── proto/         # Protocol buffer definitions
├── route/         # Route matching and building
└── strategy/      # Load balancing strategies

Quick Start

gRPC L4 Load Balancer

Simple connection-level load balancing with round-robin distribution:

package main

import (
	"context"
	
	"github.com/atlastore/belt/lb/backend"
	"github.com/atlastore/belt/lb/grpc/l4"
	"github.com/atlastore/belt/logx"
	"github.com/google/uuid"
	"go.uber.org/zap"
)

func main() {
	log := logx.New(
		context.Background(),
		logx.NewConfig(logx.Development, "grpc-l4-lb", uuid.NewString()),
		&logx.ConsoleTransport{},
	)
	
	// Define backend gRPC servers
	backends := []*backend.Backend{
		{Addr: "localhost:50051"},
		{Addr: "localhost:50052"},
		{Addr: "localhost:50053"},
	}
	
	// Create L4 load balancer
	lb := l4.New(log, backends)
	
	log.Info("Starting gRPC L4 load balancer on :9090")
	
	if err := lb.Start(context.Background(), ":9090"); err != nil {
		log.Fatal("Failed to start", zap.Error(err))
	}
}

gRPC L7 Load Balancer

Request-level routing based on gRPC service/method names and metadata:

package main

import (
	"context"
	
	"github.com/atlastore/belt/lb/backend"
	"github.com/atlastore/belt/lb/grpc/l7"
	"github.com/atlastore/belt/lb/route"
	"github.com/atlastore/belt/logx"
	"github.com/google/uuid"
	"go.uber.org/zap"
)

func main() {
	log := logx.New(
		context.Background(),
		logx.NewConfig(logx.Development, "grpc-l7-lb", uuid.NewString()),
		&logx.ConsoleTransport{},
	)
	
	backends := []*backend.Backend{
		{Addr: "localhost:50051"},
		{Addr: "localhost:50052"},
	}
	
	lb, lbInstance := l7.New(log, backends)
	
	// Add a route: send UserService requests to localhost:50051
	userServiceRoute := route.NewRouteBuilder().
		Service("myapp.UserService").
		Target("localhost:50051").
		Build()
	lbInstance.AddRoute(userServiceRoute)
	
	// Add another route: send PaymentService to localhost:50052
	paymentRoute := route.NewRouteBuilder().
		Service("myapp.PaymentService").
		Target("localhost:50052").
		Build()
	lbInstance.AddRoute(paymentRoute)
	
	log.Info("Starting gRPC L7 load balancer on :9090")
	
	if err := lb.Start(context.Background(), ":9090"); err != nil {
		log.Fatal("Failed to start", zap.Error(err))
	}
}

HTTP L7 Load Balancer

Path-based and header-based routing for HTTP services:

package main

import (
	"context"
	
	"github.com/atlastore/belt/lb/backend"
	"github.com/atlastore/belt/lb/http/l7"
	"github.com/atlastore/belt/lb/route"
	"github.com/atlastore/belt/logx"
	"github.com/google/uuid"
	"go.uber.org/zap"
)

func main() {
	log := logx.New(
		context.Background(),
		logx.NewConfig(logx.Development, "http-l7-lb", uuid.NewString()),
		&logx.ConsoleTransport{},
	)
	
	backends := []*backend.Backend{
		{Addr: "http://localhost:8081"},
		{Addr: "http://localhost:8082"},
	}
	
	lb, lbInstance := l7.New(log, backends)
	
	// Route /api/* to first backend
	apiRoute := route.NewRouteBuilder().
		Path("/api").
		Target("http://localhost:8081").
		Build()
	lbInstance.AddRoute(apiRoute)
	
	// Route /admin/* to second backend with header requirement
	adminRoute := route.NewRouteBuilder().
		Path("/admin").
		Header("X-Admin-Token", "secret").
		Target("http://localhost:8082").
		Build()
	lbInstance.AddRoute(adminRoute)
	
	log.Info("Starting HTTP L7 load balancer on :8080")
	
	if err := lb.Start(context.Background(), ":8080"); err != nil {
		log.Fatal("Failed to start", zap.Error(err))
	}
}

Dynamic Backend Management

All load balancers support dynamic backend and route management at runtime.

gRPC Management API

The load balancers expose a lb.LoadBalancer service with AddL4 and AddL7 methods:

service LoadBalancer {
  rpc AddL4(AddL4Request) returns (AddL4Response);
  rpc AddL7(AddL7Request) returns (AddL7Response);
}

Add a backend to L4 load balancer:

grpcurl -plaintext -d '{"addr": "localhost:50054"}' localhost:9090 lb.LoadBalancer/AddL4

Add a route to L7 load balancer:

grpcurl -plaintext -d '{
  "backend_addr": "localhost:50055",
  "services": ["myapp.UserService", "myapp.AuthService"],
  "methods": ["Login", "Logout"]
}' localhost:9090 lb.LoadBalancer/AddL7

HTTP Management API

All load balancers also expose POST /lb/add endpoints:

Add backend to HTTP L4:

curl -X POST http://localhost:8080/lb/add
# Uses client IP as backend address

Add route to HTTP L7:

curl -X POST http://localhost:8080/lb/add \
  -H "Content-Type: application/json" \
  -d '{
    "backend_addr": "http://localhost:8083",
    "paths": ["/api/v2"],
    "headers": {"X-Version": "v2"}
  }'

Route Building

The route builder provides a fluent API for creating complex routing rules:

route := route.NewRouteBuilder().
	// gRPC matchers
	Service("myapp.UserService").
	Method("GetUser").
	FullMethod("/myapp.UserService/GetUser").
	GRPCHeader("authorization", "Bearer token").
	
	// HTTP matchers
	Path("/api/users").
	Header("Content-Type", "application/json").
	Param("version", "v2").
	
	// Target backend
	Target("localhost:50051").
	
	Build()

Load Balancing Strategies

The strategy package provides different backend selection algorithms:

• Round Robin: Distribute requests evenly across backends
• Random: Randomly select a backend
• Least Connections: Route to backend with fewest active connections
• Weighted Round Robin: Distribute based on backend weights

import "github.com/atlastore/belt/lb/strategy"

// Round robin example
backends := []*backend.Backend{
	{Addr: "localhost:50051"},
	{Addr: "localhost:50052"},
}

rr := strategy.NewRoundRobin(backends)
selected := rr.Next() // Returns backend in rotation

Reflection Service Aggregation

The gRPC load balancers include a custom reflection service that aggregates reflection data from all backends, allowing clients to discover all available services across the cluster:

# List all services from all backends
grpcurl -plaintext localhost:9090 list

# Returns combined services from all backends + local LB services
# myapp.UserService
# myapp.PaymentService
# lb.LoadBalancer
# grpc.health.v1.Health
# grpc.reflection.v1.ServerReflection

The reflection proxy supports both grpc.reflection.v1 and grpc.reflection.v1alpha for maximum compatibility.

Health Checks

gRPC load balancers include health check support via the standard grpc.health.v1.Health service:

grpcurl -plaintext localhost:9090 grpc.health.v1.Health/Check

Protocol Details

gRPC L4

• Uses transparent proxy to forward all gRPC calls to backends
• Connection-level load balancing
• Supports all gRPC features (streaming, metadata, etc.)
• Minimal overhead

gRPC L7

• Request-level inspection and routing
• Routes based on service name, method name, or full method path
• Supports metadata-based routing
• Each request can go to a different backend
• Includes local services (LoadBalancer, Health, Reflection)

HTTP L4

• Connection-level HTTP load balancing
• Simple round-robin across backends
• Lowest latency option for HTTP

HTTP L7

• Path-based routing using patterns
• Header-based routing
• Query parameter matching
• Full URL forwarding to backends

Examples

See the *_test.go and example_test.go files in each package for more detailed examples:

• gRPC L4 Examples
• gRPC L7 Examples
• HTTP L4 Examples
• HTTP L7 Examples

Performance Considerations

• L4 vs L7: L4 offers lower latency but less flexibility; L7 provides advanced routing at the cost of per-request inspection
• Connection Pooling: The load balancers maintain connection pools to backends
• Zero-Allocation Paths: Critical paths use zero-allocation techniques for maximum performance
• Streaming Support: Full support for gRPC streaming (unary, server-streaming, client-streaming, bidirectional)

License

MIT License - see LICENSE.txt for details.

Documentation

Index

• type Builder
  • func New(proto types.Protocol, mode types.Mode, log *logx.Logger) *Builder
  • func (b *Builder) Init(backends []*backend.Backend, strat ...strategy.Type) server.Server
• type LoadBalancer

Types

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

func New

func New(proto types.Protocol, mode types.Mode, log *logx.Logger) *Builder

func (*Builder) Init

func (b *Builder) Init(backends []*backend.Backend, strat ...strategy.Type) server.Server

type LoadBalancer

type LoadBalancer interface {
	Start() error
	Stop() error
}