Documentation
¶
Overview ¶
Package breaker implements the circuit breaker pattern for resilient distributed systems.
breaker protects services from cascading failures by:
- Tracking Failures: Consecutive errors trip the circuit open
- Fast Rejection: Open circuits reject calls immediately without load
- Gradual Recovery: Half-open state tests if the service has recovered
- Lifecycle Hooks: OnStateChange, OnCall, OnReject for observability
- Zero Dependencies: Only the Go standard library
Quick Start ¶
Create a circuit and protect calls:
circuit := breaker.New("payment-service")
err := circuit.Do(ctx, func(ctx context.Context) error {
return client.Charge(ctx, amount)
})
if breaker.IsOpen(err) {
return handleFallback()
}
For functions that return values, use the generic Run helper:
user, err := breaker.Run(ctx, circuit, func(ctx context.Context) (*User, error) {
return client.GetUser(ctx, id)
})
Circuit States ¶
The circuit breaker has three states:
Closed (normal):
- Requests flow through to the protected function
- Failures are counted
- When failures reach threshold, circuit opens
Open (tripped):
- Requests are rejected immediately with ErrOpen
- After timeout, circuit transitions to half-open
HalfOpen (testing):
- Limited requests are allowed through
- Success closes the circuit
- Failure reopens it
Configuration ¶
Configure thresholds and timing with options:
circuit := breaker.New("api",
breaker.WithFailureThreshold(5), // Open after 5 consecutive failures
breaker.WithSuccessThreshold(2), // Close after 2 consecutive successes
breaker.WithOpenDuration(30*time.Second), // Wait 30s before half-open
breaker.WithHalfOpenRequests(3), // Allow 3 requests in half-open
)
Default values:
- FailureThreshold: 5 consecutive failures
- SuccessThreshold: 2 consecutive successes
- OpenDuration: 30 seconds
- HalfOpenRequests: 1 request
Failure Conditions ¶
By default, any non-nil error counts as a failure. Customize this with If:
// Only count specific errors as failures
circuit := breaker.New("api",
breaker.If(func(err error) bool {
return errors.Is(err, ErrTimeout) || errors.Is(err, ErrUnavailable)
}),
)
Use IfNot to exclude certain errors:
// Don't count 404s as failures
circuit := breaker.New("api",
breaker.IfNot(func(err error) bool {
return errors.Is(err, ErrNotFound)
}),
)
Use Not to invert any condition:
isTransient := func(err error) bool { return errors.Is(err, ErrTimeout) }
isPermanent := breaker.Not(isTransient)
Lifecycle Hooks ¶
Hooks provide observability without coupling to a specific logger or metrics system:
circuit := breaker.New("service",
breaker.OnStateChange(func(name string, from, to breaker.State) {
logger.Info("circuit state change",
"circuit", name,
"from", from,
"to", to,
)
metrics.Gauge("circuit.state", float64(to), "circuit:"+name)
}),
breaker.OnCall(func(name string, state breaker.State, err error) {
if err != nil {
metrics.Increment("circuit.failure", "circuit:"+name)
} else {
metrics.Increment("circuit.success", "circuit:"+name)
}
}),
breaker.OnReject(func(name string) {
metrics.Increment("circuit.rejected", "circuit:"+name)
}),
)
Available hooks:
- OnStateChange: Called when circuit transitions between states
- OnCall: Called after each call attempt (success or failure)
- OnReject: Called when a call is rejected due to open circuit
Fallback Pattern ¶
Use IsOpen to detect open circuits and provide fallback behavior:
func GetUser(ctx context.Context, id string) (*User, error) {
user, err := breaker.Run(ctx, circuit, func(ctx context.Context) (*User, error) {
return client.GetUser(ctx, id)
})
if breaker.IsOpen(err) {
return getCachedUser(id) // Fallback to cache
}
return user, err
}
Generic Helper ¶
The Run function provides type-safe return values:
// Returns (T, error) instead of just error
result, err := breaker.Run(ctx, circuit, func(ctx context.Context) (MyType, error) {
return doSomething(ctx)
})
This avoids the need for closures to capture return values.
Manual Reset ¶
Reset the circuit to closed state programmatically:
circuit.Reset()
Useful for admin endpoints or after deploying fixes.
Inspecting State ¶
Query the circuit's current status:
state := circuit.State() // Closed, Open, or HalfOpen name := circuit.Name() // The circuit's name failures, successes := circuit.Counts()
Testing ¶
Inject a fake clock to control time in tests:
type fakeClock struct {
now time.Time
}
func (c *fakeClock) Now() time.Time { return c.now }
func (c *fakeClock) Advance(d time.Duration) { c.now = c.now.Add(d) }
func TestCircuitOpensAfterTimeout(t *testing.T) {
clock := &fakeClock{now: time.Now()}
circuit := breaker.New("test",
breaker.WithFailureThreshold(1),
breaker.WithOpenDuration(30*time.Second),
breaker.WithClock(clock),
)
// Trip the circuit
_ = circuit.Do(ctx, func(ctx context.Context) error {
return errors.New("fail")
})
assert.Equal(t, breaker.Open, circuit.State())
// Advance time past open duration
clock.Advance(31 * time.Second)
assert.Equal(t, breaker.HalfOpen, circuit.State())
}
Best Practices ¶
1. Name circuits after the service they protect:
breaker.New("payment-gateway")
breaker.New("user-service")
2. Use hooks for observability instead of wrapping:
breaker.OnStateChange(func(name string, from, to breaker.State) {
// Log, metric, alert
})
3. Provide fallbacks for open circuits:
if breaker.IsOpen(err) {
return cachedValue, nil
}
4. Tune thresholds based on your traffic patterns:
// High-traffic: higher threshold to avoid false positives breaker.WithFailureThreshold(10) // Low-traffic: lower threshold for faster detection breaker.WithFailureThreshold(3)
5. Consider multiple half-open requests for gradual recovery:
breaker.WithHalfOpenRequests(3) breaker.WithSuccessThreshold(3)
Comparison to Other Patterns ¶
Circuit breaker vs retry:
- Retry: Repeats failed calls with backoff
- Circuit breaker: Stops calling after repeated failures
They work well together:
err := retry.Do(ctx, func(ctx context.Context) error {
return circuit.Do(ctx, func(ctx context.Context) error {
return client.Call(ctx)
})
}, retry.If(func(err error) bool {
return !breaker.IsOpen(err) // Don't retry if circuit is open
}))
Example (Fallback) ¶
Example_fallback demonstrates graceful degradation when circuit is open.
package main
import (
"context"
"errors"
"fmt"
"github.com/bjaus/breaker"
)
func main() {
circuit := breaker.New("user-service",
breaker.WithFailureThreshold(1),
)
getUser := func(ctx context.Context, _ int) (string, error) {
user, err := breaker.Run(ctx, circuit, func(ctx context.Context) (string, error) {
return "", errors.New("service unavailable")
})
if breaker.IsOpen(err) {
return "guest", nil
}
if err != nil {
return "", err
}
return user, nil
}
_, err1 := getUser(context.Background(), 1)
user2, _ := getUser(context.Background(), 2)
fmt.Println("User 1 error:", err1 != nil)
fmt.Println("User 2:", user2)
}
Output: User 1 error: true User 2: guest
Index ¶
- Constants
- Variables
- func IsOpen(err error) bool
- func Run[T any](ctx context.Context, c *Circuit, fn func(context.Context) (T, error)) (T, error)
- type Circuit
- type Clock
- type Condition
- type Func
- type OnCallFunc
- type OnRejectFunc
- type OnStateChangeFunc
- type Option
- func If(cond Condition) Option
- func IfNot(cond Condition) Option
- func OnCall(fn OnCallFunc) Option
- func OnReject(fn OnRejectFunc) Option
- func OnStateChange(fn OnStateChangeFunc) Option
- func WithClock(clock Clock) Option
- func WithFailureThreshold(n int) Option
- func WithHalfOpenRequests(n int) Option
- func WithOpenDuration(d time.Duration) Option
- func WithSuccessThreshold(n int) Option
- type State
Examples ¶
Constants ¶
const ( DefaultFailureThreshold = 5 DefaultSuccessThreshold = 2 DefaultOpenDuration = 30 * time.Second DefaultHalfOpenRequests = 1 )
Default values.
Variables ¶
var ErrOpen = errors.New("circuit open")
ErrOpen is returned when the circuit is open and rejecting requests.
Functions ¶
func IsOpen ¶
IsOpen reports whether err is because the circuit is open.
Example ¶
ExampleIsOpen demonstrates checking if an error is due to an open circuit.
package main
import (
"context"
"errors"
"fmt"
"github.com/bjaus/breaker"
)
func main() {
circuit := breaker.New("service",
breaker.WithFailureThreshold(1),
)
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return errors.New("fail")
})
err := circuit.Do(context.Background(), func(ctx context.Context) error {
return nil
})
if breaker.IsOpen(err) {
fmt.Println("Circuit is open, using fallback")
}
}
Output: Circuit is open, using fallback
func Run ¶
Run executes fn and returns its result with circuit breaker protection. This is a convenience wrapper for functions that return a value.
Example ¶
ExampleRun demonstrates the generic helper for returning values.
package main
import (
"context"
"fmt"
"github.com/bjaus/breaker"
)
func main() {
circuit := breaker.New("user-service")
user, err := breaker.Run(context.Background(), circuit, func(ctx context.Context) (string, error) {
return "john_doe", nil
})
fmt.Println("User:", user)
fmt.Println("Error:", err)
}
Output: User: john_doe Error: <nil>
Types ¶
type Circuit ¶
type Circuit struct {
// contains filtered or unexported fields
}
Circuit is a circuit breaker. Safe for concurrent use.
func New ¶
New creates a Circuit with the given options.
Example ¶
ExampleNew demonstrates creating a circuit breaker with default settings.
package main
import (
"context"
"fmt"
"github.com/bjaus/breaker"
)
func main() {
circuit := breaker.New("my-service")
err := circuit.Do(context.Background(), func(ctx context.Context) error {
return nil
})
fmt.Println("Error:", err)
fmt.Println("State:", circuit.State())
}
Output: Error: <nil> State: closed
Example (WithOptions) ¶
ExampleNew_withOptions demonstrates creating a circuit breaker with custom settings.
package main
import (
"fmt"
"time"
"github.com/bjaus/breaker"
)
func main() {
circuit := breaker.New("payment-service",
breaker.WithFailureThreshold(3),
breaker.WithSuccessThreshold(2),
breaker.WithOpenDuration(30*time.Second),
)
fmt.Println("Name:", circuit.Name())
fmt.Println("State:", circuit.State())
}
Output: Name: payment-service State: closed
func (*Circuit) Do ¶
Do executes fn with circuit breaker protection.
Example ¶
ExampleCircuit_Do demonstrates basic circuit breaker usage.
package main
import (
"context"
"errors"
"fmt"
"github.com/bjaus/breaker"
)
func main() {
circuit := breaker.New("api",
breaker.WithFailureThreshold(2),
)
attempts := 0
for range 5 {
err := circuit.Do(context.Background(), func(ctx context.Context) error {
attempts++
return errors.New("service unavailable")
})
if breaker.IsOpen(err) {
fmt.Println("Circuit is open, skipping call")
}
}
fmt.Println("Attempts:", attempts)
fmt.Println("State:", circuit.State())
}
Output: Circuit is open, skipping call Circuit is open, skipping call Circuit is open, skipping call Attempts: 2 State: open
func (*Circuit) Reset ¶
func (c *Circuit) Reset()
Reset manually resets the circuit to closed state.
Example ¶
ExampleCircuit_Reset demonstrates manually resetting a circuit.
package main
import (
"context"
"errors"
"fmt"
"github.com/bjaus/breaker"
)
func main() {
circuit := breaker.New("service",
breaker.WithFailureThreshold(1),
)
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return errors.New("fail")
})
fmt.Println("Before reset:", circuit.State())
circuit.Reset()
fmt.Println("After reset:", circuit.State())
}
Output: Before reset: open After reset: closed
type OnCallFunc ¶
OnCallFunc is called after each call attempt.
type OnRejectFunc ¶
type OnRejectFunc func(name string)
OnRejectFunc is called when a call is rejected due to open circuit.
type OnStateChangeFunc ¶
OnStateChangeFunc is called when the circuit changes state.
type Option ¶
type Option func(*config)
Option configures a Circuit.
func If ¶
If sets the condition that determines whether an error counts as a failure. By default, any non-nil error is a failure.
Example ¶
ExampleIf demonstrates custom failure conditions.
package main
import (
"context"
"errors"
"fmt"
"github.com/bjaus/breaker"
)
func main() {
transient := errors.New("transient error")
circuit := breaker.New("api",
breaker.WithFailureThreshold(2),
breaker.If(func(err error) bool {
return errors.Is(err, transient)
}),
)
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return errors.New("permanent error")
})
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return errors.New("permanent error")
})
fmt.Println("After permanent errors:", circuit.State())
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return transient
})
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return transient
})
fmt.Println("After transient errors:", circuit.State())
}
Output: After permanent errors: closed After transient errors: open
func IfNot ¶
IfNot sets a condition where matching errors are NOT counted as failures. This is equivalent to If(Not(cond)).
func OnCall ¶
func OnCall(fn OnCallFunc) Option
OnCall sets a hook called after each call attempt.
Example ¶
ExampleOnCall demonstrates the call hook for metrics.
package main
import (
"context"
"errors"
"fmt"
"github.com/bjaus/breaker"
)
func main() {
successCount := 0
failureCount := 0
circuit := breaker.New("service",
breaker.OnCall(func(name string, state breaker.State, err error) {
if err != nil {
failureCount++
} else {
successCount++
}
}),
)
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return nil
})
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return errors.New("fail")
})
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return nil
})
fmt.Println("Successes:", successCount)
fmt.Println("Failures:", failureCount)
}
Output: Successes: 2 Failures: 1
func OnReject ¶
func OnReject(fn OnRejectFunc) Option
OnReject sets a hook called when a call is rejected due to open circuit.
Example ¶
ExampleOnReject demonstrates the reject hook.
package main
import (
"context"
"errors"
"fmt"
"github.com/bjaus/breaker"
)
func main() {
rejectCount := 0
circuit := breaker.New("service",
breaker.WithFailureThreshold(1),
breaker.OnReject(func(name string) {
rejectCount++
}),
)
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return errors.New("fail")
})
for range 3 {
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return nil
})
}
fmt.Println("Rejected:", rejectCount)
}
Output: Rejected: 3
func OnStateChange ¶
func OnStateChange(fn OnStateChangeFunc) Option
OnStateChange sets a hook called when the circuit changes state.
Example ¶
ExampleOnStateChange demonstrates the state change hook.
package main
import (
"context"
"errors"
"fmt"
"github.com/bjaus/breaker"
)
func main() {
circuit := breaker.New("service",
breaker.WithFailureThreshold(1),
breaker.OnStateChange(func(name string, from, to breaker.State) {
fmt.Printf("Circuit %s: %s -> %s\n", name, from, to)
}),
)
_ = circuit.Do(context.Background(), func(ctx context.Context) error {
return errors.New("fail")
})
}
Output: Circuit service: closed -> open
func WithFailureThreshold ¶
WithFailureThreshold sets consecutive failures before opening the circuit. Default is 5.
func WithHalfOpenRequests ¶
WithHalfOpenRequests sets how many requests are allowed through in the half-open state. Default is 1.
func WithOpenDuration ¶
WithOpenDuration sets how long the circuit stays open before transitioning to half-open. Default is 30 seconds.
func WithSuccessThreshold ¶
WithSuccessThreshold sets consecutive successes in half-open state required before closing the circuit. Default is 2.
type State ¶
type State int
State represents the circuit breaker state.
func (State) String ¶
String returns the string representation of the state.
Example ¶
ExampleState_String demonstrates state string representation.
package main
import (
"fmt"
"github.com/bjaus/breaker"
)
func main() {
fmt.Println(breaker.Closed.String())
fmt.Println(breaker.Open.String())
fmt.Println(breaker.HalfOpen.String())
}
Output: closed open half-open