Documentation
¶
Overview ¶
Package api provides the HTTP API server for fred.
The API serves three main purposes:
Tenant Access: Authenticated endpoints for tenants to retrieve connection details and upload deployment payloads for their leases.
Backend Callbacks: Endpoint for backends to report provisioning results with HMAC-SHA256 authentication.
Observability: Health check and Prometheus metrics endpoints.
Authentication ¶
Tenant endpoints use ADR-036 signature-based authentication. Tenants create a bearer token containing their address, the lease UUID, a timestamp, their public key, and a signature over the message. The server validates:
- The signature matches the message content
- The public key derives to the tenant address
- The timestamp is at most 30 seconds in the past and at most 10 seconds in the future (clock-skew tolerance)
- Replay protection (TokenTracker): required for connection, restart, and update — where a replayed token would re-leak sensitive data or re-run a mutating operation. Idempotent reads (status, provision, logs, releases, events) skip this check. The data upload endpoint skips it too and relies on its own idempotency (409 on a duplicate upload for the lease).
The token tracker uses fail-closed semantics: if the database is unavailable, requests are rejected with 503 Service Unavailable rather than proceeding without replay protection. Since token lifetime is short (30 seconds), clients can safely retry with a fresh token.
Backend callbacks use HMAC-SHA256 authentication with a shared secret configured via callback_secret.
Rate Limiting ¶
The server implements two layers of rate limiting:
- Per-IP rate limiting for all requests (via RateLimiter)
- Per-tenant rate limiting for authenticated endpoints (via TenantRateLimiter)
Both use token bucket algorithms with configurable RPS and burst sizes.
Endpoints ¶
GET /health - Health check with chain connectivity
GET /metrics - Prometheus metrics
GET /workloads?lease_uuid=<u>... - Bulk workload metadata lookup (unauthenticated)
GET /v1/leases/{lease_uuid}/connection - Get connection details (authenticated)
GET /v1/leases/{lease_uuid}/status - Get provisioning status (authenticated)
GET /v1/leases/{lease_uuid}/provision - Get provision diagnostics (authenticated)
GET /v1/leases/{lease_uuid}/logs - Get container logs (authenticated)
GET /v1/leases/{lease_uuid}/releases - Get release history (authenticated)
GET /v1/leases/{lease_uuid}/events - Stream lease events via WebSocket (authenticated)
POST /v1/leases/{lease_uuid}/data - Upload deployment payload (authenticated)
POST /v1/leases/{lease_uuid}/restart - Restart a provisioned lease (authenticated)
POST /v1/leases/{lease_uuid}/update - Update a provisioned lease (authenticated)
POST /callbacks/provision - Backend provisioning callback (HMAC auth)
Index ¶
- Constants
- Variables
- func WSTokenPromoter(next http.Handler) http.Handler
- type AuthToken
- type AuthenticatedRequest
- type CallbackAuthenticator
- func (a *CallbackAuthenticator) ComputeSignature(method, uri string, payload []byte) string
- func (a *CallbackAuthenticator) ComputeSignatureWithTime(method, uri string, payload []byte, t time.Time) string
- func (a *CallbackAuthenticator) VerifyRequest(r *http.Request) ([]byte, error)
- func (a *CallbackAuthenticator) VerifySignature(method, uri string, payload []byte, signature string) bool
- func (a *CallbackAuthenticator) VerifySignatureWithTime(method, uri string, payload []byte, signature string, now time.Time) bool
- func (a *CallbackAuthenticator) WithCanonicalPathPrefix(prefix string) *CallbackAuthenticator
- type CallbackPublisher
- type CallbackResponse
- type ChainClient
- type CheckResult
- type ConnectionDetails
- type ConnectionResponse
- type ErrorResponse
- type EventBroker
- type Handlers
- func (h *Handlers) AuthenticateLeaseRequest(r *http.Request, leaseUUID string, checkReplay bool, requireActive bool) (*AuthenticatedRequest, int, error)
- func (h *Handlers) GetLeaseConnection(w http.ResponseWriter, r *http.Request)
- func (h *Handlers) GetLeaseLogs(w http.ResponseWriter, r *http.Request)
- func (h *Handlers) GetLeaseProvision(w http.ResponseWriter, r *http.Request)
- func (h *Handlers) GetLeaseReleases(w http.ResponseWriter, r *http.Request)
- func (h *Handlers) GetLeaseStatus(w http.ResponseWriter, r *http.Request)
- func (h *Handlers) GetWorkloads(w http.ResponseWriter, r *http.Request)
- func (h *Handlers) HealthCheck(w http.ResponseWriter, r *http.Request)
- func (h *Handlers) RestartLease(w http.ResponseWriter, r *http.Request)
- func (h *Handlers) RestoreLease(w http.ResponseWriter, r *http.Request)
- func (h *Handlers) StreamLeaseEvents(w http.ResponseWriter, r *http.Request)
- func (h *Handlers) UpdateLease(w http.ResponseWriter, r *http.Request)
- type HandlersConfig
- type HealthResponse
- type HealthStats
- type InstanceInfo
- type LeaseLogsResponse
- type LeaseProvisionResponse
- type LeaseReleasesResponse
- type LeaseStatusResponse
- type PayloadAuthToken
- type PayloadHandler
- type PayloadPublisher
- type PlacementLookup
- type PortMapping
- type RateLimiter
- type RestoreInFlightTracker
- type RestorePlacementRecorder
- type Server
- type ServerConfig
- type ServerDeps
- type ServiceConnectionDetails
- type StatusChecker
- type TenantRateLimiter
- type TokenTracker
- type TokenTrackerConfig
- type TokenTrackerInterface
- type TrustedProxyConfig
- type WorkloadEntry
- type WorkloadItem
- type WorkloadLookupResponse
Examples ¶
Constants ¶
const ( // MaxTokenAge is the maximum age of a valid authentication token. // Kept short (30 seconds) to limit replay attack window. MaxTokenAge = 30 * time.Second // MaxFutureClockSkew is the maximum allowed clock skew for tokens with future timestamps. // This is intentionally smaller than MaxTokenAge to limit pre-generated token attacks. // 10 seconds allows for reasonable clock drift without enabling abuse. MaxFutureClockSkew = 10 * time.Second )
const ( // DefaultMaxSubscriptionsPerLease is the maximum number of concurrent // WebSocket subscriptions allowed per lease UUID. DefaultMaxSubscriptionsPerLease = 10 // DefaultMaxTotalSubscriptions is the maximum number of concurrent // WebSocket subscriptions allowed globally across all leases. DefaultMaxTotalSubscriptions = 1000 )
const ( // CallbackSignatureHeader is the header name for HMAC signatures on callbacks. // Format: "t=<unix-timestamp>,sha256=<hex-encoded-hmac>" CallbackSignatureHeader = hmacauth.SignatureHeader // DefaultCallbackMaxAge is the default maximum age for callback timestamps. // Callbacks older than this are rejected to prevent replay attacks. DefaultCallbackMaxAge = 5 * time.Minute // MaxCallbackMaxAge is the maximum allowed value for callback max age. // Values larger than this would undermine replay protection. MaxCallbackMaxAge = 1 * time.Hour // MinCallbackSecretLength is the minimum required length for callback secrets. // For full 256-bit security with HMAC-SHA256, use at least 32 bytes. MinCallbackSecretLength = 32 )
Variables ¶
var ( // ErrTokenAlreadyUsed indicates the token has already been used. ErrTokenAlreadyUsed = errors.New("token already used") )
var ErrTooManySubscriptions = errors.New("too many subscriptions")
ErrTooManySubscriptions is returned when a subscription limit is reached.
Functions ¶
Types ¶
type AuthToken ¶
type AuthToken struct {
Tenant string `json:"tenant"`
LeaseUUID string `json:"lease_uuid"`
Timestamp int64 `json:"timestamp"`
PubKey string `json:"pub_key"` // Base64-encoded public key
Signature string `json:"signature"` // Base64-encoded signature
}
AuthToken represents the bearer token for tenant authentication.
func AuthTokenFromContext ¶
AuthTokenFromContext retrieves the pre-validated AuthToken from request context. Returns nil if no token was stored (e.g. rate limiting disabled).
func ParseAuthToken ¶
ParseAuthToken parses a base64-encoded authentication token.
func (*AuthToken) Validate ¶
Validate verifies the token's timestamp and ADR-036 signature. The bech32Prefix is used to verify the tenant address matches the public key. On success, t.Signature is updated to the low-S canonical form to ensure consistent replay tracking regardless of which signature variant was submitted.
type AuthenticatedRequest ¶
type AuthenticatedRequest struct {
Token *AuthToken
Lease *billingtypes.Lease
}
AuthenticatedRequest contains the result of a successful authentication.
type CallbackAuthenticator ¶
type CallbackAuthenticator struct {
// contains filtered or unexported fields
}
CallbackAuthenticator verifies HMAC signatures on backend callbacks. It includes timestamp-based replay protection following the Stripe pattern.
Performance note: The current implementation uses fmt.Sprintf to build the signed payload, which allocates an intermediate string. This is acceptable because callback payloads are small by design (~100 bytes: lease_uuid, status, error). The large Payload field (potentially megabytes) is in ProvisionRequest going TO backends, not in CallbackPayload coming back. If signing large data becomes necessary, consider writing to the HMAC incrementally to avoid copying the payload.
func NewCallbackAuthenticator ¶
func NewCallbackAuthenticator(secret string) (*CallbackAuthenticator, error)
NewCallbackAuthenticator creates a new callback authenticator with the given secret. Uses DefaultCallbackMaxAge for replay protection. Returns an error if the secret is shorter than MinCallbackSecretLength bytes.
func NewCallbackAuthenticatorWithMaxAge ¶
func NewCallbackAuthenticatorWithMaxAge(secret string, maxAge time.Duration) (*CallbackAuthenticator, error)
NewCallbackAuthenticatorWithMaxAge creates a callback authenticator with a custom max age. Returns an error if the secret is shorter than MinCallbackSecretLength bytes, maxAge is not positive, or maxAge exceeds MaxCallbackMaxAge.
func (*CallbackAuthenticator) ComputeSignature ¶
func (a *CallbackAuthenticator) ComputeSignature(method, uri string, payload []byte) string
ComputeSignature computes the HMAC-SHA256 signature for a request shape with the current timestamp. method and uri must match what the verifier will see on the wire (typically req.Method and req.URL.RequestURI()). Returns the signature in the format "t=<timestamp>,sha256=<hex>".
Example ¶
Example showing the signature format
auth, err := NewCallbackAuthenticator("my-secret-key-at-least-32-characters")
if err != nil {
fmt.Println("Error:", err)
return
}
payload := []byte(`{"lease_uuid":"abc-123","status":"success"}`)
signature := auth.ComputeSignature(testCallbackMethod, testCallbackURI, payload)
// Output format: t=<unix-timestamp>,sha256=<hex-encoded-hmac>
fmt.Println("Signature format:", strings.Split(signature, ",")[0][:2]) // "t="
Output: Signature format: t=
func (*CallbackAuthenticator) ComputeSignatureWithTime ¶
func (a *CallbackAuthenticator) ComputeSignatureWithTime(method, uri string, payload []byte, t time.Time) string
ComputeSignatureWithTime computes the signature with a specific timestamp (for testing).
func (*CallbackAuthenticator) VerifyRequest ¶
func (a *CallbackAuthenticator) VerifyRequest(r *http.Request) ([]byte, error)
VerifyRequest reads the request body, verifies the signature, and returns the body bytes. Returns an error if verification fails or the timestamp is too old.
Note: do not confuse this method with hmacauth.VerifyRequest. The names live in different packages and have different semantics:
- This method reads r.Body itself (callers pass the bare *http.Request and receive the body back) and applies the callback-specific maxAge and clock-skew tolerance configured on the CallbackAuthenticator.
- hmacauth.VerifyRequest is a low-level wrapper that takes a pre-read body and an explicit maxAge; r.Body is untouched.
Internally, this method delegates the canonical-string check to hmacauth.VerifyWithTime via verifySignatureWithError.
func (*CallbackAuthenticator) VerifySignature ¶
func (a *CallbackAuthenticator) VerifySignature(method, uri string, payload []byte, signature string) bool
VerifySignature verifies that the provided signature matches the request shape. method and uri must match what the sender used (typically r.Method and r.URL.RequestURI() on the inbound request). The signature should be in the format "t=<timestamp>,sha256=<hex>". Returns false if the signature is invalid, the timestamp is too old, or the timestamp is too far in the future.
func (*CallbackAuthenticator) VerifySignatureWithTime ¶
func (a *CallbackAuthenticator) VerifySignatureWithTime(method, uri string, payload []byte, signature string, now time.Time) bool
VerifySignatureWithTime verifies the signature against a reference time (for testing).
func (*CallbackAuthenticator) WithCanonicalPathPrefix ¶
func (a *CallbackAuthenticator) WithCanonicalPathPrefix(prefix string) *CallbackAuthenticator
WithCanonicalPathPrefix configures a static path prefix that is prepended to r.URL.RequestURI() before HMAC verification. Set this when fred is deployed behind a path-stripping reverse proxy (e.g., Traefik stripPrefix mapping /api/fred/* → /*) so the verifier's canonical URI matches what the signer used. Passing the empty string is a no-op and preserves the default direct- call behavior. Returns the receiver for chaining.
type CallbackPublisher ¶
type CallbackPublisher interface {
PublishCallback(callback backend.CallbackPayload) error
}
CallbackPublisher publishes backend callbacks to the provisioner.
type CallbackResponse ¶
type CallbackResponse struct {
Status string `json:"status"`
Message string `json:"message,omitempty"`
}
CallbackResponse represents the response for backend callbacks. Used to provide debugging information to authenticated backends.
type ChainClient ¶
type ChainClient interface {
GetLease(ctx context.Context, leaseUUID string) (*billingtypes.Lease, error)
GetActiveLease(ctx context.Context, leaseUUID string) (*billingtypes.Lease, error)
Ping(ctx context.Context) error
}
ChainClient defines the chain operations needed by handlers.
type CheckResult ¶
CheckResult represents the result of a single health check.
type ConnectionDetails ¶
type ConnectionDetails struct {
Host string `json:"host"`
FQDN string `json:"fqdn,omitempty"`
Ports map[string]PortMapping `json:"ports,omitempty"`
Instances []InstanceInfo `json:"instances,omitempty"`
Services map[string]ServiceConnectionDetails `json:"services,omitempty"`
Protocol string `json:"protocol,omitempty"`
Metadata map[string]string `json:"metadata,omitempty"`
}
ConnectionDetails contains the connection information for a lease. For multi-instance leases, the Instances array contains per-instance details. For stack (multi-service) leases, the Services map contains per-service details.
type ConnectionResponse ¶
type ConnectionResponse struct {
LeaseUUID string `json:"lease_uuid"`
Tenant string `json:"tenant"`
ProviderUUID string `json:"provider_uuid"`
Connection ConnectionDetails `json:"connection"`
}
ConnectionResponse represents the response for connection details.
type ErrorResponse ¶
ErrorResponse represents an error response.
type EventBroker ¶
type EventBroker struct {
// contains filtered or unexported fields
}
EventBroker manages per-lease event client subscriptions with non-blocking fan-out. Slow clients drop events; they can re-fetch via REST.
func NewEventBroker ¶
func NewEventBroker() *EventBroker
NewEventBroker creates a new event broker with default subscription limits.
func NewEventBrokerWithLimits ¶
func NewEventBrokerWithLimits(maxPerLease, maxTotal int) *EventBroker
NewEventBrokerWithLimits creates a new event broker with the given subscription limits. maxPerLease limits concurrent subscriptions per lease UUID. maxTotal limits concurrent subscriptions globally across all leases.
func (*EventBroker) Close ¶
func (b *EventBroker) Close()
Close closes all subscriber channels and prevents new subscriptions. Safe to call multiple times.
func (*EventBroker) Publish ¶
func (b *EventBroker) Publish(event backend.LeaseStatusEvent)
Publish sends an event to all clients subscribed to the event's lease UUID. Non-blocking: if a client's channel is full, the event is dropped for that client.
func (*EventBroker) Subscribe ¶
func (b *EventBroker) Subscribe(leaseUUID string) (<-chan backend.LeaseStatusEvent, error)
Subscribe registers a client channel for events on the given lease UUID. The returned channel is buffered; the caller should read from it in a loop. Returns nil and ErrTooManySubscriptions if a subscription limit is reached. Returns nil and nil error if the broker has been closed.
func (*EventBroker) Unsubscribe ¶
func (b *EventBroker) Unsubscribe(leaseUUID string, ch <-chan backend.LeaseStatusEvent)
Unsubscribe removes a client channel. The channel is closed after removal.
type Handlers ¶
type Handlers struct {
// contains filtered or unexported fields
}
Handlers contains HTTP request handlers.
func NewHandlers ¶
func NewHandlers(cfg HandlersConfig) *Handlers
NewHandlers creates a new Handlers instance.
func (*Handlers) AuthenticateLeaseRequest ¶
func (h *Handlers) AuthenticateLeaseRequest(r *http.Request, leaseUUID string, checkReplay bool, requireActive bool) (*AuthenticatedRequest, int, error)
AuthenticateLeaseRequest performs common authentication and authorization for lease endpoints. It extracts and validates the bearer token, optionally checks for replay attacks, queries the lease from chain, and verifies tenant and provider ownership.
Parameters:
- r: the HTTP request
- leaseUUID: the lease UUID from the URL path
- checkReplay: whether to check for token replay (set false for idempotent/read-heavy endpoints like status)
- requireActive: if true, only ACTIVE leases are accepted; if false, any state is allowed
Returns AuthenticatedRequest on success, or an error with the appropriate HTTP status code.
func (*Handlers) GetLeaseConnection ¶
func (h *Handlers) GetLeaseConnection(w http.ResponseWriter, r *http.Request)
GetLeaseConnection handles GET /v1/leases/{lease_uuid}/connection
func (*Handlers) GetLeaseLogs ¶
func (h *Handlers) GetLeaseLogs(w http.ResponseWriter, r *http.Request)
GetLeaseLogs handles GET /v1/leases/{lease_uuid}/logs
func (*Handlers) GetLeaseProvision ¶
func (h *Handlers) GetLeaseProvision(w http.ResponseWriter, r *http.Request)
GetLeaseProvision handles GET /v1/leases/{lease_uuid}/provision
Like GetLeaseStatus, authz is chain-primary with a retained-record fallback (ENG-329 #5), and provision discovery uses findProvision (placement fast-path, bounded fan-out fallback). Within the grace window a retained lease returns status=retained (with retained_until + items), not 404.
func (*Handlers) GetLeaseReleases ¶
func (h *Handlers) GetLeaseReleases(w http.ResponseWriter, r *http.Request)
GetLeaseReleases handles GET /v1/leases/{lease_uuid}/releases
func (*Handlers) GetLeaseStatus ¶
func (h *Handlers) GetLeaseStatus(w http.ResponseWriter, r *http.Request)
GetLeaseStatus handles GET /v1/leases/{lease_uuid}/status
Authz is chain-primary with a retained-record fallback (ENG-329 #5): the ADR-036-signed token is validated first, then the chain lease is queried (any state). If the chain still has the lease, tenant/provider ownership is verified against it (the existing path). If the chain has PRUNED the lease (auto-closed cohort), the request is authorized iff the signed caller's tenant equals the retained record's Tenant, surfaced via the bounded fan-out GetProvision — mirroring restore's cross-tenant guard. A cross-tenant caller, or an absent retained record, is rejected.
func (*Handlers) GetWorkloads ¶
func (h *Handlers) GetWorkloads(w http.ResponseWriter, r *http.Request)
GetWorkloads handles GET /workloads?lease_uuid=<u1>&lease_uuid=<u2>...
Returns workload metadata for the requested lease UUIDs from any backend that knows about them. Caller is the manifest-admin SPA (cross-origin via CORS); the visible page of leases on the admin's leases pages drives the UUID list. The 1..MaxLookupUUIDs cap matches the admin's PAGE_SIZE (25) with headroom.
Per-backend errors are surfaced via the Warnings slice rather than failing the whole request, so a single broken backend doesn't blank the admin's image column. This is a deliberate divergence from reconciler.go's fetchAllProvisions, which aborts on any backend error to avoid mistaking a transient failure for "lease no longer exists" — that concern doesn't apply here since /workloads is read-only metadata for display.
func (*Handlers) HealthCheck ¶
func (h *Handlers) HealthCheck(w http.ResponseWriter, r *http.Request)
HealthCheck handles GET /health
func (*Handlers) RestartLease ¶
func (h *Handlers) RestartLease(w http.ResponseWriter, r *http.Request)
RestartLease handles POST /v1/leases/{lease_uuid}/restart
func (*Handlers) RestoreLease ¶ added in v0.5.0
func (h *Handlers) RestoreLease(w http.ResponseWriter, r *http.Request)
RestoreLease handles POST /v1/leases/{lease_uuid}/restore (lease_uuid is the NEW, fresh lease; the body names the original retained lease).
func (*Handlers) StreamLeaseEvents ¶
func (h *Handlers) StreamLeaseEvents(w http.ResponseWriter, r *http.Request)
StreamLeaseEvents serves a WebSocket stream of lease status events. GET /v1/leases/{lease_uuid}/events
func (*Handlers) UpdateLease ¶
func (h *Handlers) UpdateLease(w http.ResponseWriter, r *http.Request)
UpdateLease handles POST /v1/leases/{lease_uuid}/update
type HandlersConfig ¶
type HandlersConfig struct {
Client ChainClient
BackendRouter *backend.Router
TokenTracker TokenTrackerInterface // optional but recommended for replay attack protection
StatusChecker StatusChecker // optional but required for the /status endpoint
PlacementLookup PlacementLookup // optional — used for routing reads to the correct backend
RestoreRecorder RestorePlacementRecorder // optional — restore placement bookkeeping (ENG-333)
RestoreTracker RestoreInFlightTracker // optional — inline-ack restore in-flight tracking (ENG-358)
EventBroker *EventBroker // optional — if nil, the events endpoint will return 501
ProviderUUID string
Bech32Prefix string
CallbackBaseURL string // used for restart/update callbacks to the backend
}
HandlersConfig configures a Handlers instance.
type HealthResponse ¶
type HealthResponse struct {
Status string `json:"status"`
ProviderUUID string `json:"provider_uuid"`
Checks map[string]*CheckResult `json:"checks"`
Stats *HealthStats `json:"stats,omitempty"`
}
HealthResponse represents the health check response.
type HealthStats ¶
type HealthStats struct {
InFlightProvisions int `json:"in_flight_provisions"`
}
HealthStats contains operational statistics for the health response.
type InstanceInfo ¶
type InstanceInfo struct {
InstanceIndex int `json:"instance_index"`
ContainerID string `json:"container_id,omitempty"`
Image string `json:"image,omitempty"`
Status string `json:"status,omitempty"`
FQDN string `json:"fqdn,omitempty"`
Ports map[string]PortMapping `json:"ports,omitempty"`
}
InstanceInfo contains connection details for a single instance in a multi-instance lease.
type LeaseLogsResponse ¶
type LeaseLogsResponse struct {
LeaseUUID string `json:"lease_uuid"`
Tenant string `json:"tenant"`
ProviderUUID string `json:"provider_uuid"`
Logs map[string]string `json:"logs"`
}
LeaseLogsResponse represents the response for container logs.
type LeaseProvisionResponse ¶
type LeaseProvisionResponse struct {
LeaseUUID string `json:"lease_uuid"`
Tenant string `json:"tenant"`
ProviderUUID string `json:"provider_uuid"`
Status string `json:"status"`
FailCount int `json:"fail_count"`
LastError string `json:"last_error,omitempty"`
// Retention fields (populated when status=retained, ENG-329). See
// LeaseStatusResponse; the owning Tenant from ProvisionInfo is NOT surfaced.
RetainedUntil string `json:"retained_until,omitempty"`
Items []backend.LeaseItem `json:"items,omitempty"`
RestoreHint string `json:"restore_hint,omitempty"`
}
LeaseProvisionResponse represents the response for provision diagnostics.
type LeaseReleasesResponse ¶
type LeaseReleasesResponse struct {
LeaseUUID string `json:"lease_uuid"`
Tenant string `json:"tenant"`
ProviderUUID string `json:"provider_uuid"`
Releases []backend.ReleaseInfo `json:"releases"`
}
LeaseReleasesResponse represents the response for release history.
type LeaseStatusResponse ¶
type LeaseStatusResponse struct {
LeaseUUID string `json:"lease_uuid"`
Tenant string `json:"tenant"`
ProviderUUID string `json:"provider_uuid"`
State string `json:"state"`
RequiresPayload bool `json:"requires_payload"`
MetaHashHex string `json:"meta_hash_hex,omitempty"` // For debugging - shows the expected payload hash
PayloadReceived bool `json:"payload_received"`
ProvisioningStarted bool `json:"provisioning_started"`
ProvisionStatus string `json:"provision_status,omitempty"`
FailCount int `json:"fail_count,omitempty"`
LastError string `json:"last_error,omitempty"`
// Retention fields (populated when provision_status=retained, ENG-329).
// RetainedUntil is RFC3339; Items is the restore shape (service name + SKU +
// quantity) the tenant uses to build a matching fresh PENDING lease;
// RestoreHint is a short human-readable next-step. The owning Tenant from
// ProvisionInfo is intentionally NOT surfaced here (cross-tenant leak guard).
RetainedUntil string `json:"retained_until,omitempty"`
Items []backend.LeaseItem `json:"items,omitempty"`
RestoreHint string `json:"restore_hint,omitempty"`
}
LeaseStatusResponse represents the response for lease status. Includes tenant and provider_uuid for consistency with ConnectionResponse.
type PayloadAuthToken ¶
type PayloadAuthToken struct {
Tenant string `json:"tenant"`
LeaseUUID string `json:"lease_uuid"`
MetaHash string `json:"meta_hash"` // Hex-encoded SHA-256 hash
Timestamp int64 `json:"timestamp"`
PubKey string `json:"pub_key"` // Base64-encoded public key
Signature string `json:"signature"` // Base64-encoded signature
}
PayloadAuthToken represents the bearer token for payload upload authentication. It includes meta_hash to bind the signature to a specific payload.
func ParsePayloadAuthToken ¶
func ParsePayloadAuthToken(encoded string) (*PayloadAuthToken, error)
ParsePayloadAuthToken parses a base64-encoded payload authentication token.
func PayloadAuthTokenFromContext ¶
func PayloadAuthTokenFromContext(ctx context.Context) *PayloadAuthToken
PayloadAuthTokenFromContext retrieves the pre-validated PayloadAuthToken from request context. Returns nil if no token was stored (e.g. rate limiting disabled).
func (*PayloadAuthToken) Validate ¶
func (t *PayloadAuthToken) Validate(bech32Prefix string) error
Validate verifies the token's timestamp and ADR-036 signature. The bech32Prefix is used to verify the tenant address matches the public key.
type PayloadHandler ¶
type PayloadHandler struct {
// contains filtered or unexported fields
}
PayloadHandler handles payload upload requests.
func NewPayloadHandler ¶
func NewPayloadHandler(client ChainClient, publisher PayloadPublisher, providerUUID, bech32Prefix string) *PayloadHandler
NewPayloadHandler creates a new payload handler.
func (*PayloadHandler) HandlePayloadUpload ¶
func (h *PayloadHandler) HandlePayloadUpload(w http.ResponseWriter, r *http.Request)
HandlePayloadUpload handles POST /v1/leases/{lease_uuid}/data
type PayloadPublisher ¶
type PayloadPublisher interface {
PublishPayload(event payload.Event) error
StorePayload(leaseUUID string, payload []byte) bool
DeletePayload(leaseUUID string) // Used for rollback on publish failure
}
PayloadPublisher publishes payload events to the provisioner.
type PlacementLookup ¶
PlacementLookup provides lease→backend mapping for read-path routing.
type PortMapping ¶
PortMapping represents a port binding from container to host.
type RateLimiter ¶
type RateLimiter struct {
// contains filtered or unexported fields
}
RateLimiter implements per-IP rate limiting using a token bucket algorithm.
func NewRateLimiter ¶
func NewRateLimiter(rps float64, burst int, trustedProxies *TrustedProxyConfig) *RateLimiter
NewRateLimiter creates a new rate limiter. rps is requests per second, burst is the maximum burst size. trustedProxies is optional - if nil or empty, X-Forwarded-For headers are ignored.
func (*RateLimiter) Middleware ¶
func (rl *RateLimiter) Middleware(next http.Handler) http.Handler
Middleware returns an HTTP middleware that enforces rate limiting.
type RestoreInFlightTracker ¶ added in v0.5.0
type RestoreInFlightTracker interface {
// TryTrackRestoreInFlight registers the new lease as an in-flight restore.
// Returns false if the lease is already in-flight (a duplicate restore or a
// racing reconciler provision), in which case the caller must NOT call the
// backend and must NOT untrack — the entry is owned by the other writer.
TryTrackRestoreInFlight(leaseUUID, tenant string, items []backend.LeaseItem, backendName string) bool
// UntrackInFlight removes the entry. Called only to undo a successful track
// when the synchronous Restore() call subsequently fails, so a failed restore
// never leaves a phantom in-flight lease for the TimeoutChecker to reject.
UntrackInFlight(leaseUUID string)
}
RestoreInFlightTracker registers the NEW restore lease in the provisioner's in-flight tracker so the restore's provision callback is acknowledged inline (like a fresh provision) instead of ~one reconciler interval later (ENG-358). Optional — when nil, restore still converges via the reconciler ack backstop.
type RestorePlacementRecorder ¶ added in v0.5.0
type RestorePlacementRecorder interface {
RecordRestorePlacement(newLeaseUUID, backendName string)
}
RestorePlacementRecorder records the NEW lease's placement after a successful restore (it adopts the source's backend). The source placement is left for the reconciler to prune. Optional — when nil, the reconciler still converges (ENG-333).
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server is the HTTP API server.
func NewServer ¶
func NewServer(cfg ServerConfig, deps ServerDeps) (*Server, error)
NewServer creates a new API server. Returns an error if token tracker initialization fails.
func (*Server) Start ¶
Start begins serving HTTP requests and blocks until context is canceled or error. When the context is canceled, the server is gracefully shut down before returning.
func (*Server) StartBackground ¶
StartBackground starts the server in the background and returns immediately once the server is listening. Returns an error channel that will receive any server errors. This is useful when you need to ensure the server is ready before proceeding with other startup tasks (e.g., reconciliation that triggers callbacks).
type ServerConfig ¶
type ServerConfig struct {
Addr string
ProviderUUID string
Bech32Prefix string
TLSCertFile string
TLSKeyFile string
RateLimitRPS float64
RateLimitBurst int
TenantRateLimitRPS float64 // Per-tenant rate limit (requests per second), 0 = disabled
TenantRateLimitBurst int // Per-tenant burst limit
TrustedProxies []string // CIDR blocks of trusted reverse proxies for X-Forwarded-For
CORSOrigins []string // Allowed CORS origins (e.g. ["*"] for all). Empty or nil disables CORS middleware.
ReadTimeout time.Duration
WriteTimeout time.Duration
IdleTimeout time.Duration
RequestTimeout time.Duration // Timeout for individual request processing (default: 30s)
ShutdownTimeout time.Duration // Timeout for graceful shutdown (default: 30s)
MaxRequestBodySize int64
CallbackSecret string // HMAC secret for callback authentication
CallbackCanonicalPathPrefix string // Path prefix prepended to inbound URIs before HMAC verification (proxy stripPrefix compensation)
TokenTrackerDBPath string // Path to token tracker database (enables replay protection)
CallbackBaseURL string // Base URL for backend callbacks (used by restart/update)
}
ServerConfig holds configuration for the API server.
type ServerDeps ¶
type ServerDeps struct {
ChainClient ChainClient
BackendRouter *backend.Router
CallbackPublisher CallbackPublisher
PayloadPublisher PayloadPublisher
StatusChecker StatusChecker
PlacementLookup PlacementLookup // Optional — if nil, placement routing is disabled.
RestoreRecorder RestorePlacementRecorder // Optional — restore placement bookkeeping (ENG-333).
RestoreTracker RestoreInFlightTracker // Optional — inline-ack restore in-flight tracking (ENG-358).
EventBroker *EventBroker // Optional — if nil, the events endpoint returns 501.
}
ServerDeps holds the runtime dependencies for the API server. These are the collaborators injected into the server at startup.
type ServiceConnectionDetails ¶
type ServiceConnectionDetails struct {
FQDN string `json:"fqdn,omitempty"`
Instances []InstanceInfo `json:"instances"`
}
ServiceConnectionDetails contains connection details for a single service in a stack.
type StatusChecker ¶
type StatusChecker interface {
HasPayload(leaseUUID string) (bool, error)
IsInFlight(leaseUUID string) bool
InFlightCount() int
}
StatusChecker provides status information about provisioning. Typically implemented by the provisioner.Manager.
type TenantRateLimiter ¶
type TenantRateLimiter struct {
// contains filtered or unexported fields
}
TenantRateLimiter implements per-tenant rate limiting using a token bucket algorithm. This is used for authenticated endpoints where the tenant identity is known. Tokens are cryptographically validated before consuming from the bucket to prevent attackers from burning a victim's quota with forged tokens.
func NewTenantRateLimiter ¶
func NewTenantRateLimiter(rps float64, burst int, bech32Prefix string) *TenantRateLimiter
NewTenantRateLimiter creates a new per-tenant rate limiter. rps is requests per second, burst is the maximum burst size per tenant. bech32Prefix is used for cryptographic token validation before bucket consumption.
func (*TenantRateLimiter) Allow ¶
func (tl *TenantRateLimiter) Allow(tenant string) bool
Allow checks if a request from the tenant is allowed.
func (*TenantRateLimiter) AuthMiddleware ¶
func (tl *TenantRateLimiter) AuthMiddleware() func(http.Handler) http.Handler
AuthMiddleware returns middleware that validates AuthTokens and applies per-tenant rate limiting. Tokens are cryptographically validated BEFORE consuming from the bucket, preventing attackers from burning a victim's quota with forged tokens. The validated token is stored in request context so handlers skip re-validation.
func (*TenantRateLimiter) PayloadAuthMiddleware ¶
func (tl *TenantRateLimiter) PayloadAuthMiddleware() func(http.Handler) http.Handler
PayloadAuthMiddleware returns middleware that validates PayloadAuthTokens and applies per-tenant rate limiting. Same validate-before-consume pattern as AuthMiddleware. Increments PayloadUploadsTotal{invalid_auth} on rejection so the metric fires regardless of whether rate limiting is enabled (the handler fallback path also increments on its own rejections).
type TokenTracker ¶
type TokenTracker struct {
// contains filtered or unexported fields
}
TokenTracker tracks used authentication tokens to prevent replay attacks. It uses bbolt for persistence across restarts.
func NewTokenTracker ¶
func NewTokenTracker(cfg TokenTrackerConfig) (*TokenTracker, error)
NewTokenTracker creates a new token tracker with bbolt persistence.
func (*TokenTracker) Close ¶
func (t *TokenTracker) Close() error
Close shuts down the token tracker gracefully. Close is idempotent and safe to call multiple times.
func (*TokenTracker) Healthy ¶
func (t *TokenTracker) Healthy() error
Healthy checks if the bbolt database is accessible and the token bucket exists.
func (*TokenTracker) TryUse ¶
func (t *TokenTracker) TryUse(key string) error
TryUse attempts to mark a token as used. Returns nil if the token was successfully marked (first use). Returns ErrTokenAlreadyUsed if the token has already been used. The key should be the token's signature (unique per token).
type TokenTrackerConfig ¶
type TokenTrackerConfig struct {
DBPath string // Path to bbolt database file
MaxAge time.Duration // How long to track tokens (should match MaxTokenAge)
CleanupInterval time.Duration // How often to clean up expired entries
}
TokenTrackerConfig configures the token tracker.
type TokenTrackerInterface ¶
type TokenTrackerInterface interface {
TryUse(signature string) error
Healthy() error
Close() error
}
TokenTrackerInterface defines the interface for token replay protection. This interface allows for testing with mock implementations.
type TrustedProxyConfig ¶
type TrustedProxyConfig struct {
// contains filtered or unexported fields
}
TrustedProxyConfig holds parsed trusted proxy CIDR ranges.
func NewTrustedProxyConfig ¶
func NewTrustedProxyConfig(cidrs []string) *TrustedProxyConfig
NewTrustedProxyConfig parses CIDR strings into a trusted proxy configuration. Invalid CIDR strings are logged and skipped.
func (*TrustedProxyConfig) IsTrusted ¶
func (c *TrustedProxyConfig) IsTrusted(ipStr string) bool
IsTrusted returns true if the given IP address is within a trusted proxy range.
type WorkloadEntry ¶
type WorkloadEntry struct {
Status backend.ProvisionStatus `json:"status"`
CreatedAt time.Time `json:"created_at"`
BackendName string `json:"backend_name"`
Items []WorkloadItem `json:"items"`
}
WorkloadEntry describes a single lease's workload for observability. LeaseUUID is intentionally absent — the map key in WorkloadLookupResponse carries it.
type WorkloadItem ¶
type WorkloadItem struct {
ServiceName string `json:"service_name,omitempty"`
SKU string `json:"sku"`
Image string `json:"image,omitempty"`
Count int `json:"count"`
}
WorkloadItem describes a single SKU+image within a workload.
type WorkloadLookupResponse ¶
type WorkloadLookupResponse struct {
Workloads map[string]WorkloadEntry `json:"workloads"`
Warnings []string `json:"warnings"`
}
WorkloadLookupResponse is the response from the GET /workloads endpoint. Workloads is a map keyed by lease_uuid so callers can join by UUID without building a client-side index. Unknown leases are absent from the map. Warnings is non-nil and may be empty (initialized as []string{} for stable JSON serialization as `[]` rather than `null`).