azservicebus

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 22, 2026 License: MIT Imports: 17 Imported by: 0

README

gopipe-azservicebus

Go Reference

Azure Service Bus adapter for gopipe — publish and subscribe with batching, backpressure, lock renewal, and OpenTelemetry metrics.

Quick Start

import (
    "github.com/fxsml/gopipe-azservicebus"
    "github.com/fxsml/gopipe/message"
)

client, _ := azservicebus.NewClient("Endpoint=sb://...")
Publish
pub := azservicebus.NewPublisher(client, "my-topic", azservicebus.PublisherConfig{})

// Stream from a channel — batches automatically
pub.Publish(ctx, messages)

// Or send a single batch
pub.PublishBatch(ctx, batch)
Subscribe
sub := azservicebus.NewSubscriber(client, "my-topic", "my-subscription", azservicebus.SubscriberConfig{
    MaxInFlight: 50,
})

ch, _ := sub.Subscribe(ctx)
for msg := range ch {
    // process msg
    msg.Ack()
}

Features

Feature Details
Batching Configurable batch size and flush timeout
Backpressure Semaphore-based in-flight limiting
Lock renewal Auto-detected from broker; configurable override
Reconnection Automatic receiver/sender recreation on failure
Graceful shutdown Drains in-flight messages before closing
Telemetry OpenTelemetry counters and histograms
CloudEvents Preserves CloudEvents AMQP properties
Auth Connection string or DefaultAzureCredential

Installation

go get github.com/fxsml/gopipe-azservicebus

CLI

A command-line tool for publishing and subscribing to Azure Service Bus using JSONL (JSON Lines).

Install
go install github.com/fxsml/gopipe-azservicebus/cmd/gopipe-azservicebus@latest
Connection

Pass the Service Bus connection string via flag or environment variable:

export SERVICEBUS_CONNECTION="Endpoint=sb://..."
# or per-command: --connection "Endpoint=sb://..."
# or via env file: --env-file .env.local
Publish

Read JSONL from stdin or a file and publish to a topic:

# from stdin
echo '{"specversion":"1.0","type":"com.example.event","source":"/test","id":"1"}' \
  | gopipe-azservicebus pub --topic my-topic

# from file
gopipe-azservicebus pub --topic my-topic --input-file events.jsonl
Subscribe

Receive messages and write them as JSONL to stdout or a file:

# to stdout (Ctrl+C to stop)
gopipe-azservicebus sub --subscription my-topic/my-subscription

# to file, stop after 100 messages or 30 seconds
gopipe-azservicebus sub --subscription my-topic/my-subscription \
  --output-file out.jsonl --limit 100 --timeout 30s

# non-destructive peek (messages are not acknowledged)
gopipe-azservicebus sub --subscription my-topic/my-subscription --peek

Known Issues

See KNOWN-ISSUES.md for documented edge cases and mitigations.

License

MIT

Documentation

Index

Constants

View Source
const DefaultLockRenewalInterval = 30 * time.Second

DefaultLockRenewalInterval is the fallback renewal interval when LockedUntil is unavailable.

Variables

View Source
var ErrPublisherClosed = errors.New("publisher is closed")

ErrPublisherClosed is returned when operations are attempted on a closed publisher.

View Source
var ErrSubscriberClosed = errors.New("subscriber is closed")

ErrSubscriberClosed is returned when operations are attempted on a closed subscriber.

View Source
var ErrSubscriptionActive = errors.New("subscription already active")

ErrSubscriptionActive is returned when Subscribe() is called while another subscription is active.

Functions

func NewClient

func NewClient(connOrNamespace string) (*azservicebus.Client, error)

NewClient creates a new Azure Service Bus client. If connOrNamespace starts with "Endpoint=", it is treated as a connection string. Otherwise, it is treated as a namespace and DefaultAzureCredential is used.

Types

type ErrorClass

type ErrorClass string

ErrorClass categorizes Service Bus errors for telemetry and handling.

const (
	ErrorClassNone       ErrorClass = ""
	ErrorClassThrottling ErrorClass = "throttling"
	ErrorClassConnection ErrorClass = "connection"
	ErrorClassTimeout    ErrorClass = "timeout"
	ErrorClassAuth       ErrorClass = "auth"
	ErrorClassNotFound   ErrorClass = "not_found"
	ErrorClassUnknown    ErrorClass = "unknown"
)

type Publisher

type Publisher struct {
	// contains filtered or unexported fields
}

Publisher publishes messages to a single Azure Service Bus topic or queue. One instance = one topic. Create multiple instances for multiple topics.

func NewPublisher

func NewPublisher(client *azservicebus.Client, topic string, config PublisherConfig) (*Publisher, error)

NewPublisher creates a publisher for a single topic.

func (*Publisher) Close

func (p *Publisher) Close() error

Close stops the publisher and waits for in-flight operations.

func (*Publisher) Publish

func (p *Publisher) Publish(ctx context.Context, pipeline string, in <-chan *message.Message) (<-chan struct{}, error)

Publish consumes messages from the input channel and publishes them to the topic. The pipeline parameter is used for telemetry correlation (e.g., "cache", "derived"). Messages are automatically batched for efficient network usage (see BatchSize, BatchTimeout). Returns a done channel that closes when all messages are published or the publisher is closed.

func (*Publisher) PublishBatch

func (p *Publisher) PublishBatch(ctx context.Context, pipeline string, messages ...*message.Message) error

PublishBatch publishes messages to the topic. The pipeline parameter is used for telemetry correlation (e.g., "cache", "derived"). Successfully published messages are Acked; on error, unpublished messages are Nacked.

type PublisherConfig

type PublisherConfig struct {
	// Logger for the publisher. Defaults to slog.Default().
	Logger *slog.Logger

	// ErrorHandler is called when batch publishing fails in streaming Publish().
	// The batch contains the messages that failed to publish.
	// Defaults to logging via Logger.
	ErrorHandler func(batch []*message.Message, err error)

	// PublishTimeout is the timeout for each publish operation.
	// Defaults to 60 seconds.
	PublishTimeout time.Duration

	// ShutdownTimeout is the maximum time allowed for graceful shutdown.
	// Defaults to 30 seconds.
	ShutdownTimeout time.Duration

	// OperationTimeout is the timeout for ServiceBus operations.
	// Defaults to 5 seconds.
	OperationTimeout time.Duration

	// BatchSize is the maximum number of messages per batch for streaming Publish.
	// Defaults to 100.
	BatchSize int

	// BatchTimeout is the maximum duration to wait before sending a partial batch.
	// Defaults to 1 second.
	BatchTimeout time.Duration

	// Worker is the number of concurrent workers for streaming Publish.
	// Defaults to 10.
	Worker int

	// Properties controls Azure Service Bus broker-level message properties.
	// Each field is a function receiving the outgoing message and returning the
	// property value; a nil function or empty/zero return value leaves the field unset.
	Properties PublisherProperties
}

PublisherConfig contains configuration for the Publisher.

type PublisherProperties

type PublisherProperties struct {
	SessionID            func(*message.Message) string
	ReplyTo              func(*message.Message) string
	ReplyToSessionID     func(*message.Message) string
	To                   func(*message.Message) string
	Subject              func(*message.Message) string
	PartitionKey         func(*message.Message) string
	TimeToLive           func(*message.Message) time.Duration
	ScheduledEnqueueTime func(*message.Message) time.Time
}

PublisherProperties holds per-field resolver functions for Azure Service Bus broker properties. Resolvers receive the outgoing message so values can be derived statically or dynamically. Nil functions and empty/zero return values are ignored.

type SubQueue added in v0.3.0

type SubQueue int

SubQueue selects a Service Bus sub-queue to receive from instead of the main queue or subscription.

const (
	// SubQueueNone receives from the main queue or subscription. This is the default.
	SubQueueNone SubQueue = iota

	// SubQueueDeadLetter receives from the dead-letter sub-queue, which holds messages
	// rejected by normal delivery rules (max delivery count exceeded, TTL expired,
	// or explicitly dead-lettered).
	SubQueueDeadLetter

	// SubQueueTransferDeadLetter receives from the transfer dead-letter sub-queue, which
	// holds messages that permanently failed while being auto-forwarded to another
	// queue or topic. Only relevant when the entity uses auto-forwarding (ForwardTo).
	SubQueueTransferDeadLetter
)

type Subscriber

type Subscriber struct {
	// contains filtered or unexported fields
}

Subscriber subscribes to a single Azure Service Bus topic/subscription or queue. One instance = one topic. Create multiple instances for multiple topics.

Lifecycle:

  • Subscribe(ctx) starts a subscription. The subscription ends when ctx is cancelled.
  • After context cancellation, the subscriber can be reused with a new Subscribe() call.
  • Close() permanently closes the subscriber and releases the receiver connection.
  • Only one active subscription is allowed at a time.

func NewSubscriber

func NewSubscriber(client *azservicebus.Client, topic, pipeline string, config SubscriberConfig) (*Subscriber, error)

NewSubscriber creates a subscriber for a single topic/subscription. Topic format: "topic/subscription" for subscriptions, or "queue" for queues. The pipeline parameter is used for telemetry labels (e.g., "cache", "derived").

func (*Subscriber) Close

func (s *Subscriber) Close() error

Close permanently closes the subscriber and releases the receiver connection. If a subscription is active, it will be gracefully drained first. After Close(), Subscribe() returns ErrSubscriberClosed. Close is safe to call multiple times.

func (*Subscriber) Subscribe

func (s *Subscriber) Subscribe(ctx context.Context, pipeline string) (<-chan *message.Message, error)

Subscribe starts receiving messages and returns a channel. The channel closes automatically when the context is cancelled, triggering graceful shutdown.

Subscription Lifecycle:

  • Only one subscription can be active at a time (returns ErrSubscriptionActive otherwise)
  • Context cancellation triggers graceful shutdown of the subscription
  • After shutdown completes, Subscribe() can be called again with a new context

Graceful Shutdown (context cancellation):

  1. Message reception stops immediately
  2. In-flight handlers continue processing naturally (NOT signaled yet)
  3. Wait up to ShutdownTimeout for handlers to settle (ack/nack)
  4. If timeout: signal handlers to abandon, messages will be redelivered
  5. Receiver stays open (subscriber can be reused)

Each message is handled by a dedicated goroutine that manages lock renewal and settlement. Subscribe starts consuming messages from the subscription. The pipeline parameter is used for telemetry correlation (e.g., "cache", "derived").

IMPORTANT: This method now updates semaphore metrics with pipeline/subscription labels. On first Subscribe() call, the semaphore labels are set and cannot be changed afterward.

type SubscriberConfig

type SubscriberConfig struct {
	// Logger for the subscriber. Defaults to slog.Default().
	Logger *slog.Logger

	// MaxInFlight is the maximum number of messages being processed concurrently.
	// This provides backpressure - new messages won't be received until in-flight
	// messages are acked/nacked. Defaults to 100.
	MaxInFlight int

	// PrefetchCount is the number of messages to fetch from ServiceBus at once.
	// Higher values improve throughput but increase memory usage.
	// Defaults to MaxInFlight if not set.
	PrefetchCount int

	// ShutdownTimeout is the maximum time allowed for graceful shutdown.
	// This is used when draining in-flight messages during context cancellation.
	// Defaults to 30 seconds.
	ShutdownTimeout time.Duration

	// OperationTimeout is the timeout for ServiceBus operations.
	// Defaults to 5 seconds.
	OperationTimeout time.Duration

	// ReconnectBackoff is the delay before retrying after a failed reconnect.
	// Defaults to 5 seconds.
	ReconnectBackoff time.Duration

	// LockRenewalInterval overrides the auto-detected renewal interval.
	// If zero (default), the interval is calculated per-message from LockedUntil:
	//   renewalInterval = time.Until(msg.LockedUntil) / 2
	// If non-zero, this value is used for all messages.
	LockRenewalInterval time.Duration

	// DisableLockRenewal disables automatic lock renewal.
	// Use only for short-processing scenarios where lock never expires.
	DisableLockRenewal bool

	// Properties maps Azure Service Bus broker properties onto the received message.
	// Each field is a function receiving the broker property value and the message to mutate.
	// Nil functions and nil broker property fields are skipped.
	Properties SubscriberProperties

	// EnablePeekMode causes the subscriber to use PeekMessages instead of ReceiveMessages.
	// Messages are read without acquiring a lock and are left on the bus after processing.
	// No settlement (complete/abandon) or lock renewal is performed.
	EnablePeekMode bool

	// PeekPollInterval is the delay between poll attempts when no messages are available
	// in peek mode. Ignored when EnablePeekMode is false. Defaults to 1 second.
	PeekPollInterval time.Duration

	// SubQueue selects a sub-queue to receive from instead of the main queue/subscription.
	// Defaults to SubQueueNone (the main queue/subscription).
	SubQueue SubQueue
}

SubscriberConfig contains configuration for the Subscriber.

type SubscriberProperties

type SubscriberProperties struct {
	SessionID            func(string, *message.Message)
	ReplyTo              func(string, *message.Message)
	ReplyToSessionID     func(string, *message.Message)
	To                   func(string, *message.Message)
	Subject              func(string, *message.Message)
	PartitionKey         func(string, *message.Message)
	TimeToLive           func(time.Duration, *message.Message)
	ScheduledEnqueueTime func(time.Time, *message.Message)

	// DeadLetterReason is set by the broker when a message was dead-lettered. Populated
	// regardless of which sub-queue is being read; typically only useful when SubQueue
	// is SubQueueDeadLetter or SubQueueTransferDeadLetter.
	DeadLetterReason func(string, *message.Message)

	// DeadLetterErrorDescription is set by the broker when a message was dead-lettered.
	// Populated regardless of which sub-queue is being read; typically only useful when
	// SubQueue is SubQueueDeadLetter or SubQueueTransferDeadLetter.
	DeadLetterErrorDescription func(string, *message.Message)
}

SubscriberProperties holds per-field mapping functions from Azure Service Bus broker properties to the received gopipe message. Functions receive the dereferenced property value and a mutable *message.Message and may write to msg.Attributes freely.

Directories

Path Synopsis
cmd
internal
semaphore
Package semaphore provides a weighted semaphore with optional OpenTelemetry metrics.
Package semaphore provides a weighted semaphore with optional OpenTelemetry metrics.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL