kafka

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 26, 2026 License: MIT Imports: 10 Imported by: 0

README

kafkakit

Go version Go Reference License

A configurable Kafka toolkit for Go with synchronous and asynchronous publishing, batch support, and consumer subscriptions with message redelivery.

Contents

Client initialization

  • Connect() and Close() are idempotent
  • Unsubscribe() lets you unsubscribe from a specific topic without unsubscribing from the others
ctx, cancel := context.WithCancel(context.Background())
cfg := kafka.DefaultConfig()
cfg.DSN = "<address>:<port>"
cfg.GroupID = "test_group_id"
b, err := kafka.NewBroker(&cfg)
if err != nil {
    log.Fatalf("failed to initialize Kafka client: %s", err.Error())
}

err = b.Connect(ctx)
if err != nil {
    log.Fatalf("failed to connect Kafka client: %s", err.Error())
}
defer b.Close(ctx)

Configuration

Start with kafka.DefaultConfig(), then override the values you need. The defaults in the tables below refer to this function. NewBroker does not apply defaults to every zero-valued field: an empty Mode or zero MaxBytes fails validation, and ConnectionTest: false disables connection checking.

Configuration fields have json, yaml, and env tags for use with a decoder or environment loader of your choice. Load settings into the initialized config to preserve defaults for omitted fields. The library does not load configuration files or environment variables itself.

Field yaml / env Default Description
DSN dsn / KAFKA_DSN — Broker address. Required
Hosts hosts / KAFKA_HOSTS from DSN Comma-separated list of brokers. Falls back to DSN when empty
GroupID group_id / KAFKA_GROUP_ID — Consumer group. Required for subscriptions
Mode mode / KAFKA_MODE mixed producer, consumer, or mixed. Only affects which fields are required
MaxBytes max_bytes / KAFKA_MAX_BYTES 10 MB Maximum message size when reading
AutoCommit auto_commit / KAFKA_AUTO_COMMIT false Automatically commit offsets
NackBackoff nack_backoff / KAFKA_NACK_BACKOFF 1s Delay before redelivering a message after a handler error
ConnectionTest connection_test / KAFKA_CONNECTION_TEST true Check the connection during Connect()

Producer settings:

Field yaml / env Default Description
BatchSize batch_size / KAFKA_BATCH_SIZE 100 Number of messages to accumulate before sending a batch
BatchBytes batch_bytes / KAFKA_BATCH_BYTES 1 MB Maximum batch size in bytes
BatchTimeout batch_timeout / KAFKA_BATCH_TIMEOUT 10ms How long to wait before sending an incomplete batch
TopicCreateAttempts topic_create_attempts / KAFKA_TOPIC_CREATE_ATTEMPTS 0 Number of attempts to create a missing topic; automatic creation is disabled by default
TopicCreateBackoff topic_create_backoff / KAFKA_TOPIC_CREATE_BACKOFF 100ms Delay between attempts while waiting for the topic to appear in metadata

The client uses a single long-lived asynchronous Sarama producer. Messages from concurrent Publish and PublishBatch calls may be combined into one network batch. An incomplete batch is sent after BatchTimeout; reaching BatchSize or BatchBytes triggers an earlier send. Synchronous methods return only after Sarama reports success or failure for every submitted message.

Message encoding

Publish accepts any JSON-serializable value. Structs, maps, slices, strings, numbers, and booleans are encoded directly into the Kafka message body. A []byte value is sent unchanged. Publish always treats its argument as a single message, even when the value is a slice. These messages are published without Kafka headers.

A message envelope struct lets you provide Kafka headers. If the struct has exported Headers and Payload fields, and Headers is a struct, Payload is JSON-encoded as the message body, while fields in Headers with JSON tags become Kafka headers:

type Headers struct {
    MessageID string `json:"message_id"`
}
type Payload struct {
    Field1 string `json:"field1"`
    Field2 []string `json:"field2"`
    Field3 int64  `json:"field3"`
}
type Message struct {
    Headers Headers
    Payload Payload
}

The FromBrokerMessage function passed to each subscription handler performs the reverse conversion. The destination can be a pointer to any type that can be decoded from JSON, or a pointer to []byte to retrieve the raw body. For message envelopes, Kafka headers and the body are decoded separately.

Producer

  • With TopicCreateAttempts == 0, the library does not create missing topics. A positive value enables a limited number of creation attempts before publishing. This setting applies only to the producer; it is unused in consumer mode and affects only publishing in mixed mode
  • Publish() blocks until the message is sent to the broker
err = b.Publish(ctx, "topic-name", &exampleMessage)
if err != nil {
    log.Printf("failed to publish message to Kafka: %s", err.Error())
    return
}
log.Printf("message with ID %s sent successfully", id)
Batch publishing

PublishBatch() accepts a slice and submits its elements to a single long-lived asynchronous producer. Sarama builds the actual network batches according to BatchSize, BatchBytes, and BatchTimeout. Publish() always treats its argument as a single message.

msgs := []any{&firstMessage, &secondMessage, &thirdMessage}

err = b.PublishBatch(ctx, "topic-name", msgs)

PublishBatch() accepts any typed slice. Results follow the order of elements in the input slice.

Asynchronous publishing

PublishAsync and PublishBatchAsync submit messages to the same long-lived producer as their synchronous counterparts, but return before Kafka reports the final delivery result. Encoding and enqueueing happen before the method returns, so a call may temporarily block if the producer's input queue is full.

result := b.PublishAsync(ctx, "topic-name", &message)

// Other work can run here while Kafka processes the message.

<-result.Done()
if err := result.Err(); err != nil {
    return err
}

Access Err and MessageErrors only after Done is closed. PublishBatchAsync attempts to send every element that was successfully encoded, even if another element failed. Individual message results always follow the order of elements in the input slice:

result := b.PublishBatchAsync(ctx, "topic-name", messages)
<-result.Done()

if err := result.Err(); err != nil {
    messageErrors := result.MessageErrors()
    if messageErrors == nil {
        // The operation failed as a whole; individual results are unknown.
        return err
    }

    for index, messageErr := range messageErrors {
        if messageErr != nil {
            // messages[index] was not sent. nil indicates a successful send.
        }
    }
}

Err is nil only when publishing succeeds completely. For a partial batch failure, it returns PublishError, and MessageErrors contains one entry for each input message. MessageErrors returns nil if the operation ends before individual results are available, for example because producer initialization failed or the batch argument was invalid.

When the context is canceled, all pending results complete with the context error. A message already submitted to the producer may still reach Kafka, so its delivery status should be considered unknown after context cancellation.

Publish and PublishBatch use the same asynchronous implementation internally, but wait for it to complete before returning.

Error handling

Publishing errors take two forms. Distinguishing between them matters because they require different responses.

The entire call failed — a regular error is returned, and nothing was sent to the broker. Examples include an unavailable cluster or a failed topic creation request.

Part of the batch failed — a *PublishError is returned with a result for each message. The remaining messages have been published. Use BatchErrors to inspect the results:

if err := b.PublishBatch(ctx, "topic-name", msgs); err != nil {
    results, ok := kafka.BatchErrors(err)
    if !ok {
        // The entire batch was not sent; retry the whole batch.
        return err
    }

    for i, res := range results {
        if res != nil {
            log.Printf("message %d was not published: %s", i, res)
        }
        // res == nil means message msgs[i] was published.
    }
}

The results slice is aligned with the input: results[i] corresponds to msgs[i]. Messages that failed conversion are included in the same results and do not prevent the remaining messages from being sent.

You can also identify the cause without inspecting the slice: errors.Is and errors.As work with both error forms:

if errors.Is(err, context.DeadlineExceeded) {
    // ...
}

Consumer

Subscribe() subscribes the client to a topic and invokes the handler for each message.

  • Subscribe() never creates a missing topic
  • Returning nil from the handler acknowledges the message and allows the partition to advance
  • Returning an error from the handler rejects the message. The same message is redelivered after NackBackoff, and its partition does not advance until processing succeeds
  • The library does not impose a retry limit. After terminal handling or exhausting its own retry limit, the service must return nil to skip the message
  • Format and decoding errors are usually terminal: log them and return nil. Return transient errors, such as database unavailability, from the handler to trigger redelivery
done, err := b.Subscribe(ctx, "topic-name", handler)
if err != nil {
    log.Printf("failed to subscribe to Kafka: %s", err.Error())
}
Handler example
  • FromBrokerMessage is passed as the handler's last argument and can decode a message into any supported destination type
  • The error returned by the handler controls negative acknowledgment (nack); it is not sent through done
handler := func(_ context.Context, msg any, unmarshal func(any, any) error) error {
    var serializedMessage Message
    err := unmarshal(msg, &serializedMessage)
    if err != nil {
        log.Printf("skipping invalid message: %s", err.Error())
        return nil
    }
    log.Printf("message %s received successfully: %v", serializedMessage.Headers.MessageID, serializedMessage.Payload)
    // Process the message here. Return transient errors to trigger redelivery.
    return nil
}

Shutdown

The done channel returned by Subscribe() signals that processing has finished for one topic.

If you subscribe to multiple topics, wait for each of their done channels to complete.

The primary way to shut down is to cancel the context:

cancel()
<-done

Close() stops all subscriptions and closes the producer, flushing pending batches and releasing connections:

b.Close(ctx)
<-done

Unsubscribe() stops one subscription by topic name:

err := b.Unsubscribe(ctx, "topic-name")
if err != nil {
    // An error is returned if the subscription did not exist.
}
<-done // The channel returned by Subscribe for "topic-name".

For multiple topics:

var wg sync.WaitGroup
for _, ch := range doneChannels {
    wg.Add(1)
    go func(c <-chan struct{}) {
        defer wg.Done()
        <-c
    }(ch)
}
cancel() // Or b.Close(ctx).
wg.Wait()

Development

Use the Go version specified in go.mod and Make. To run tests, install Mockery v3 separately and make the mockery executable available on PATH. Mockery is not managed as a Go module tool dependency.

Generated mocks are not committed to the repository. After cloning, generate them before running tests locally or in CI:

make gen
make test

Run make gen again after changing any mocked interface. It uses .mockery.yml to generate mocks/mocks.gen.go for Controller, Subscription, and ControllerFactory. Neither test target generates mocks automatically.

Target Description
make gen Generate test mocks using the separately installed Mockery v3
make test Run all tests with verbose output (go test -v ./...)
make test-coverage Run all tests with coverage and generate coverage.out and coverage.html

To generate a coverage report after generating mocks:

make test-coverage

Open coverage.html in a browser to inspect coverage. gen files are local artifacts excluded by .gitignore; do not commit them or the generated mocks. Keep .mockery.yml and .golangci.yml in version control so contributors and CI share the same generation and lint settings.

Applications can import and use the library without generating mocks or installing Mockery; mocks are only required to run this repository's tests.

Documentation

Overview

Package kafka provides synchronous and asynchronous Kafka publishing and subscription methods.

Messages may be any JSON-serializable value. Byte slices are transported as raw payloads. Structs with exported Headers and Payload fields retain the envelope format: both parts use JSON, and JSON-tagged fields from the Headers struct are transported as Kafka headers. Other message forms are transported without Kafka headers.

The client supports producer-only, consumer-only, and mixed modes. All core components are exposed as interfaces for testing.

Index

Constants

View Source
const (
	// DefaultMode is the default broker usage mode.
	DefaultMode Mode = ModeMixed
	// DefaultMaxBytes is the default maximum Kafka message size.
	DefaultMaxBytes = 10 * 1024 * 1024
	// DefaultTopicCreateAttempts disables automatic topic creation by default.
	DefaultTopicCreateAttempts = 0
	// DefaultTopicCreateBackoff is the default delay between topic creation attempts.
	DefaultTopicCreateBackoff = 100 * time.Millisecond
	// DefaultBatchSize is the default number of messages buffered in a producer batch.
	DefaultBatchSize = 100
	// DefaultBatchBytes is the default producer batch size limit in bytes.
	DefaultBatchBytes int64 = 1024 * 1024
	// DefaultBatchTimeout is the default maximum wait for an incomplete producer batch.
	DefaultBatchTimeout = 10 * time.Millisecond
	// DefaultNackBackoff is the default delay before retrying a negatively acknowledged message.
	DefaultNackBackoff = time.Second
	// DefaultAutoCommit disables consumer offset auto commit by default.
	DefaultAutoCommit = false
	// DefaultConnectionTest enables the connection check by default.
	DefaultConnectionTest = true
)

Variables

View Source
var ErrWriterClosed = errors.New("kafka writer closed")

ErrWriterClosed is returned by Publish when the producer has been closed.

Functions

func BatchErrors

func BatchErrors(err error) ([]error, bool)

BatchErrors returns the per-message errors of a batch publish, aligned with the batch that was passed to PublishBatch.

func FromBrokerMessage

func FromBrokerMessage(msg any, target any) error

FromBrokerMessage deserializes a message received by a Subscribe handler into target. Target must be a non-nil pointer.

JSON is used for arbitrary values. A byte slice receives the raw Kafka payload. An envelope message with exported Headers and Payload fields decodes the payload and Kafka headers separately.

Types

type AcknowledgeableBrokerMessage

type AcknowledgeableBrokerMessage struct {
	BrokerMessage
	// contains filtered or unexported fields
}

AcknowledgeableBrokerMessage is a transport message with delivery control. It is exposed for controller and subscription test adapters; application handlers receive a *BrokerMessage as an opaque any value.

func NewAcknowledgeableBrokerMessage

func NewAcknowledgeableBrokerMessage(
	message BrokerMessage,
	acknowledgment messageAcknowledgment,
) AcknowledgeableBrokerMessage

NewAcknowledgeableBrokerMessage attaches delivery control to a broker message. It is intended for custom controller and subscription adapters.

func (AcknowledgeableBrokerMessage) Ack

Ack acknowledges the delivery when acknowledgment control is present.

func (AcknowledgeableBrokerMessage) Nak

Nak rejects the delivery when acknowledgment control is present.

type BatchPublishResult

type BatchPublishResult interface {
	PublishResult
	MessageErrors() []error
}

BatchPublishResult reports completion of an asynchronous batch publication. MessageErrors must only be inspected after Done is closed. It returns one error per input message when individual outcomes are available, or nil when the operation failed as a whole.

type Broker

type Broker struct {
	// contains filtered or unexported fields
}

Broker is a client for interacting with Kafka.

func NewBroker

func NewBroker(config *Config) (*Broker, error)

NewBroker creates a new Kafka client.

func (*Broker) Close

func (b *Broker) Close(ctx context.Context)

Close drops all active subscriptions (idempotently).

func (*Broker) Connect

func (b *Broker) Connect(_ context.Context) error

Connect establishes connection with Kafka (idempotently).

func (*Broker) Publish

func (b *Broker) Publish(ctx context.Context, topic string, msg any) error

Publish sends the message to the given topic. Broker controller should be connected - returns error otherwise. If the topic does not exist, it is created automatically unless TopicCreateAttempts is 0.

Any JSON-serializable value can be published. Byte slices are sent as raw payloads. A struct with exported Headers and Payload fields uses the envelope format: Payload is JSON encoded as the Kafka value, and JSON-tagged fields from the Headers struct become Kafka headers. Other message forms are published without Kafka headers.

Example envelope message:

type HeadersType struct{
	// Some serializable struct
}
type PayloadType struct{
	// Some serializable struct
}
type Message struct{
	Headers HeadersType
	Payload PayloadType
}

func (*Broker) PublishAsync

func (b *Broker) PublishAsync(ctx context.Context, topic string, msg any) PublishResult

PublishAsync begins publishing one message and returns its completion result. Encoding and submission may apply backpressure before this method returns. Done closes after Sarama reports the final delivery outcome.

func (*Broker) PublishBatch

func (b *Broker) PublishBatch(ctx context.Context, topic string, msgs any) error

PublishBatch submits every element of a slice to the given topic as one application-facing batch. Sarama determines the actual broker request boundaries from the configured batching limits. Broker controller should be connected - returns error otherwise. An empty slice is a successful no-op. Any non-slice value results in an error.

A partial failure is reported as a *PublishError holding one error per input element, which BatchErrors unpacks:

if err := broker.PublishBatch(ctx, topic, msgs); err != nil {
	results, ok := BatchErrors(err)
	if !ok {
		// the call failed as a whole, nothing was published
	}
	for i, res := range results {
		// res is nil when msgs[i] was published
	}
}

func (*Broker) PublishBatchAsync

func (b *Broker) PublishBatchAsync(ctx context.Context, topic string, msgs any) BatchPublishResult

PublishBatchAsync begins publishing every element of a slice and returns a completion result. All encodable messages are attempted even when another element fails. MessageErrors remains aligned with the original input order.

func (*Broker) Subscribe

func (b *Broker) Subscribe(
	ctx context.Context,
	topic string,
	handler func(ctx context.Context, msg any, unmarshal func(any, any) error) error,
) (<-chan struct{}, error)

Subscribe subscribes to the given topic and processes incoming messages using the provided handler. The broker must be connected via Connect() before subscribing. Missing topics are never created by subscribing.

On success, returns a channel that is closed when message processing stops (e.g., due to context cancellation or Unsubscribe), and a nil error. On failure, returns a closed channel and an error.

The handler receives each message as an opaque `any` value, which is actually a pointer to the library's broker message structure (*BrokerMessage):

type BrokerMessage struct {
	Headers map[string][]byte  // raw header values as byte slices
	Payload []byte             // raw message payload as byte slice
}

A built-in unmarshal helper (FromBrokerMessage) is provided to simplify deserialization. Its target may be a pointer to any JSON-decodable value. A byte slice receives the raw payload. Envelope structs with exported Headers and Payload fields decode Kafka headers and payload separately.

Example envelope message type for unmarshaling:

type HeadersType struct {
	CorrelationID string `json:"correlation_id"`
	Token      	  []byte `json:"token"`
}
type PayloadType struct {
	UserID string `json:"user_id"`
	Action string `json:"action"`
}
type Message struct {
	Headers HeadersType
	Payload PayloadType
}

Example handler using the built-in unmarshaler:

func handler(ctx context.Context, msg any, unmarshal func(any, any) error) error {
	var m Message
	if err := unmarshal(msg, &m); err != nil {
		log.Printf("discard malformed message: %v", err)
		return nil
	}
	// process m.Headers and m.Payload
	return nil
}

Using FromBrokerMessage is optional. A handler may type-assert *BrokerMessage to inspect the raw broker message directly.

A nil handler result acknowledges the message. An error negatively acknowledges it; the same message is delivered again after NackBackoff and its partition does not advance. To stop retrying, the service must handle the terminal failure itself and return nil. Malformed or undecryptable data is normally terminal, while transient dependency errors are normally returned.

func (*Broker) Unsubscribe

func (b *Broker) Unsubscribe(ctx context.Context, topic string) error

Unsubscribe cancels the topic subscription.

type BrokerMessage

type BrokerMessage struct {
	Headers map[string][]byte
	Payload []byte
}

BrokerMessage is the Kafka library's transport representation. Application code normally uses Publish and FromBrokerMessage instead.

type Config

type Config struct {
	ControllerFactory ControllerFactory // Factory for creating Kafka controllers; used for testing.
	SaramaFactory     SaramaFactory     // Factory for constructing Sarama clients; used for testing and customization.

	DSN string `json:"dsn" yaml:"dsn" env:"KAFKA_DSN"`
	// Consumer group ID, required for consumer. No defaults.
	GroupID string `json:"group_id" yaml:"group_id" env:"KAFKA_GROUP_ID"`
	// Default - ModeMixed.
	Mode Mode `json:"mode" yaml:"mode" env:"KAFKA_MODE"`
	// Broker addresses. When empty, DSN is used as the only address.
	Hosts []string `json:"hosts" yaml:"hosts" env:"KAFKA_HOSTS" env-separator:","`
	// Max message size (default - 10MB). Must be less than Sarama's
	// maximum request size.
	MaxBytes int `json:"max_bytes" yaml:"max_bytes" env:"KAFKA_MAX_BYTES"`
	// How many times a missing topic is created before publishing fails. The
	// default is 0, which disables automatic topic creation.
	TopicCreateAttempts int `json:"topic_create_attempts" yaml:"topic_create_attempts" env:"KAFKA_TOPIC_CREATE_ATTEMPTS"`
	// Delay between topic creation attempts, letting the cluster propagate
	// the new topic metadata (default - 100ms).
	TopicCreateBackoff time.Duration `json:"topic_create_backoff" yaml:"topic_create_backoff" env:"KAFKA_TOPIC_CREATE_BACKOFF"` //nolint:lll
	// How many messages a producer buffers before sending them as one batch
	// (default - 100).
	BatchSize int `json:"batch_size" yaml:"batch_size" env:"KAFKA_BATCH_SIZE"`
	// Max size of a batch in bytes (default - 1MB). Must be less than
	// Sarama's maximum request size.
	BatchBytes int64 `json:"batch_bytes" yaml:"batch_bytes" env:"KAFKA_BATCH_BYTES"`
	// How long an incomplete batch waits before being flushed anyway
	// (default - 10ms).
	BatchTimeout time.Duration `json:"batch_timeout" yaml:"batch_timeout" env:"KAFKA_BATCH_TIMEOUT"`
	// Delay before retrying a message whose handler returned an error
	// (default - 1s).
	NackBackoff time.Duration `json:"nack_backoff" yaml:"nack_backoff" env:"KAFKA_NACK_BACKOFF"`
	// Offset auto commit (default - false).
	AutoCommit bool `json:"auto_commit" yaml:"auto_commit" env:"KAFKA_AUTO_COMMIT"`
	// Default - true.
	ConnectionTest bool `json:"connection_test" yaml:"connection_test" env:"KAFKA_CONNECTION_TEST"`
}

Config is an internal structure for storing parameters.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns the default config.

type Controller

type Controller interface {
	Publish(ctx context.Context, topic string, msgs []BrokerMessage) ([]error, error)
	Subscribe(ctx context.Context, topic string) (Subscription, error)
	Close() error
}

Controller defines the interface for Kafka controller operations.

type ControllerAdapter

type ControllerAdapter struct {
	// contains filtered or unexported fields
}

ControllerAdapter implements Controller with Sarama-backed producer and consumer adapters.

func NewControllerAdapter

func NewControllerAdapter(
	writer *ControllerWriter,
	consumerGroupFactory consumerGroupFactory,
	nackBackoff time.Duration,
) *ControllerAdapter

NewControllerAdapter creates a new adapter for Kafka controller.

func (*ControllerAdapter) Close

func (ca *ControllerAdapter) Close() error

Close closes the producer adapter.

func (*ControllerAdapter) Publish

func (ca *ControllerAdapter) Publish(
	ctx context.Context,
	topic string,
	msgs []BrokerMessage,
) ([]error, error)

Publish writes the messages through the asynchronous producer adapter.

func (*ControllerAdapter) Subscribe

func (ca *ControllerAdapter) Subscribe(ctx context.Context, topic string) (Subscription, error)

Subscribe starts a Sarama consumer-group subscription.

type ControllerFactory

type ControllerFactory interface {
	New(cfg Config) (Controller, error)
}

ControllerFactory creates Kafka controllers.

type ControllerWriter

type ControllerWriter struct {
	// contains filtered or unexported fields
}

ControllerWriter adapts Sarama's asynchronous producer to synchronous and asynchronous per-message publish results.

func (*ControllerWriter) Close

func (c *ControllerWriter) Close() error

Close flushes the asynchronous producer, waits for the result dispatcher, closes the topic manager, and is idempotent.

func (*ControllerWriter) Publish

func (c *ControllerWriter) Publish(
	ctx context.Context,
	topic string,
	messages []BrokerMessage,
) ([]error, error)

Publish submits all messages to the asynchronous producer and blocks until every submitted message succeeds, fails, or the context is cancelled.

type Mode

type Mode = string

Mode is a preferred usage mode: producer-only, consumer-only or full client (mixed). Either choice affects only the list of required options.

const (
	// ModeProducer is used when the broker is only used to publish messages.
	// When this mode is set, consumer-specific options like GroupID are not required.
	ModeProducer Mode = "producer"

	// ModeConsumer is used when the broker is only used to subscribe to messages.
	// When this mode is set, GroupID option is required for consumer group management.
	ModeConsumer Mode = "consumer"

	// ModeMixed is used when the broker is used for both publishing and subscribing.
	// This is the default mode. When this mode is set, GroupID option is required
	// for any subscription operations.
	ModeMixed Mode = "mixed"
)

type PublishError

type PublishError struct {
	Errs []error
}

PublishError reports a batch publish in which some messages were written and some were not. A batch that failed as a whole is reported as a plain error instead, since there is nothing to tell apart.

func (*PublishError) Error

func (e *PublishError) Error() string

Error implements error interface.

func (*PublishError) Unwrap

func (e *PublishError) Unwrap() []error

Unwrap exposes the per-message errors to errors.Is and errors.As.

type PublishResult

type PublishResult interface {
	Done() <-chan struct{}
	Err() error
}

PublishResult reports completion of one asynchronous publication. Err must only be inspected after Done is closed.

type SaramaFactory

type SaramaFactory interface {
	NewClient(hosts []string, cfg *sarama.Config) (sarama.Client, error)
	NewAsyncProducer(hosts []string, cfg *sarama.Config) (sarama.AsyncProducer, error)
	NewClusterAdmin(hosts []string, cfg *sarama.Config) (sarama.ClusterAdmin, error)
	NewConsumerGroup(hosts []string, groupID string, cfg *sarama.Config) (sarama.ConsumerGroup, error)
}

SaramaFactory constructs the Sarama clients used by the Kafka controller. Applications normally use the default implementation. The interface allows deterministic adapter tests and specialized client construction.

type Subscription

type Subscription interface {
	MessagesChannel() <-chan AcknowledgeableBrokerMessage
	Cancel(ctx context.Context)
}

Subscription defines the interface for Kafka subscription operations.

type SubscriptionAdapter

type SubscriptionAdapter struct {
	// contains filtered or unexported fields
}

SubscriptionAdapter adapts a Sarama consumer group to Subscription.

func (*SubscriptionAdapter) Cancel

func (s *SubscriptionAdapter) Cancel(ctx context.Context)

Cancel stops the consumer group and waits for bounded cleanup.

func (*SubscriptionAdapter) MessagesChannel

func (s *SubscriptionAdapter) MessagesChannel() <-chan AcknowledgeableBrokerMessage

MessagesChannel returns messages delivered by the consumer group.

Directories

Path Synopsis
Package main is a simple library demo.
Package main is a simple library demo.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL