Documentation
¶
Index ¶
- Constants
- func NewActivePartitionPartitioner(cfg ActivePartitionConfig) sarama.PartitionerConstructor
- func NewConsistentHashPartition(topic string) sarama.Partitioner
- func OTelGlobalPropagator() propagation.TextMapPropagator
- func RetryWithBackoff(cfg RetryConfig, handler func() error) error
- type ActivePartitionConfig
- type AsyncProducer
- type ConsistentHashPartitioner
- type ConsumerGroup
- type ConsumerMessageConverter
- type ConsumerMessageConverterFunc
- type FallbackFunc
- type IsolationLevel
- type KafkaConfig
- type KafkaConfigOption
- func WithAsyncErrorHandler(fn func(*sarama.ProducerError)) KafkaConfigOption
- func WithAsyncSuccessHandler(fn func(*sarama.ProducerMessage)) KafkaConfigOption
- func WithConsumerWorkers(n int) KafkaConfigOption
- func WithDefaultPartitioner() KafkaConfigOption
- func WithLogger(logger Logger) KafkaConfigOption
- func WithPartitioner(partitioner sarama.PartitionerConstructor) KafkaConfigOption
- func WithProducerIdempotent(enable bool) KafkaConfigOption
- func WithTLSConfig(tlsConfig *tls.Config) KafkaConfigOption
- func WithTracePropagator(propagator propagation.TextMapPropagator) KafkaConfigOption
- type Logger
- type Message
- type MessageHandler
- func WrapWithFallbackHandler(handler MessageHandler[*sarama.ConsumerMessage], fallback FallbackFunc) MessageHandler[*sarama.ConsumerMessage]
- func WrapWithRetryBackoffHandler(handler MessageHandler[*sarama.ConsumerMessage], cfg RetryConfig) MessageHandler[*sarama.ConsumerMessage]
- func WrapWithSaramaMessageHandler[T any](handler MessageHandler[T], converter ConsumerMessageConverter[T]) MessageHandler[*sarama.ConsumerMessage]
- type MessageWithContext
- type MessageWithHeader
- type MessageWithPartitionKey
- type MessagesHandler
- func WrapWithFallbackBatchHandler(handler MessagesHandler[*sarama.ConsumerMessage], fallback FallbackFunc) MessagesHandler[*sarama.ConsumerMessage]
- func WrapWithRetryBackoffBatchHandler(handler MessagesHandler[*sarama.ConsumerMessage], cfg RetryConfig) MessagesHandler[*sarama.ConsumerMessage]
- func WrapWithSaramaMessagesHandler[T any](handler MessagesHandler[T], converter ConsumerMessageConverter[T]) MessagesHandler[*sarama.ConsumerMessage]
- type Option
- type PartitionFeedback
- type RetryConfig
- type SyncProducer
Constants ¶
const ( HeaderDeadLetterError = "dead-letter-error" HeaderDeadLetterTopic = "dead-letter-source-topic" HeaderDeadLetterPartition = "dead-letter-source-partition" HeaderDeadLetterOffset = "dead-letter-source-offset" )
Headers DeadLetterTopic adds to a dead-lettered message, next to the original headers.
const RecordHeaderKeyPartitionKey = "partition_key"
RecordHeaderKeyPartitionKey is the header carrying a message's partition key (see MessageWithPartitionKey).
Variables ¶
This section is empty.
Functions ¶
func NewActivePartitionPartitioner ¶
func NewActivePartitionPartitioner(cfg ActivePartitionConfig) sarama.PartitionerConstructor
NewActivePartitionPartitioner returns a partitioner that routes around partitions that keep failing, e.g. while a broker is down and its partitions wait for a new leader.
It partitions exactly like the default ConsistentHashPartitioner until a partition fails MaxFailures times in a row. That partition's circuit breaker then opens for OpenDuration, and messages hashed to it are re-hashed onto the partitions whose breaker is closed. After OpenDuration the partition gets messages again; one success closes the breaker, one failure re-opens it. If every partition is open, messages keep their normal partition.
Use it with WithPartitioner:
kafkaclient.NewSyncProducer(cfg, kafkaclient.WithPartitioner(
kafkaclient.NewActivePartitionPartitioner(kafkaclient.ActivePartitionConfig{})))
Trade-off: while a breaker is open, messages with the same key can land on different partitions, so per-key ordering is not guaranteed. Only use it for topics whose consumers don't rely on that ordering.
Messages without a key are spread randomly over the partitions whose breaker is closed.
Rerouting protects the messages sent after the breaker opened. A message that already failed is not moved: it is returned as an error (sync) or passed to the async error handler.
func NewConsistentHashPartition ¶
func NewConsistentHashPartition(topic string) sarama.Partitioner
NewConsistentHashPartition is the default partitioner. See ConsistentHashPartitioner.
func OTelGlobalPropagator ¶
func OTelGlobalPropagator() propagation.TextMapPropagator
OTelGlobalPropagator returns a propagator that delegates to OpenTelemetry's global propagator at call time. This is the default used when WithTracePropagator is not set.
func RetryWithBackoff ¶
func RetryWithBackoff(cfg RetryConfig, handler func() error) error
RetryWithBackoff calls handler until it succeeds, RetryCondition rejects the error, or MaxRetries is exhausted. Then it returns nil (the message is marked as consumed), or the last error when cfg.ReturnError is set.
Types ¶
type ActivePartitionConfig ¶
type ActivePartitionConfig struct {
// MaxFailures is the number of consecutive failed sends that opens a partition's circuit breaker. Default 3.
MaxFailures int
// OpenDuration is how long an open circuit breaker keeps a partition out of rotation before one probe
// message is allowed through again (half-open). Default 30s.
OpenDuration time.Duration
}
ActivePartitionConfig configures NewActivePartitionPartitioner.
type AsyncProducer ¶
type AsyncProducer interface {
// PublishRawAtMostOnce queues a message; see PublishAtMostOnce.
PublishRawAtMostOnce(ctx context.Context, topic string, messageKey string, payload []byte, headers map[string]string)
// PublishAtMostOnce queues msg and returns. It blocks only while sarama's input queue is full, and at most until
// ctx ends. Every message ends in exactly one delivery report: the success handler, or the error handler (also
// for messages that could not be encoded or queued before ctx ended).
PublishAtMostOnce(ctx context.Context, msg Message, opts ...Option)
// Close flushes buffered messages, waits for all delivery reports to be handled and releases resources.
Close() error
Unwrap() sarama.Client
}
func NewAsyncProducer ¶
func NewAsyncProducer(config KafkaConfig, opts ...KafkaConfigOption) (AsyncProducer, error)
NewAsyncProducer creates an asynchronous (at-most-once) producer.
type ConsistentHashPartitioner ¶
type ConsistentHashPartitioner struct {
// contains filtered or unexported fields
}
ConsistentHashPartitioner hashes the partition key (header "partition_key", set from MessageWithPartitionKey) or, without one, the message key, using xxhash + jump consistent hash. Messages with neither go to a random partition.
func (*ConsistentHashPartitioner) MessageRequiresConsistency ¶
func (c *ConsistentHashPartitioner) MessageRequiresConsistency(message *sarama.ProducerMessage) bool
MessageRequiresConsistency lets sarama pick among partitions that currently have a leader for messages without a key: they have no key → partition mapping to keep, so they avoid offline partitions. Keyed messages always see every partition, so the same key keeps landing on the same partition.
func (*ConsistentHashPartitioner) Partition ¶
func (c *ConsistentHashPartitioner) Partition(message *sarama.ProducerMessage, numPartitions int32) (int32, error)
func (*ConsistentHashPartitioner) RequiresConsistency ¶
func (c *ConsistentHashPartitioner) RequiresConsistency() bool
type ConsumerGroup ¶
type ConsumerGroup interface {
// Start starts the consumer in another goroutine. This returns error if the consumer is running.
Start() error
// Close releases the used resources.
Close()
// IsRunning returns true if the consumer is running.
IsRunning() bool
// Health returns error if consumer is not running. This is usually used in health check REST endpoint.
Health() error
// Run starts the consumer and blocks until ctx is canceled or the consumer stops on its own, then closes it.
// It has the signature of a gosoline kernel.Module.
Run(ctx context.Context) error
}
ConsumerGroup is an abstraction of kafka consumer group.
func NewBatchConsumerGroup ¶
func NewBatchConsumerGroup( cfg KafkaConfig, groupName string, topic string, batchSize uint16, batchTimeout time.Duration, ignoreOldMessage bool, handler MessagesHandler[*sarama.ConsumerMessage], opts ...KafkaConfigOption, ) (ConsumerGroup, error)
NewBatchConsumerGroup creates a consumer group that handles messages in batches of up to batchSize, flushing at least every batchTimeout.
func NewConsumerGroup ¶
func NewConsumerGroup( cfg KafkaConfig, consumerGroup string, topic string, ignoreOldMessage bool, handler MessageHandler[*sarama.ConsumerMessage], opts ...KafkaConfigOption, ) (ConsumerGroup, error)
NewConsumerGroup creates a consumer group that handles messages one at a time.
type ConsumerMessageConverter ¶
type ConsumerMessageConverter[T any] interface { Convert(*sarama.ConsumerMessage) (T, error) }
type ConsumerMessageConverterFunc ¶
type ConsumerMessageConverterFunc[T any] func(*sarama.ConsumerMessage) (T, error)
func (ConsumerMessageConverterFunc[T]) Convert ¶
func (c ConsumerMessageConverterFunc[T]) Convert(message *sarama.ConsumerMessage) (T, error)
type FallbackFunc ¶
FallbackFunc parks a message that its handler could not process, e.g. by writing it to a dead-letter topic or a queue. It returns nil once the message is stored safely; an error means the message must be redelivered.
func DeadLetterTopic ¶
func DeadLetterTopic(producer SyncProducer, topic string) FallbackFunc
DeadLetterTopic returns a FallbackFunc that publishes the failed message, unchanged (key, value, headers), to topic, adding the error and the original topic, partition and offset as headers.
type IsolationLevel ¶
type IsolationLevel string
const ( IsolationLevelReadCommitted IsolationLevel = "READ_COMMITTED" IsolationLevelReadUncommitted IsolationLevel = "READ_UNCOMMITTED" )
type KafkaConfig ¶
type KafkaConfig struct {
// A user-provided string sent with every request to the brokers for logging,
// debugging, and auditing purposes. Defaults to current host name, but you should
// probably set it to something specific to your application.
ClientID string `env:"KAFKA_CLIENT_ID" envconfig:"KAFKA_CLIENT_ID" cfg:"client_id"`
BootstrapServers []string `env:"KAFKA_BOOTSTRAP_SERVERS" envconfig:"KAFKA_BOOTSTRAP_SERVERS" cfg:"bootstrap_servers"`
SaslEnabled bool `env:"KAFKA_SASL_ENABLED" envconfig:"KAFKA_SASL_ENABLED" cfg:"sasl_enabled" default:"true"`
SaslMechanisms string `env:"KAFKA_SASL_MECHANISMS" envconfig:"KAFKA_SASL_MECHANISMS" cfg:"sasl_mechanisms" default:"PLAIN"`
SaslUsername string `env:"KAFKA_SASL_USERNAME" envconfig:"KAFKA_SASL_USERNAME" cfg:"sasl_username"`
SaslPassword string `env:"KAFKA_SASL_PASSWORD" envconfig:"KAFKA_SASL_PASSWORD" cfg:"sasl_password"`
TlsEnabled bool `env:"KAFKA_TLS_ENABLED" envconfig:"KAFKA_TLS_ENABLED" cfg:"tls_enabled" default:"true"` // nolint ST1003
IsolationLevel IsolationLevel `env:"KAFKA_ISOLATION_LEVEL" envconfig:"KAFKA_ISOLATION_LEVEL" cfg:"isolation_level" default:"READ_COMMITTED"`
AutoOffsetReset string `env:"KAFKA_AUTO_OFFSET_RESET" envconfig:"KAFKA_AUTO_OFFSET_RESET" cfg:"auto_offset_reset"`
EnableAutoCommit bool `env:"KAFKA_ENABLE_AUTO_COMMIT" envconfig:"KAFKA_ENABLE_AUTO_COMMIT" cfg:"enable_auto_commit"`
ProducerRetryMax int `env:"KAFKA_PRODUCER_RETRY_MAX" envconfig:"KAFKA_PRODUCER_RETRY_MAX" cfg:"producer_retry_max" default:"3"`
ProducerRetryBackoff time.Duration `env:"KAFKA_PRODUCER_RETRY_BACKOFF" envconfig:"KAFKA_PRODUCER_RETRY_BACKOFF" cfg:"producer_retry_backoff" default:"100ms"`
ProducerFlushMessages int `env:"KAFKA_PRODUCER_FLUSH_MESSAGES" envconfig:"KAFKA_PRODUCER_FLUSH_MESSAGES" cfg:"producer_flush_messages" default:"0"`
ProducerFlushMaxMessages int `` /* 131-byte string literal not displayed */
ProducerFlushBytes int `env:"KAFKA_PRODUCER_FLUSH_BYTES" envconfig:"KAFKA_PRODUCER_FLUSH_BYTES" cfg:"producer_flush_bytes" default:"1048576"`
ProducerFlushFrequency time.Duration `` /* 126-byte string literal not displayed */
ProducerRequiredAcks int16 `env:"KAFKA_PRODUCER_REQUIRED_ACKS" envconfig:"KAFKA_PRODUCER_REQUIRED_ACKS" cfg:"producer_required_acks" default:"-1"`
ProducerTimeout time.Duration `env:"KAFKA_PRODUCER_TIMEOUT" envconfig:"KAFKA_PRODUCER_TIMEOUT" cfg:"producer_timeout" default:"10s"`
ProducerIdempotent bool `env:"KAFKA_PRODUCER_IDEMPOTENT" envconfig:"KAFKA_PRODUCER_IDEMPOTENT" cfg:"producer_idempotent" default:"true"`
// ProducerHeartbeatEnabled If set to true, It will start the go routine heartbeat to keep the connection
// between producer and brokers alive which help reduce the latency of reconnection during the producer idle time.
ProducerHeartbeatEnabled bool `` /* 132-byte string literal not displayed */
// ProducerHeartbeatInterval control the interval time that the heartbeat sent the keepalive request to all available brokers.
ProducerHeartbeatInterval time.Duration `` /* 132-byte string literal not displayed */
CompressionType string `env:"KAFKA_COMPRESSION_TYPE" envconfig:"KAFKA_COMPRESSION_TYPE" cfg:"compression_type" default:"lz4"`
Debug bool `env:"SARAMA_DEBUG" envconfig:"SARAMA_DEBUG" cfg:"debug" default:"false"`
InsecureSkipVerify bool `env:"KAFKA_INSECURE_SKIP_VERIFY" envconfig:"KAFKA_INSECURE_SKIP_VERIFY" cfg:"insecure_skip_verify" default:"false"`
TLSConfigServerName string `env:"KAFKA_TLS_CONFIG_SERVER_NAME" envconfig:"KAFKA_TLS_CONFIG_SERVER_NAME" cfg:"tls_config_server_name"`
ConsumerGroupMaxRetry int `env:"KAFKA_CONSUMER_GROUP_MAX_RETRY" envconfig:"KAFKA_CONSUMER_GROUP_MAX_RETRY" cfg:"consumer_group_max_retry" default:"10"`
MetricsDisabled bool `env:"KAFKA_METRICS_DISABLED" envconfig:"KAFKA_METRICS_DISABLED" cfg:"metrics_disabled" default:"false"`
KeepAlive time.Duration `env:"KAFKA_KEEPALIVE" envconfig:"KAFKA_KEEPALIVE" cfg:"keepalive"`
SlowMessageThreshold time.Duration `env:"KAFKA_SLOW_MESSAGE_THRESHOLD" envconfig:"KAFKA_SLOW_MESSAGE_THRESHOLD" cfg:"slow_message_threshold" default:"5s"`
// NetDialTimeout, NetReadTimeout and NetWriteTimeout bound each broker connection attempt and request.
// A broker that vanished without closing its connections (e.g. a reclaimed node) is only detected when they
// expire. 0 = sarama's default of 30s.
NetDialTimeout time.Duration `env:"KAFKA_NET_DIAL_TIMEOUT" envconfig:"KAFKA_NET_DIAL_TIMEOUT" cfg:"net_dial_timeout"`
NetReadTimeout time.Duration `env:"KAFKA_NET_READ_TIMEOUT" envconfig:"KAFKA_NET_READ_TIMEOUT" cfg:"net_read_timeout"`
NetWriteTimeout time.Duration `env:"KAFKA_NET_WRITE_TIMEOUT" envconfig:"KAFKA_NET_WRITE_TIMEOUT" cfg:"net_write_timeout"`
// ProducerRetryMaxBufferLength and ProducerRetryMaxBufferBytes cap the messages a producer holds in memory for
// retry, e.g. while partitions have no leader. When full, the oldest are failed with
// sarama.ErrProducerRetryBufferOverflow (reported like any delivery failure) instead of growing without bound.
// 0 = the library default (10000 messages / 64 MiB); negative = unlimited. sarama raises values below
// 4096 messages / 32 MiB to those minimums.
ProducerRetryMaxBufferLength int `` /* 134-byte string literal not displayed */
ProducerRetryMaxBufferBytes int64 `` /* 131-byte string literal not displayed */
// contains filtered or unexported fields
}
func (KafkaConfig) ToSaramaConfig ¶
func (k KafkaConfig) ToSaramaConfig(opts ...KafkaConfigOption) *sarama.Config
type KafkaConfigOption ¶
type KafkaConfigOption interface {
// contains filtered or unexported methods
}
func WithAsyncErrorHandler ¶
func WithAsyncErrorHandler(fn func(*sarama.ProducerError)) KafkaConfigOption
WithAsyncErrorHandler sets a callback for every message an AsyncProducer failed to deliver. Defaults to logging the error through the configured Logger (a no-op if no logger is set).
func WithAsyncSuccessHandler ¶
func WithAsyncSuccessHandler(fn func(*sarama.ProducerMessage)) KafkaConfigOption
WithAsyncSuccessHandler sets a callback for every message acknowledged by an AsyncProducer.
func WithConsumerWorkers ¶
func WithConsumerWorkers(n int) KafkaConfigOption
WithConsumerWorkers handles each partition's messages with n goroutines instead of one (NewConsumerGroup only).
Messages are routed by key (the partition key header, else the message key): all messages with the same key go to the same worker, in order, so per-key ordering is kept. Messages without a key are spread over the workers. Offsets are committed only up to the oldest message not yet handled, so a restart never skips one.
n is per partition: a consumer that owns 6 partitions with n=4 runs up to 24 handlers at once. Handlers must be safe for concurrent use. n <= 1 keeps the default of one goroutine per partition.
When a handler returns an error, no new messages are started, the ones already running finish, and the session ends so the failed message (and everything after it that wasn't committed) is redelivered.
func WithDefaultPartitioner ¶
func WithDefaultPartitioner() KafkaConfigOption
func WithLogger ¶
func WithLogger(logger Logger) KafkaConfigOption
WithLogger opts in to logging. Without it, kafka-client logs nothing. When KafkaConfig.Debug is also true, sarama's internal logs are forwarded to this logger at debug level.
func WithPartitioner ¶
func WithPartitioner(partitioner sarama.PartitionerConstructor) KafkaConfigOption
func WithProducerIdempotent ¶
func WithProducerIdempotent(enable bool) KafkaConfigOption
func WithTLSConfig ¶
func WithTLSConfig(tlsConfig *tls.Config) KafkaConfigOption
func WithTracePropagator ¶
func WithTracePropagator(propagator propagation.TextMapPropagator) KafkaConfigOption
WithTracePropagator sets how trace context is written to (producer) and read from (consumer) message headers. Default: OpenTelemetry's global propagator (a no-op until an OpenTelemetry SDK registers one).
type Logger ¶
type Logger interface {
Debug(ctx context.Context, format string, args ...any)
Info(ctx context.Context, format string, args ...any)
Warn(ctx context.Context, format string, args ...any)
Error(ctx context.Context, format string, args ...any)
}
Logger is the logging interface used by kafka-client.
Logging is opt-in: nothing is logged unless a Logger is supplied via WithLogger (or RetryConfig.Logger). The interface is satisfied as-is by gosoline's log.Logger.
type Message ¶
type Message interface {
KafkaTopic() string
// KafkaMessageKey is a Kafka message key. By default, this is used to determine Kafka partition.
// An empty message key will cause a message to publish into a random partition.
// If you want to specify partition key that differ from the message key, you should implement MessageWithPartitionKey interface.
KafkaMessageKey() string
KafkaMessagePayload() ([]byte, error)
}
type MessageHandler ¶
MessageHandler handles a single message. ctx carries the trace context propagated in the message headers.
func WrapWithFallbackHandler ¶
func WrapWithFallbackHandler(handler MessageHandler[*sarama.ConsumerMessage], fallback FallbackFunc) MessageHandler[*sarama.ConsumerMessage]
WrapWithFallbackHandler calls fallback when handler fails, then treats the message as consumed, so one failing message doesn't block its partition. If fallback also fails, both errors are returned and the message is redelivered: it is never marked as consumed without being stored somewhere.
Combine it with retries by wrapping a retry handler that returns its final error:
kafkaclient.WrapWithFallbackHandler(
kafkaclient.WrapWithRetryBackoffHandler(handler, kafkaclient.RetryConfig{MaxRetries: 3, ReturnError: true, ...}),
kafkaclient.DeadLetterTopic(producer, "orders.dead-letter"),
)
func WrapWithRetryBackoffHandler ¶
func WrapWithRetryBackoffHandler(handler MessageHandler[*sarama.ConsumerMessage], cfg RetryConfig) MessageHandler[*sarama.ConsumerMessage]
WrapWithRetryBackoffHandler wraps a MessageHandler with retry and backoff logic. The wait between attempts ends early when ctx is canceled.
func WrapWithSaramaMessageHandler ¶
func WrapWithSaramaMessageHandler[T any](handler MessageHandler[T], converter ConsumerMessageConverter[T]) MessageHandler[*sarama.ConsumerMessage]
type MessageWithContext ¶
type MessageWithContext[T any] struct { // contains filtered or unexported fields }
func NewMessageWithContext ¶
func NewMessageWithContext[T any](ctx context.Context, msg T) *MessageWithContext[T]
NewMessageWithContext pairs a message with its context (e.g. to re-wrap messages in a handler middleware).
func (*MessageWithContext[T]) Get ¶
func (m *MessageWithContext[T]) Get() (context.Context, T)
type MessageWithHeader ¶
type MessageWithPartitionKey ¶
type MessageWithPartitionKey interface {
// KafkaPartitionKey is Kafka partition key. An empty partition key will cause a message to publish into a random partition.
KafkaPartitionKey() string
}
type MessagesHandler ¶
type MessagesHandler[T any] func(messages []*MessageWithContext[T]) error
MessagesHandler processes a batch of messages
func WrapWithFallbackBatchHandler ¶
func WrapWithFallbackBatchHandler(handler MessagesHandler[*sarama.ConsumerMessage], fallback FallbackFunc) MessagesHandler[*sarama.ConsumerMessage]
WrapWithFallbackBatchHandler is WrapWithFallbackHandler for batch consumers: when the batch handler fails, every message of the batch goes to fallback.
func WrapWithRetryBackoffBatchHandler ¶
func WrapWithRetryBackoffBatchHandler(handler MessagesHandler[*sarama.ConsumerMessage], cfg RetryConfig) MessagesHandler[*sarama.ConsumerMessage]
WrapWithRetryBackoffBatchHandler wraps a MessagesHandler with retry and backoff logic
func WrapWithSaramaMessagesHandler ¶
func WrapWithSaramaMessagesHandler[T any](handler MessagesHandler[T], converter ConsumerMessageConverter[T]) MessagesHandler[*sarama.ConsumerMessage]
type Option ¶
type Option func(*messageOptions)
Option tunes a single publish.
func WithTimestamp ¶
WithTimestamp sets the message timestamp (default: now).
type PartitionFeedback ¶
type PartitionFeedback interface {
OnSuccess(msg *sarama.ProducerMessage)
OnError(msg *sarama.ProducerMessage, err error)
}
PartitionFeedback is implemented by partitioners that want to know the outcome of every send. The sync and async producers report each acknowledged message to OnSuccess and each failed one to OnError.
type RetryConfig ¶
type RetryConfig struct {
MaxRetries int
BaseDelay time.Duration
MaxDelay time.Duration
RetryCondition func(error) bool
// ReturnError returns the last error once retries are exhausted or RetryCondition rejects it, instead of nil.
// Use it to chain a WrapWithFallbackHandler after the retries; on its own, an error blocks the partition.
ReturnError bool
// Logger is optional; retries are logged only when it is set.
Logger Logger
}
type SyncProducer ¶
type SyncProducer interface {
// PublishRawAtLeastOnce publishes a message and waits for the broker's acknowledgement.
// See PublishAtLeastOnce for the ctx semantics.
PublishRawAtLeastOnce(ctx context.Context, topic string, messageKey string, payload []byte, headers map[string]string) (partition int32, offset int64, err error)
// PublishAtLeastOnce publishes msg and waits for the broker's acknowledgement.
//
// It returns early with ctx.Err() when ctx is canceled or its deadline passes. The message may still be
// delivered afterwards: it can't be recalled from sarama's pipeline. Treat that error as "outcome unknown";
// retrying can create a duplicate.
PublishAtLeastOnce(ctx context.Context, msg Message, opts ...Option) (partition int32, offset int64, err error)
// Close stops accepting messages, waits for in-flight ones and releases the connection.
Close() error
Unwrap() sarama.Client
}
func NewSyncProducer ¶
func NewSyncProducer(config KafkaConfig, opts ...KafkaConfigOption) (SyncProducer, error)
NewSyncProducer creates a synchronous (acknowledged) producer.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
carrier
Package carrier exposes sarama message headers as OpenTelemetry TextMapCarriers.
|
Package carrier exposes sarama message headers as OpenTelemetry TextMapCarriers. |
|
cmd/waitkafka
command
Command waitkafka blocks until the broker in KAFKA_BOOTSTRAP_SERVERS (default localhost:9092) answers metadata requests, or fails after 2 minutes.
|
Command waitkafka blocks until the broker in KAFKA_BOOTSTRAP_SERVERS (default localhost:9092) answers metadata requests, or fails after 2 minutes. |
|
hash
Package hash maps keys to partitions with xxhash + jump consistent hash.
|
Package hash maps keys to partitions with xxhash + jump consistent hash. |
|
scram
Package scram adapts github.com/xdg-go/scram to sarama's SCRAMClient interface.
|
Package scram adapts github.com/xdg-go/scram to sarama's SCRAMClient interface. |
|
timeutil
Package timeutil is the library's clock.
|
Package timeutil is the library's clock. |