README
¶
Admission Package
Overview
The admission package provides a BoundedQueue object which is a semaphore implementation that limits the number of bytes admitted into a collector pipeline. Additionally the BoundedQueue limits the number of waiters that can block on a call to bq.Acquire(sz int64).
This package is an experiment to improve the behavior of Collector pipelines having their exporterhelper configured to apply backpressure. This package is meant to be used in receivers, via an interceptor or custom logic. Therefore, the BoundedQueue helps limit memory within the entire collector pipeline by limiting two dimensions that cause memory issues:
- bytes: large requests that enter the collector pipeline can require large allocations even if downstream components will eventually limit or ratelimit the request.
- waiters: limiting on bytes alone is not enough because requests that enter the pipeline and block on
bq.Acquire()can still consume memory within the receiver. If there are enough waiters this can be a significant contribution to memory usage.
Usage
Create a new BoundedQueue by calling bq := admission.NewBoundedQueue(maxLimitBytes, maxLimitWaiters)
Within the component call bq.Acquire(ctx, requestSize) which will either
- succeed immediately if there is enough available memory
- fail immediately if there are too many waiters
- block until context cancelation or enough bytes becomes available
Once a request has finished processing and is sent downstream call bq.Release(requestSize) to allow waiters to be admitted for processing. Release should only fail if releasing more bytes than previously acquired.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrTooManyWaiters = fmt.Errorf("rejecting request, too many waiters")
Functions ¶
This section is empty.
Types ¶
type BoundedQueue ¶
type BoundedQueue struct {
// contains filtered or unexported fields
}
func NewBoundedQueue ¶
func NewBoundedQueue(maxLimitBytes, maxLimitWaiters int64) *BoundedQueue
func (*BoundedQueue) Acquire ¶
func (bq *BoundedQueue) Acquire(ctx context.Context, pendingBytes int64) error
func (*BoundedQueue) Release ¶
func (bq *BoundedQueue) Release(pendingBytes int64) error
func (*BoundedQueue) TryAcquire ¶
func (bq *BoundedQueue) TryAcquire(pendingBytes int64) bool
Source Files