util

Documentation

Index

• func RetryWithFixedDelay(retries uint, wait time.Duration, f func() (interface{}, error)) (interface{}, error)
• type Election
• type Elector
  • func NewElector(election Election) *Elector
  • func (e *Elector) BecomeLeader() (bool, error)
  • func (e *Elector) ID() string
  • func (e *Elector) IsLeader() (bool, error)
  • func (e *Elector) Resign() error
• type PgAdvisoryLock
  • func NewPgAdvisoryLock(groupLockID int, connPool *sql.DB) (*PgAdvisoryLock, error)
  • func (l *PgAdvisoryLock) BecomeLeader() (bool, error)
  • func (l *PgAdvisoryLock) ID() string
  • func (l *PgAdvisoryLock) IsLeader() (bool, error)
  • func (l *PgAdvisoryLock) Locked() bool
  • func (l *PgAdvisoryLock) Release() error
  • func (l *PgAdvisoryLock) Resign() error
  • func (l *PgAdvisoryLock) TryLock() (bool, error)
• type RestElection
  • func NewRestElection() *RestElection
  • func (r *RestElection) BecomeLeader() (bool, error)
  • func (r *RestElection) ID() string
  • func (r *RestElection) IsLeader() (bool, error)
  • func (r *RestElection) Resign() error
• type ScheduledElector
  • func NewScheduledElector(election Election, electionInterval time.Duration) *ScheduledElector
  • func (se *ScheduledElector) Elect() (bool, error)
  • func (se *ScheduledElector) IsPausedScheduledElection() bool
  • func (se *ScheduledElector) PrometheusLivenessCheck(lastRequestUnixNano int64, timeout time.Duration)
• type ThroughputCalc
  • func NewThroughputCalc(interval time.Duration) *ThroughputCalc
  • func (dt *ThroughputCalc) SetCurrent(value float64)
  • func (dt *ThroughputCalc) Start()

Functions

func RetryWithFixedDelay

func RetryWithFixedDelay(retries uint, wait time.Duration, f func() (interface{}, error)) (interface{}, error)

Blocking retry with a fixed delay

Types

type Election

type Election interface {
	ID() string
	BecomeLeader() (bool, error)
	IsLeader() (bool, error)
	Resign() error
}

Election defines an interface for adapter leader election. If you are running Prometheus in HA mode where each Prometheus instance sends data to corresponding adapter you probably want to allow writes into the database from only one adapter at the time. We need to elect a leader who can write to the database. If leader goes down, another leader is elected. Look at `lock.go` for an implementation based on PostgreSQL advisory locks. Should be easy to plug in different leader election implementations.

type Elector

type Elector struct {
	// contains filtered or unexported fields
}

Elector is `Election` wrapper that provides cross-cutting concerns(eg. logging) and some common features shared among all election implementations.

func NewElector

func NewElector(election Election) *Elector

func (*Elector) BecomeLeader

func (e *Elector) BecomeLeader() (bool, error)

func (*Elector) ID

func (e *Elector) ID() string

func (*Elector) IsLeader

func (e *Elector) IsLeader() (bool, error)

func (*Elector) Resign

func (e *Elector) Resign() error

type PgAdvisoryLock

type PgAdvisoryLock struct {
	// contains filtered or unexported fields
}

PgAdvisoryLock is implementation of leader election based on PostgreSQL advisory locks. All adapters withing a HA group are trying to obtain an advisory lock for particular group. The one who holds the lock can write to the database. Due to the fact that Prometheus HA setup provides no consistency guarantees this implementation is best effort in regards to metrics that is written (some duplicates or data loss are possible during fail-over) `leader-election-pg-advisory-lock-prometheus-timeout` config must be set when using PgAdvisoryLock. It will trigger leader resign (if instance is a leader) and will prevent an instance to become a leader if there are no requests coming from Prometheus within a given timeout. Make sure to provide a reasonable value for the timeout (should be co-related with Prometheus scrape interval, eg. 2x or 3x more then scrape interval to prevent leader flipping). Recommended architecture when using PgAdvisoryLock is to have one adapter instance for one Prometheus instance.

func NewPgAdvisoryLock

func NewPgAdvisoryLock(groupLockID int, connPool *sql.DB) (*PgAdvisoryLock, error)

NewPgAdvisoryLock creates a new instance with specified lock ID, connection pool and lock timeout.

func (*PgAdvisoryLock) BecomeLeader

func (l *PgAdvisoryLock) BecomeLeader() (bool, error)

BecomeLeader tries to become a leader by acquiring the lock.

func (*PgAdvisoryLock) ID

func (l *PgAdvisoryLock) ID() string

ID returns the group lock ID for this instance.

func (*PgAdvisoryLock) IsLeader

func (l *PgAdvisoryLock) IsLeader() (bool, error)

IsLeader returns the current leader status for this instance.

func (*PgAdvisoryLock) Locked

func (l *PgAdvisoryLock) Locked() bool

Locked returns if the instance was able to obtain the leader lock.

func (*PgAdvisoryLock) Release

func (l *PgAdvisoryLock) Release() error

Release releases the already obtained leader lock.

func (*PgAdvisoryLock) Resign

func (l *PgAdvisoryLock) Resign() error

Resign releases the leader status of this instance.

func (*PgAdvisoryLock) TryLock

func (l *PgAdvisoryLock) TryLock() (bool, error)

TryLock tries to obtain the lock if its not already the leader. In the case that it is the leader, it verifies the connection to make sure the lock hasn't been already lost.

type RestElection

type RestElection struct {
	// contains filtered or unexported fields
}

RestElection is a REST interface allowing to plug in any external leader election mechanism. Remote service can use REST endpoints to manage leader election thus block or allow writes. Using RestElection over PgAdvisoryLock is encouraged as it is more robust and gives more control over the election process, however it does require additional engineering effort.

func NewRestElection

func NewRestElection() *RestElection

func (*RestElection) BecomeLeader

func (r *RestElection) BecomeLeader() (bool, error)

func (*RestElection) ID

func (r *RestElection) ID() string

func (*RestElection) IsLeader

func (r *RestElection) IsLeader() (bool, error)

func (*RestElection) Resign

func (r *RestElection) Resign() error

type ScheduledElector

type ScheduledElector struct {
	Elector
	// contains filtered or unexported fields
}

ScheduledElector triggers election on scheduled interval. Currently used in combination with PgAdvisoryLock

func NewScheduledElector

func NewScheduledElector(election Election, electionInterval time.Duration) *ScheduledElector

func (*ScheduledElector) Elect

func (se *ScheduledElector) Elect() (bool, error)

func (*ScheduledElector) IsPausedScheduledElection

func (se *ScheduledElector) IsPausedScheduledElection() bool

func (*ScheduledElector) PrometheusLivenessCheck

func (se *ScheduledElector) PrometheusLivenessCheck(lastRequestUnixNano int64, timeout time.Duration)

type ThroughputCalc

type ThroughputCalc struct {
	Values chan float64
	// contains filtered or unexported fields
}

ThroughputCalc runs on scheduled interval to calculate the throughput per second and sends results to a channel

func NewThroughputCalc

func NewThroughputCalc(interval time.Duration) *ThroughputCalc

func (*ThroughputCalc) SetCurrent

func (dt *ThroughputCalc) SetCurrent(value float64)

func (*ThroughputCalc) Start

func (dt *ThroughputCalc) Start()