verifier

Documentation

Index

• Constants
• func BackupAndDelete(ctx context.Context, logger log.Logger, bkt, backupBkt objstore.Bucket, ...) error
• func BackupAndDeleteDownloaded(ctx context.Context, logger log.Logger, bdir string, ...) error
• func DuplicatedCompactionIssue(ctx context.Context, logger log.Logger, bkt objstore.Bucket, ...) error
• func IndexIssue(ctx context.Context, logger log.Logger, bkt objstore.Bucket, ...) error
• func OverlappedBlocksIssue(ctx context.Context, logger log.Logger, _ objstore.Bucket, _ objstore.Bucket, ...) error
• func TSDBBlockExistsInBucket(ctx context.Context, bkt objstore.Bucket, id ulid.ULID) (bool, error)
• type Issue
• type Verifier
  • func New(logger log.Logger, reg prometheus.Registerer, bkt objstore.Bucket, ...) *Verifier
  • func NewWithRepair(logger log.Logger, reg prometheus.Registerer, bkt objstore.Bucket, ...) *Verifier
  • func (v *Verifier) Verify(ctx context.Context, idMatcher func(ulid.ULID) bool) error

Constants

const DuplicatedCompactionIssueID = "duplicated_compaction"

const IndexIssueID = "index_issue"

const OverlappedBlocksIssueID = "overlapped_blocks"

Functions

func BackupAndDelete

func BackupAndDelete(ctx context.Context, logger log.Logger, bkt, backupBkt objstore.Bucket, id ulid.ULID, deleteDelay time.Duration, blocksMarkedForDeletion prometheus.Counter) error

BackupAndDelete moves a TSDB block to a backup bucket and on success removes it from the source bucket. If deleteDelay is zero, block is removed from source bucket. else the block is marked for deletion. It returns error if block dir already exists in the backup bucket (blocks should be immutable) or if any of the operations fail.

func BackupAndDeleteDownloaded

func BackupAndDeleteDownloaded(ctx context.Context, logger log.Logger, bdir string, bkt, backupBkt objstore.Bucket, id ulid.ULID, deleteDelay time.Duration, blocksMarkedForDeletion prometheus.Counter) error

BackupAndDeleteDownloaded works much like BackupAndDelete in that it will move a TSDB block from a bucket to a backup bucket. If deleteDelay param is zero, block is removed from source bucket. else the block is marked for deletion. The bdir parameter points to the location on disk where the TSDB block was previously downloaded allowing this function to avoid downloading the TSDB block from the source bucket again. An error is returned if any operation fails.

func DuplicatedCompactionIssue

func DuplicatedCompactionIssue(ctx context.Context, logger log.Logger, bkt objstore.Bucket, backupBkt objstore.Bucket, repair bool, idMatcher func(ulid.ULID) bool, fetcher block.MetadataFetcher, deleteDelay time.Duration, metrics *verifierMetrics) error

DuplicatedCompactionIssue was a bug fixed in https://github.com/thanos-io/thanos/commit/94e26c63e52ba45b713fd998638d0e7b2492664f. Bug resulted in source block not being removed immediately after compaction, so we were compacting again and again same sources until sync-delay passes. The expected print of this are same overlapped blocks with exactly the same sources, time ranges and stats. If repair is enabled, all but one duplicates are safely deleted.

func IndexIssue

func IndexIssue(ctx context.Context, logger log.Logger, bkt objstore.Bucket, backupBkt objstore.Bucket, repair bool, idMatcher func(ulid.ULID) bool, fetcher block.MetadataFetcher, deleteDelay time.Duration, metrics *verifierMetrics) error

IndexIssue verifies any known index issue. It rewrites the problematic blocks while fixing repairable inconsistencies. If the replacement was created successfully it is uploaded to the bucket and the input block is deleted. NOTE: This also verifies all indexes against chunks mismatches and duplicates.

func OverlappedBlocksIssue

func OverlappedBlocksIssue(ctx context.Context, logger log.Logger, _ objstore.Bucket, _ objstore.Bucket, repair bool, idMatcher func(ulid.ULID) bool, fetcher block.MetadataFetcher, _ time.Duration, _ *verifierMetrics) error

OverlappedBlocksIssue checks bucket for blocks with overlapped time ranges. No repair is available for this issue.

func TSDBBlockExistsInBucket

func TSDBBlockExistsInBucket(ctx context.Context, bkt objstore.Bucket, id ulid.ULID) (bool, error)

TSDBBlockExistsInBucket checks to see if a given TSDB block ID exists in a bucket. If so, true is returned. An error is returned on failure and in such case the boolean result has no meaning.

Types

type Issue

type Issue func(ctx context.Context, logger log.Logger, bkt objstore.Bucket, backupBkt objstore.Bucket, repair bool, idMatcher func(ulid.ULID) bool, fetcher block.MetadataFetcher, deleteDelay time.Duration, metrics *verifierMetrics) error

Issue is an function that does verification and repair only if repair arg is true. It should log affected blocks using warn level logs. It should be safe for issue to run on healthy bucket.

type Verifier

type Verifier struct {
	// contains filtered or unexported fields
}

Verifier runs given issues to verify if bucket is healthy.

func New

func New(logger log.Logger, reg prometheus.Registerer, bkt objstore.Bucket, fetcher block.MetadataFetcher, deleteDelay time.Duration, issues []Issue) *Verifier

New returns verifier that only logs affected blocks.

func NewWithRepair

func NewWithRepair(logger log.Logger, reg prometheus.Registerer, bkt objstore.Bucket, backupBkt objstore.Bucket, fetcher block.MetadataFetcher, deleteDelay time.Duration, issues []Issue) *Verifier

NewWithRepair returns verifier that logs affected blocks and attempts to repair them.

func (*Verifier) Verify

func (v *Verifier) Verify(ctx context.Context, idMatcher func(ulid.ULID) bool) error

Verify verifies registered issues.