utils

package
v0.0.0-...-8586071 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 22, 2020 License: Apache-2.0 Imports: 42 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// LockFile represents file name,
	LockFile = "backup.lock"
	// MetaFile represents file name
	MetaFile = "backupmeta"
	// MetaJSONFile represents backup meta json file name
	MetaJSONFile = "backupmeta.json"
	// SavedMetaFile represents saved meta file name for recovering later
	SavedMetaFile = "backupmeta.bak"
)
View Source
const (
	// B is number of bytes in one byte.
	B = uint64(1) << (iota * 10)
	// KB is number of bytes in one kibibyte.
	KB
	// MB is number of bytes in one mebibyte.
	MB
	// GB is number of bytes in one gibibyte.
	GB
	// TB is number of bytes in one tebibyte.
	TB
)
View Source
const (

	// DefaultBRGCSafePointTTL means PD keep safePoint limit at least 5min
	DefaultBRGCSafePointTTL = 5 * 60
)

Variables

View Source
var (
	BRReleaseVersion = "None"
	BRBuildTS        = "None"
	BRGitHash        = "None"
	BRGitBranch      = "None"

	VersionHash = regexp.MustCompile("-[0-9]+-g[0-9a-f]{7,}")
)

Version information.

Functions

func ArchiveSize

func ArchiveSize(meta *backup.BackupMeta) uint64

ArchiveSize returns the total size of the backup archive.

func BRInfo

func BRInfo() string

BRInfo returns version information about BR.

func CheckClusterVersion

func CheckClusterVersion(ctx context.Context, client pd.Client) error

CheckClusterVersion check TiKV version.

func CheckGCSafePoint

func CheckGCSafePoint(ctx context.Context, pdClient pd.Client, ts uint64) error

CheckGCSafePoint checks whether the ts is older than GC safepoint. Note: It ignores errors other than exceed GC safepoint.

func ClampInt

func ClampInt(n, min, max int) int

ClampInt restrict a value to a certain interval.

func CompareEndKey

func CompareEndKey(a, b []byte) int

CompareEndKey compared two keys that BOTH represent the EXCLUSIVE ending of some range. An empty end key is the very end, so an empty key is greater than any other keys. Please note that this function is not applicable if any one argument is not an EXCLUSIVE ending of a range.

func EncloseName

func EncloseName(name string) string

EncloseName formats name in sql.

func GetPlacementRules

func GetPlacementRules(pdAddr string, tlsConf *tls.Config) ([]placement.Rule, error)

GetPlacementRules return the current placement rules.

func IsTiFlash

func IsTiFlash(store *metapb.Store) bool

IsTiFlash tests whether the store is based on tiflash engine.

func LoadBackupTables

func LoadBackupTables(meta *backup.BackupMeta) (map[string]*Database, error)

LoadBackupTables loads schemas from BackupMeta.

func LogBRInfo

func LogBRInfo()

LogBRInfo logs version information about BR.

func MakeSafePointID

func MakeSafePointID() string

MakeSafePointID makes a unique safe point ID, for reduce name conflict.

func MaxInt

func MaxInt(x int, xs ...int) int

MaxInt choice biggest integer from its arguments.

func MinInt

func MinInt(x int, xs ...int) int

MinInt choice smallest integer from its arguments.

func MinInt64

func MinInt64(x int64, xs ...int64) int64

MinInt64 choice smallest integer from its arguments.

func NeedAutoID

func NeedAutoID(tblInfo *model.TableInfo) bool

NeedAutoID checks whether the table needs backing up with an autoid.

func ParseKey

func ParseKey(format, key string) ([]byte, error)

ParseKey parse key by given format.

func ResetTS

func ResetTS(pdAddr string, ts uint64, tlsConf *tls.Config) error

ResetTS resets the timestamp of PD to a bigger value.

func SearchPlacementRule

func SearchPlacementRule(tableID int64, placementRules []placement.Rule, role placement.PeerRoleType) *placement.Rule

SearchPlacementRule returns the placement rule matched to the table or nil.

func StartDynamicPProfListener

func StartDynamicPProfListener()

StartDynamicPProfListener starts the listener that will enable pprof when received `startPProfSignal`.

func StartPProfListener

func StartPProfListener(statusAddr string)

StartPProfListener forks a new goroutine listening on specified port and provide pprof info.

func StartServiceSafePointKeeper

func StartServiceSafePointKeeper(
	ctx context.Context,
	pdClient pd.Client,
	sp BRServiceSafePoint,
)

StartServiceSafePointKeeper will run UpdateServiceSafePoint periodicity hence keeping service safepoint won't lose.

func UpdateServiceSafePoint

func UpdateServiceSafePoint(ctx context.Context, pdClient pd.Client, sp BRServiceSafePoint) error

UpdateServiceSafePoint register BackupTS to PD, to lock down BackupTS as safePoint with TTL seconds.

func WithRetry

func WithRetry(
	ctx context.Context,
	retryableFunc RetryableFunc,
	backoffer Backoffer,
) error

WithRetry retries a given operation with a backoff policy.

Returns nil if `retryableFunc` succeeded at least once. Otherwise, returns a multierr containing all errors encountered.

func WrapKey

func WrapKey(key []byte) fmt.Stringer

WrapKey wrap a key as a Stringer that can print proper upper hex format.

func WrapKeys

func WrapKeys(keys [][]byte) zapcore.ArrayMarshaler

WrapKeys wrap keys as an ArrayMarshaler that can print proper upper hex format.

func ZapFile

func ZapFile(file *backup.File) zapcore.Field

ZapFile make the zap fields for a file.

func ZapFiles

func ZapFiles(fs []*backup.File) zapcore.Field

ZapFiles make the zap field for a set of file.

func ZapRegion

func ZapRegion(region *metapb.Region) zapcore.Field

ZapRegion make the zap fields for a region.

func ZapRewriteRule

func ZapRewriteRule(rewriteRule *import_sstpb.RewriteRule) zapcore.Field

ZapRewriteRule make the zap fields for a rewrite rule.

func ZapSSTMeta

func ZapSSTMeta(sstMeta *import_sstpb.SSTMeta) zapcore.Field

ZapSSTMeta make the zap fields for a SST meta.

Types

type BRServiceSafePoint

type BRServiceSafePoint struct {
	ID       string
	TTL      int64
	BackupTS uint64
}

BRServiceSafePoint is metadata of service safe point from a BR 'instance'.

func (BRServiceSafePoint) MarshalLogObject

func (sp BRServiceSafePoint) MarshalLogObject(encoder zapcore.ObjectEncoder) error

MarshalLogObject implements zapcore.ObjectMarshaler.

type Backoffer

type Backoffer interface {
	// NextBackoff returns a duration to wait before retrying again
	NextBackoff(err error) time.Duration
	// Attempt returns the remain attempt times
	Attempt() int
}

Backoffer implements a backoff policy for retrying operations.

type BytesBuffer

type BytesBuffer struct {
	// contains filtered or unexported fields
}

BytesBuffer represents the reuse buffer.

func NewBytesBuffer

func NewBytesBuffer() *BytesBuffer

NewBytesBuffer creates the BytesBuffer.

func (*BytesBuffer) AddBuf

func (b *BytesBuffer) AddBuf()

AddBuf adds buffer to BytesBuffer.

func (*BytesBuffer) AddBytes

func (b *BytesBuffer) AddBytes(bytes []byte) []byte

AddBytes add the bytes into this BytesBuffer.

func (*BytesBuffer) Destroy

func (b *BytesBuffer) Destroy()

Destroy free all buffer.

func (*BytesBuffer) Reset

func (b *BytesBuffer) Reset()

Reset reset the buffer.

func (*BytesBuffer) TotalSize

func (b *BytesBuffer) TotalSize() int64

TotalSize represents the total memory size of this BytesBuffer.

type Database

type Database struct {
	Info   *model.DBInfo
	Tables []*Table
}

Database wraps the schema and tables of a database.

func (*Database) GetTable

func (db *Database) GetTable(name string) *Table

GetTable returns a table of the database by name.

type ProgressPrinter

type ProgressPrinter struct {
	// contains filtered or unexported fields
}

ProgressPrinter prints a progress bar.

func NewProgressPrinter

func NewProgressPrinter(
	name string,
	total int64,
	redirectLog bool,
) *ProgressPrinter

NewProgressPrinter returns a new progress printer.

func StartProgress

func StartProgress(
	ctx context.Context,
	name string,
	total int64,
	redirectLog bool,
	log logFunc,
) *ProgressPrinter

StartProgress starts progress bar.

func (*ProgressPrinter) Close

func (pp *ProgressPrinter) Close()

Close closes the current progress bar.

func (*ProgressPrinter) Inc

func (pp *ProgressPrinter) Inc()

Inc increases the current progress bar.

type RetryableFunc

type RetryableFunc func() error

RetryableFunc presents a retryable operation.

type Table

type Table struct {
	DB              *model.DBInfo
	Info            *model.TableInfo
	Crc64Xor        uint64
	TotalKvs        uint64
	TotalBytes      uint64
	Files           []*backup.File
	TiFlashReplicas int
}

Table wraps the schema and files of a table.

func (*Table) NoChecksum

func (tbl *Table) NoChecksum() bool

NoChecksum checks whether the table has a calculated checksum.

type UndoFunc

type UndoFunc func(context.Context) error

UndoFunc is a 'undo' operation of some undoable command. (e.g. RemoveSchedulers).

var Nop UndoFunc = func(context.Context) error { return nil }

Nop is the 'zero value' of undo func.

type Worker

type Worker struct {
	ID uint64
}

Worker identified by ID.

type WorkerPool

type WorkerPool struct {
	// contains filtered or unexported fields
}

WorkerPool contains a pool of workers.

func NewWorkerPool

func NewWorkerPool(limit uint, name string) *WorkerPool

NewWorkerPool returns a WorkPool.

func (*WorkerPool) Apply

func (pool *WorkerPool) Apply(fn taskFunc)

Apply executes a task.

func (*WorkerPool) ApplyOnErrorGroup

func (pool *WorkerPool) ApplyOnErrorGroup(eg *errgroup.Group, fn func() error)

ApplyOnErrorGroup executes a task in an errorgroup.

func (*WorkerPool) ApplyWithID

func (pool *WorkerPool) ApplyWithID(fn identifiedTaskFunc)

ApplyWithID execute a task and provides it with the worker ID.

func (*WorkerPool) ApplyWithIDInErrorGroup

func (pool *WorkerPool) ApplyWithIDInErrorGroup(eg *errgroup.Group, fn func(id uint64) error)

ApplyWithIDInErrorGroup executes a task in an errorgroup and provides it with the worker ID.

func (*WorkerPool) HasWorker

func (pool *WorkerPool) HasWorker() bool

HasWorker checks if the pool has unallocated workers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL