oplog

package
v1.8.1-0...-da3bd6c Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 26, 2025 License: Apache-2.0 Imports: 35 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrNoCloningNamespace = errors.New("cloning namespace desn't exist")

Functions

func AllOplogRSNames 

func AllOplogRSNames(ctx context.Context, m connect.Client, from, to primitive.Timestamp) ([]string, error)

func DefaultOpFilter 

func DefaultOpFilter(*Record) bool

func FetchSlicersWithActiveLocks 

func FetchSlicersWithActiveLocks(ctx context.Context, conn connect.Client) ([]string, error)

FetchSlicersWithActiveLocks fetches the list of slicers (agents) that are holding active OpLock.

func FormatChunkFilepath 

func FormatChunkFilepath(rs string, first, last primitive.Timestamp, c compress.CompressionType) string

FormatChunkFilepath returns filepath for a oplog chunk. Current format is 20200715155939-0.20200715160029-1.oplog.snappy

!!! should be agreed with oplog.MakeChunkMetaFromFilepath()

func GetAgentsWithACK 

func GetAgentsWithACK(ctx context.Context, conn connect.Client) ([]string, error)

GetAgentsWithACK returns the list of all acknowledged agents.

func GetOplogStartTime 

func GetOplogStartTime(ctx context.Context, m *mongo.Client) (primitive.Timestamp, error)

func HasSingleTimelineToCover 

func HasSingleTimelineToCover(chunks []OplogChunk, from, till uint32) bool

func InitMeta 

func InitMeta(ctx context.Context, conn connect.Client) error

Init add initial PITR document.

func IsOplogSlicing 

func IsOplogSlicing(ctx context.Context, conn connect.Client) (bool, error)

IsOplogSlicing checks if PITR slicing is running. It looks for PITR locks and returns true if there is at least one not stale.

func PITRAddChunk 

func PITRAddChunk(ctx context.Context, m connect.Client, c OplogChunk) error

PITRAddChunk stores PITR chunk metadata

func SetClusterStatus 

func SetClusterStatus(ctx context.Context, conn connect.Client, status Status) error

SetClusterStatus sets cluster status field of PITR Meta doc. It also resets all content of replsets field doc.

func SetErrorRSStatus 

func SetErrorRSStatus(ctx context.Context, conn connect.Client, rs, node, errText string) error

SetErrorRSStatus sets Error status for specified replicaset and includes error descrioption.

func SetHbForPITR 

func SetHbForPITR(ctx context.Context, conn connect.Client) error

func SetPITRNomination 

func SetPITRNomination(ctx context.Context, conn connect.Client, rs string) error

SetPITRNomination adds nomination fragment for specified RS within PITRMeta.

func SetPITRNomineeACK 

func SetPITRNomineeACK(
	ctx context.Context,
	conn connect.Client,
	rs,
	node string,
) error

SetPITRNomineeACK add ack for specific nomination. It is used by nominee, after the nomination is created by cluster leader.

func SetPITRNominees 

func SetPITRNominees(
	ctx context.Context,
	conn connect.Client,
	rs string,
	nodes []string,
) error

SetPITRNominees add nominee(s) for specific RS. It is used by cluster leader within nomination process.

func SetReadyRSStatus 

func SetReadyRSStatus(ctx context.Context, conn connect.Client, rs, node string) error

SetReadyRSStatus sets Ready status for specified replicaset.

Types

type InsuffRangeError 

type InsuffRangeError struct {
	primitive.Timestamp
}

func (InsuffRangeError) Error 

func (e InsuffRangeError) Error() string

type OpFilter 

type OpFilter func(*Record) bool

OpFilter can be used to filter out oplog records by content. Useful for apply only subset of operations depending on conditions

type OplogBackup 

type OplogBackup struct {
	// contains filtered or unexported fields
}

OplogBackup is used for reading the Mongodb oplog

func NewOplogBackup 

func NewOplogBackup(m *mongo.Client) *OplogBackup

NewOplogBackup creates a new Oplog instance

func (*OplogBackup) Cancel 

func (ot *OplogBackup) Cancel()

func (*OplogBackup) IsSufficient 

func (ot *OplogBackup) IsSufficient(from primitive.Timestamp) (bool, error)

IsSufficient check is oplog is sufficient back from the given date

func (*OplogBackup) SetTailingSpan 

func (ot *OplogBackup) SetTailingSpan(start, end primitive.Timestamp)

SetTailingSpan sets oplog tailing window

func (*OplogBackup) WriteTo 

func (ot *OplogBackup) WriteTo(w io.Writer) (int64, error)

WriteTo writes an oplog slice between start and end timestamps into the given io.Writer

To be sure we have read ALL records up to the specified cluster time. Specifically, to be sure that no operations from the past gonna came after we finished the slicing, we have to tail until some record with ts > endTS. And it might be a noop.

type OplogChunk 

type OplogChunk struct {
	RS          string                   `bson:"rs"`
	FName       string                   `bson:"fname"`
	Compression compress.CompressionType `bson:"compression"`
	StartTS     primitive.Timestamp      `bson:"start_ts"`
	EndTS       primitive.Timestamp      `bson:"end_ts"`
	Size        int64                    `bson:"size"`
}

OplogChunk is index metadata for the oplog chunks

func MakeChunkMetaFromFilepath 

func MakeChunkMetaFromFilepath(f string) *OplogChunk

MakeChunkMetaFromFilepath parses given file name and returns PITRChunk metadata it returns nil if file wasn't parse successfully (e.g. wrong format) current format is 20200715155939-0.20200715160029-1.oplog.snappy

!!! should be agreed with oplog.FormatChunkFilepath()

func PITRFirstChunkMeta 

func PITRFirstChunkMeta(ctx context.Context, m connect.Client, rs string) (*OplogChunk, error)

PITRFirstChunkMeta returns the oldest PITR chunk for the given Replset

func PITRGetChunkStarts 

func PITRGetChunkStarts(
	ctx context.Context,
	m connect.Client,
	rs string,
	ts primitive.Timestamp,
) (*OplogChunk, error)

PITRGetChunkStarts returns a pitr slice chunk that belongs to the given replica set and start from the given timestamp

func PITRGetChunksSlice 

func PITRGetChunksSlice(
	ctx context.Context,
	m connect.Client,
	rs string,
	from, to primitive.Timestamp,
) ([]OplogChunk, error)

PITRGetChunksSlice returns slice of PITR oplog chunks which Start TS lies in a given time frame. Returns all chunks if `to` is 0.

func PITRGetChunksSliceUntil 

func PITRGetChunksSliceUntil(
	ctx context.Context,
	m connect.Client,
	rs string,
	t primitive.Timestamp,
) ([]OplogChunk, error)

PITRGetChunksSliceUntil returns slice of PITR oplog chunks that starts up until timestamp (exclusively)

func PITRLastChunkMeta 

func PITRLastChunkMeta(ctx context.Context, m connect.Client, rs string) (*OplogChunk, error)

PITRLastChunkMeta returns the most recent PITR chunk for the given Replset

type OplogRestore 

type OplogRestore struct {
	// contains filtered or unexported fields
}

OplogRestore is the oplog applyer

func NewOplogRestore 

func NewOplogRestore(
	m *mongo.Client,
	ic *idx.IndexCatalog,
	sv *version.MongoVersion,
	unsafe,
	preserveUUID bool,
	ctxn chan phys.RestoreTxn,
	txnErr chan error,
) (*OplogRestore, error)

NewOplogRestore creates an object for an oplog applying

func (*OplogRestore) Apply 

func (o *OplogRestore) Apply(src io.ReadCloser) (primitive.Timestamp, error)

Apply applys an oplog from a given source

func (*OplogRestore) HandleUncommittedTxn 

func (o *OplogRestore) HandleUncommittedTxn(
	commits map[string]primitive.Timestamp,
) (partial, uncommitted []Txn, err error)

func (*OplogRestore) LastOpTS 

func (o *OplogRestore) LastOpTS() uint32

func (*OplogRestore) SetCloneNS 

func (o *OplogRestore) SetCloneNS(ctx context.Context, ns snapshot.CloneNS) error

SetCloneNS sets all needed data for cloning namespace: collection names and target namespace UUID

func (*OplogRestore) SetIncludeNS 

func (o *OplogRestore) SetIncludeNS(nss []string)

func (*OplogRestore) SetOpFilter 

func (o *OplogRestore) SetOpFilter(f OpFilter)

SetOpFilter allows to restrict skip ops by specific conditions

func (*OplogRestore) SetTimeframe 

func (o *OplogRestore) SetTimeframe(start, end primitive.Timestamp)

SetTimeframe sets boundaries for the replayed operations. All operations that happened before `start` and after `end` are going to be discarded. Zero `end` (primitive.Timestamp{T:0}) means all chunks will be replayed utill the end (no tail trim).

func (*OplogRestore) TxnLeftovers 

func (o *OplogRestore) TxnLeftovers() (uncommitted map[string]Txn, lastCommits []phys.RestoreTxn)

type PITRMeta 

type PITRMeta struct {
	StartTS    int64               `bson:"start_ts" json:"start_ts"`
	Hb         primitive.Timestamp `bson:"hb" json:"hb"`
	Status     Status              `bson:"status" json:"status"`
	Nomination []PITRNomination    `bson:"n" json:"n"`
	Replsets   []PITRReplset       `bson:"replsets" json:"replsets"`
}

PITRMeta contains all operational data about PITR execution process.

func GetMeta 

func GetMeta(
	ctx context.Context,
	conn connect.Client,
) (*PITRMeta, error)

GetMeta fetches PITR meta doc from pbmPITR collection.

type PITRNomination 

type PITRNomination struct {
	RS    string   `bson:"rs" json:"rs"`
	Nodes []string `bson:"n" json:"n"`
	Ack   string   `bson:"ack" json:"ack"`
}

PITRNomination is used to choose (nominate and elect) member(s) which will perform PITR process within a replica set(s).

func GetPITRNominees 

func GetPITRNominees(
	ctx context.Context,
	conn connect.Client,
	rs string,
) (*PITRNomination, error)

GetPITRNominees fetches nomination fragment for specified RS from pmbPITR document. If document is not found, or document fragment for specific RS is not found, error ErrNotFound is returned.

type PITRReplset 

type PITRReplset struct {
	Name   string `bson:"name" json:"name"`
	Node   string `bson:"node" json:"node"`
	Status Status `bson:"status" json:"status"`
	Error  string `bson:"error,omitempty" json:"error,omitempty"`
}

PITRReplset holds status for each replica set. Each replicaset tries to reach cluster status set by Cluser Leader.

func GetReplSetsWithStatus 

func GetReplSetsWithStatus(ctx context.Context, conn connect.Client, status Status) ([]PITRReplset, error)

GetReplSetsWithStatus fetches all replica sets which reported status specified with parameter.

type Record 

type Record = db.Oplog

type Status 

type Status string

Status is a PITR status. It is used within pbmPITR collection to sync operation between cluster leader and agents. 

const (
	StatusReady    Status = "ready"
	StatusRunning  Status = "running"
	StatusReconfig Status = "reconfig"
	StatusError    Status = "error"
	StatusUnset    Status = ""
)

func GetClusterStatus 

func GetClusterStatus(ctx context.Context, conn connect.Client) (Status, error)

type Timeline 

type Timeline struct {
	Start uint32 `json:"start"`
	End   uint32 `json:"end"`
	Size  int64  `json:"-"`
}

func MergeTimelines 

func MergeTimelines(tlns ...[]Timeline) []Timeline

MergeTimelines merges overlapping sets on timelines it presumes timelines are sorted and don't start from 0

func PITRGetValidTimelines 

func PITRGetValidTimelines(
	ctx context.Context,
	m connect.Client,
	rs string,
	until primitive.Timestamp,
) ([]Timeline, error)

PITRGetValidTimelines returns time ranges valid for PITR restore for the given replicaset. We don't check for any "restore intrusions" or other integrity issues since it's guaranteed be the slicer that any saved chunk already belongs to some valid timeline, the slice wouldn't be done otherwise. `flist` is a cache of chunk sizes.

func PITRGetValidTimelinesBetween 

func PITRGetValidTimelinesBetween(
	ctx context.Context,
	m connect.Client,
	rs string,
	from primitive.Timestamp,
	until primitive.Timestamp,
) ([]Timeline, error)

func PITRTimelines 

func PITRTimelines(ctx context.Context, m connect.Client) ([]Timeline, error)

PITRTimelines returns cluster-wide time ranges valid for PITR restore

func PITRTimelinesBetween 

func PITRTimelinesBetween(ctx context.Context, m connect.Client, from, until primitive.Timestamp) ([]Timeline, error)

func (Timeline) String 

func (t Timeline) String() string

type Txn 

type Txn struct {
	Oplog []db.Oplog
	// contains filtered or unexported fields
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.