clone

Documentation

Overview

Package clone is the engine. It knows nothing engine-specific: it asks each driver for a Plan, then runs every stream of every database through one global, weighted pool.

Scheduling:

• The pool (Options.Parallel slots) is shared by all databases, so a slot freed by a small database is picked up immediately by a big one instead of sitting idle.
• A stream holds Weight slots — mongodump with 4 parallel collections costs 4.
• Streams are dispatched biggest first across all databases (longest-processing-time first), which keeps the tail of the run short.
• Options.Workers caps the streams of one database, so one huge database cannot take every connection to staging.

Index

• type Job
  • func (j Job) Label() string
  • func (j Job) String() string
• type Options
• type Reporter
• type Result
  • func Run(ctx context.Context, jobs []Job, reps []Reporter, opt Options) []Result

Types

type Job

type Job struct {
	Driver driver.Driver
	DB     string
	Only   []string // tables / collections to clone; empty = the whole database
}

func (Job) Label

func (j Job) Label() string

Label is String plus the subset, for display.

func (Job) String

func (j Job) String() string

type Options

type Options struct {
	Parallel int  // global pool size: concurrent streams across all databases
	Workers  int  // max concurrent streams for one database
	Fresh    bool // drop each target database cloned whole before cloning it
	Source   driver.Endpoint
	Target   driver.Endpoint
	LogDir   string
}

type Reporter

type Reporter interface {
	Start(estimate int64)
	Add(n int64)
	Stage(s string)
	Active(streams int)
	Done()
	Fail()
}

Reporter receives progress for one job. The UI implements it; the engine only calls it.

type Result

type Result struct {
	Job      Job
	Err      error
	Log      string
	Warnings []string
}

func Run

func Run(ctx context.Context, jobs []Job, reps []Reporter, opt Options) []Result