wrangler

package
v0.11.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 9, 2021 License: Apache-2.0 Imports: 63 Imported by: 16

Documentation

Overview

Package wrangler contains the Wrangler object to manage complex topology actions.

Index

Constants

View Source 
const (
	// ChangeTabletTypeActionName is the name of the action to change a tablet type
	// (can be used to find such an action by name)
	ChangeTabletTypeActionName = "ChangeTabletTypeAction"

	// TabletTagActionName is the name of the Tag action
	TabletTagActionName = "TabletTagAction"

	// StartReplicationActionName is the name of the start replication action
	StartReplicationActionName = "StartReplicationAction"

	// VReplicationActionName is the name of the action to execute VReplication commands
	VReplicationActionName = "VReplicationAction"
)
View Source 
const (
	MoveTablesWorkflow = VReplicationWorkflowType(iota)
	ReshardWorkflow
	MigrateWorkflow
)

VReplicationWorkflowType enums

View Source 
const (
	WorkflowStateNotCreated     = "Not Created"
	WorkflowStateNotSwitched    = "Reads Not Switched. Writes Not Switched"
	WorkflowStateReadsSwitched  = "All Reads Switched. Writes Not Switched"
	WorkflowStateWritesSwitched = "Reads Not Switched. Writes Switched"
	WorkflowStateAllSwitched    = "All Reads Switched. Writes Switched"
)

Workflow state display strings

View Source 
const (
	ErrWorkflowNotFullySwitched  = "cannot complete workflow because you have not yet switched all read and write traffic"
	ErrWorkflowPartiallySwitched = "cannot cancel workflow because you have already switched some or all read and write traffic"
)

Workflow errors

View Source 
const (
	// DefaultFilteredReplicationWaitTime is the default value for argument filteredReplicationWaitTime.
	DefaultFilteredReplicationWaitTime = 30 * time.Second
)
View Source 
const (
	// DefaultWaitForFilteredReplicationMaxDelay is the default maximum delay value used in WaitForFilteredReplication.
	DefaultWaitForFilteredReplicationMaxDelay = 30 * time.Second
)
View Source 
const (
	// DefaultWaitReplicasTimeout is the default value for waitReplicasTimeout, which is used when calling method CopySchemaShardFromShard.
	DefaultWaitReplicasTimeout = 10 * time.Second
)

Variables

View Source 
var (
	// DefaultActionTimeout is a good default for interactive
	// remote actions. We usually take a lock then do an action,
	// lock actions use RemoteOperationTimeout,
	// so basing this to be greater than RemoteOperationTimeout is good.
	// Use this as the default value for Context that need a deadline.
	DefaultActionTimeout = *topo.RemoteOperationTimeout * 4
)

Functions

func RecordChangeTabletTypeAction 

func RecordChangeTabletTypeAction(cleaner *Cleaner, tabletAlias *topodatapb.TabletAlias, from topodatapb.TabletType, to topodatapb.TabletType)

RecordChangeTabletTypeAction records a new ChangeTabletTypeAction into the specified Cleaner

func RecordStartReplicationAction 

func RecordStartReplicationAction(cleaner *Cleaner, tablet *topodatapb.Tablet)

RecordStartReplicationAction records a new action to restart binlog replication on a server into the specified Cleaner

func RecordVReplicationAction 

func RecordVReplicationAction(cleaner *Cleaner, tablet *topodatapb.Tablet, query string)

RecordVReplicationAction records an action to restart binlog replication on a server into the specified Cleaner

func ResetDebugVarsGetVersion 

func ResetDebugVarsGetVersion()

ResetDebugVarsGetVersion is used by tests to reset the getVersionFromTablet variable to the default one. That way we can run the unit tests in testlib/ even when another implementation of getVersionFromTablet is used.

Types

type Cleaner 

type Cleaner struct {
	// contains filtered or unexported fields
}

Cleaner remembers a list of cleanup steps to perform. Just record action cleanup steps, and execute them at the end in reverse order, with various guarantees.

func (*Cleaner) CleanUp 

func (cleaner *Cleaner) CleanUp(wr *Wrangler) error

CleanUp will run the recorded actions. If an action on a target fails, it will not run the next action on the same target. We return the aggregate errors for all cleanups. CleanUp uses its own context, with a timeout of 5 minutes, so that clean up action will run even if the original context times out. TODO(alainjobart) Actions should run concurrently on a per target basis. They are then serialized on each target.

func (*Cleaner) Record 

func (cleaner *Cleaner) Record(name, target string, action CleanerFunction)

Record will add a cleaning action to the list

type CleanerFunction 

type CleanerFunction func(context.Context, *Wrangler) error

CleanerFunction is the interface that clean-up actions need to implement

type CopyProgress added in v0.9.0 

type CopyProgress map[string]*TableCopyProgress

CopyProgress stores the TableCopyProgress for all tables still being copied

type DiffMismatch added in v0.11.0 

type DiffMismatch struct {
	Source *RowDiff
	Target *RowDiff
}

DiffMismatch is a sample of row diffs between source and target.

type DiffReport 

type DiffReport struct {
	ProcessedRows         int
	MatchingRows          int
	MismatchedRows        int
	ExtraRowsSource       int
	ExtraRowsSourceSample []*RowDiff
	ExtraRowsTarget       int
	ExtraRowsTargetSample []*RowDiff
	MismatchedRowsSample  []*DiffMismatch
	TableName             string
}

DiffReport is the summary of differences for one table.

type LogRecorder 

type LogRecorder struct {
	// contains filtered or unexported fields
}

LogRecorder is used to collect logs for a specific purpose. Not thread-safe since it is expected to be generated in repeatable sequence

func NewLogRecorder 

func NewLogRecorder() *LogRecorder

NewLogRecorder creates a new instance of LogRecorder

func (*LogRecorder) GetLogs 

func (lr *LogRecorder) GetLogs() []string

GetLogs returns all recorded logs in sequence

func (*LogRecorder) Log 

func (lr *LogRecorder) Log(log string)

Log records a new log message

func (*LogRecorder) LogSlice 

func (lr *LogRecorder) LogSlice(logs []string)

LogSlice sorts a given slice using natural sort, so that the result is predictable. Useful when logging arrays or maps where order of objects can vary

type ReplicationLocation added in v0.8.0 

type ReplicationLocation struct {
	Keyspace string
	Shards   []string
}

ReplicationLocation represents a location that data is either replicating from, or replicating into.

type ReplicationStatus added in v0.8.0 

type ReplicationStatus struct {
	// Shard represents the relevant shard name.
	Shard string
	// Tablet is the tablet alias that the ReplicationStatus came from.
	Tablet string
	// ID represents the id column from the _vt.vreplication table.
	ID int64
	// Bls represents the BinlogSource.
	Bls *binlogdatapb.BinlogSource
	// Pos represents the pos column from the _vt.vreplication table.
	Pos string
	// StopPos represents the stop_pos column from the _vt.vreplication table.
	StopPos string
	// State represents the state column from the _vt.vreplication table.
	State string
	// DbName represents the db_name column from the _vt.vreplication table.
	DBName string
	// TransactionTimestamp represents the transaction_timestamp column from the _vt.vreplication table.
	TransactionTimestamp int64
	// TimeUpdated represents the time_updated column from the _vt.vreplication table.
	TimeUpdated int64
	// Message represents the message column from the _vt.vreplication table.
	Message string

	// CopyState represents the rows from the _vt.copy_state table.
	CopyState []copyState
}

ReplicationStatus includes data from the _vt.vreplication table, along with other useful relevant data.

type ReplicationStatusResult added in v0.8.0 

type ReplicationStatusResult struct {
	// Workflow represents the name of the workflow relevant to the related replication statuses.
	Workflow string
	// SourceLocation represents the keyspace and shards that we are vreplicating from.
	SourceLocation ReplicationLocation
	// TargetLocation represents the keyspace and shards that we are vreplicating into.
	TargetLocation ReplicationLocation
	// MaxVReplicationLag represents the maximum vreplication lag seen across all shards.
	MaxVReplicationLag int64

	// Statuses is a map of <shard>/<master tablet alias> : ShardReplicationStatus (for the given shard).
	ShardStatuses map[string]*ShardReplicationStatus
}

ReplicationStatusResult represents the result of trying to get the replication status for a given workflow.

type RowDiff added in v0.11.0 

type RowDiff struct {
	Row   map[string]sqltypes.Value
	Query string
}

RowDiff is a row that didn't match as part of the comparison.

type ShardReplicationStatus added in v0.8.0 

type ShardReplicationStatus struct {
	// MasterReplicationStatuses represents all of the replication statuses for the master tablets in the given shard.
	MasterReplicationStatuses []*ReplicationStatus
	// TabletControls represents the tablet controls for the tablets in the shard.
	TabletControls []*topodatapb.Shard_TabletControl
	// MasterIsServing indicates whether the master tablet of the given shard is currently serving write traffic.
	MasterIsServing bool
}

ShardReplicationStatus holds relevant vreplication related info for the given shard.

type TableCopyProgress added in v0.9.0 

type TableCopyProgress struct {
	TargetRowCount, TargetTableSize int64
	SourceRowCount, SourceTableSize int64
}

TableCopyProgress stores the row counts and disk sizes of the source and target tables

type VReplicationWorkflow added in v0.9.0 

type VReplicationWorkflow struct {
	// contains filtered or unexported fields
}

VReplicationWorkflow stores various internal objects for a workflow

func (*VReplicationWorkflow) CachedState added in v0.9.0 

func (vrw *VReplicationWorkflow) CachedState() string

CachedState returns a human readable workflow state at the time the workflow was created

func (*VReplicationWorkflow) Cancel added in v0.9.0 

func (vrw *VReplicationWorkflow) Cancel() error

Cancel deletes all artifacts from a workflow which has not yet been switched

func (*VReplicationWorkflow) Complete added in v0.9.0 

func (vrw *VReplicationWorkflow) Complete() (*[]string, error)

Complete cleans up a successful workflow

func (*VReplicationWorkflow) Create added in v0.9.0 

func (vrw *VReplicationWorkflow) Create(ctx context.Context) error

Create initiates a workflow

func (*VReplicationWorkflow) CurrentState added in v0.9.0 

func (vrw *VReplicationWorkflow) CurrentState() string

CurrentState reloads and returns a human readable workflow state

func (*VReplicationWorkflow) Exists added in v0.9.0 

func (vrw *VReplicationWorkflow) Exists() bool

Exists checks if the workflow has already been initiated

func (*VReplicationWorkflow) GetCopyProgress added in v0.9.0 

func (vrw *VReplicationWorkflow) GetCopyProgress() (*CopyProgress, error)

GetCopyProgress returns the progress of all tables being copied in the workflow

func (*VReplicationWorkflow) GetStreamCount added in v0.9.0 

func (vrw *VReplicationWorkflow) GetStreamCount() (int64, int64, []*WorkflowError, error)

GetStreamCount returns a count of total and running streams and any stream errors

func (*VReplicationWorkflow) IsCopyInProgress added in v0.9.0 

func (vrw *VReplicationWorkflow) IsCopyInProgress() (bool, error)

IsCopyInProgress returns true if any table remains to be copied

func (*VReplicationWorkflow) ReverseTraffic added in v0.9.0 

func (vrw *VReplicationWorkflow) ReverseTraffic() (*[]string, error)

ReverseTraffic switches traffic backwards for tablet_types passed

func (*VReplicationWorkflow) String added in v0.9.0 

func (vrw *VReplicationWorkflow) String() string

func (*VReplicationWorkflow) SwitchTraffic added in v0.9.0 

func (vrw *VReplicationWorkflow) SwitchTraffic(direction workflow.TrafficSwitchDirection) (*[]string, error)

SwitchTraffic switches traffic in the direction passed for specified tablet_types

type VReplicationWorkflowParams added in v0.9.0 

type VReplicationWorkflowParams struct {
	WorkflowType                      VReplicationWorkflowType
	Workflow, TargetKeyspace          string
	Cells, TabletTypes, ExcludeTables string
	EnableReverseReplication, DryRun  bool
	KeepData                          bool
	Timeout                           time.Duration
	Direction                         workflow.TrafficSwitchDirection

	// MoveTables specific
	SourceKeyspace, Tables  string
	AllTables, RenameTables bool

	// Reshard specific
	SourceShards, TargetShards []string
	SkipSchemaCopy             bool
	AutoStart, StopAfterCopy   bool

	// Migrate specific
	ExternalCluster string
}

VReplicationWorkflowParams stores args and options passed to a VReplicationWorkflow command

type VReplicationWorkflowType added in v0.9.0 

type VReplicationWorkflowType int

VReplicationWorkflowType specifies whether workflow is MoveTables or Reshard

type WorkflowError added in v0.9.0 

type WorkflowError struct {
	Tablet      string
	ID          int64
	Description string
}

WorkflowError has per stream errors if present in a workflow

func NewWorkflowError added in v0.9.0 

func NewWorkflowError(tablet string, id int64, description string) *WorkflowError

NewWorkflowError returns a new WorkflowError object

type Wrangler 

type Wrangler struct {
	// contains filtered or unexported fields
}

Wrangler manages complex actions on the topology, like reparents, backups, resharding, ...

Multiple go routines can use the same Wrangler at the same time, provided they want to share the same logger / topo server / lock timeout.

func New 

func New(logger logutil.Logger, ts *topo.Server, tmc tmclient.TabletManagerClient) *Wrangler

New creates a new Wrangler object.

func (*Wrangler) CancelResharding 

func (wr *Wrangler) CancelResharding(ctx context.Context, keyspace, shard string) (err error)

CancelResharding cancels any resharding in progress on the specified keyspace/shard. This works for horizontal as well as vertical resharding.

func (*Wrangler) ChangeTabletType 

func (wr *Wrangler) ChangeTabletType(ctx context.Context, tabletAlias *topodatapb.TabletAlias, tabletType topodatapb.TabletType) error

ChangeTabletType changes the type of tablet and recomputes all necessary derived paths in the serving graph, if necessary.

Note we don't update the master record in the Shard here, as we can't ChangeType from and out of master anyway.

func (*Wrangler) CopySchemaShard 

func (wr *Wrangler) CopySchemaShard(ctx context.Context, sourceTabletAlias *topodatapb.TabletAlias, tables, excludeTables []string, includeViews bool, destKeyspace, destShard string, waitReplicasTimeout time.Duration, skipVerify bool) error

CopySchemaShard copies the schema from a source tablet to the specified shard. The schema is applied directly on the master of the destination shard, and is propagated to the replicas through binlogs.

func (*Wrangler) CopySchemaShardFromShard 

func (wr *Wrangler) CopySchemaShardFromShard(ctx context.Context, tables, excludeTables []string, includeViews bool, sourceKeyspace, sourceShard, destKeyspace, destShard string, waitReplicasTimeout time.Duration, skipVerify bool) error

CopySchemaShardFromShard copies the schema from a source shard to the specified destination shard. For both source and destination it picks the master tablet. See also CopySchemaShard.

func (*Wrangler) CreateLookupVindex 

func (wr *Wrangler) CreateLookupVindex(ctx context.Context, keyspace string, specs *vschemapb.Keyspace, cell, tabletTypes string) error

CreateLookupVindex creates a lookup vindex and sets up the backfill.

func (*Wrangler) DeleteKeyspace 

func (wr *Wrangler) DeleteKeyspace(ctx context.Context, keyspace string, recursive bool) error

DeleteKeyspace will do all the necessary changes in the topology server to entirely remove a keyspace.

func (*Wrangler) DeleteShard 

func (wr *Wrangler) DeleteShard(ctx context.Context, keyspace, shard string, recursive, evenIfServing bool) error

DeleteShard will do all the necessary changes in the topology server to entirely remove a shard.

func (*Wrangler) DeleteTablet 

func (wr *Wrangler) DeleteTablet(ctx context.Context, tabletAlias *topodatapb.TabletAlias, allowMaster bool) (err error)

DeleteTablet removes a tablet from a shard. - if allowMaster is set, we can Delete a master tablet (and clear its record from the Shard record if it was the master).

func (*Wrangler) DropSources 

func (wr *Wrangler) DropSources(ctx context.Context, targetKeyspace, workflowName string, removalType workflow.TableRemovalType, keepData, force, dryRun bool) (*[]string, error)

DropSources cleans up source tables, shards and blacklisted tables after a MoveTables/Reshard is completed

func (*Wrangler) DropTargets added in v0.9.0 

func (wr *Wrangler) DropTargets(ctx context.Context, targetKeyspace, workflow string, keepData, dryRun bool) (*[]string, error)

DropTargets cleans up target tables, shards and blacklisted tables if a MoveTables/Reshard is cancelled

func (*Wrangler) EmergencyReparentShard 

func (wr *Wrangler) EmergencyReparentShard(ctx context.Context, keyspace, shard string, masterElectTabletAlias *topodatapb.TabletAlias, waitReplicasTimeout time.Duration, ignoredTablets sets.String) (err error)

EmergencyReparentShard will make the provided tablet the master for the shard, when the old master is completely unreachable.

func (*Wrangler) ExecuteFetchAsApp 

func (wr *Wrangler) ExecuteFetchAsApp(ctx context.Context, tabletAlias *topodatapb.TabletAlias, usePool bool, query string, maxRows int) (*querypb.QueryResult, error)

ExecuteFetchAsApp executes a query remotely using the App pool

func (*Wrangler) ExecuteFetchAsDba 

func (wr *Wrangler) ExecuteFetchAsDba(ctx context.Context, tabletAlias *topodatapb.TabletAlias, query string, maxRows int, disableBinlogs bool, reloadSchema bool) (*querypb.QueryResult, error)

ExecuteFetchAsDba executes a query remotely using the DBA pool

func (*Wrangler) ExecuteHook 

func (wr *Wrangler) ExecuteHook(ctx context.Context, tabletAlias *topodatapb.TabletAlias, hook *hk.Hook) (hookResult *hk.HookResult, err error)

ExecuteHook will run the hook on the tablet

func (*Wrangler) ExecuteTabletHook 

func (wr *Wrangler) ExecuteTabletHook(ctx context.Context, tablet *topodatapb.Tablet, hook *hk.Hook) (hookResult *hk.HookResult, err error)

ExecuteTabletHook will run the hook on the provided tablet.

func (*Wrangler) ExternalizeVindex 

func (wr *Wrangler) ExternalizeVindex(ctx context.Context, qualifiedVindexName string) error

ExternalizeVindex externalizes a lookup vindex that's finished backfilling or has caught up.

func (*Wrangler) GenericVExec added in v0.8.0 

func (wr *Wrangler) GenericVExec(ctx context.Context, tabletAlias *topodatapb.TabletAlias, query, workflow, keyspace string) (*querypb.QueryResult, error)

GenericVExec executes a query remotely using the DBA pool

func (*Wrangler) GetPermissions 

func (wr *Wrangler) GetPermissions(ctx context.Context, tabletAlias *topodatapb.TabletAlias) (*tabletmanagerdatapb.Permissions, error)

GetPermissions returns the permissions set on a remote tablet

func (*Wrangler) GetSchema 

func (wr *Wrangler) GetSchema(ctx context.Context, tabletAlias *topodatapb.TabletAlias, tables, excludeTables []string, includeViews bool) (*tabletmanagerdatapb.SchemaDefinition, error)

GetSchema uses an RPC to get the schema from a remote tablet

func (*Wrangler) GetVersion 

func (wr *Wrangler) GetVersion(ctx context.Context, tabletAlias *topodatapb.TabletAlias) (string, error)

GetVersion returns the version string from a tablet

func (*Wrangler) InitShardMaster 

func (wr *Wrangler) InitShardMaster(ctx context.Context, keyspace, shard string, masterElectTabletAlias *topodatapb.TabletAlias, force bool, waitReplicasTimeout time.Duration) (err error)

InitShardMaster will make the provided tablet the master for the shard.

func (*Wrangler) InitTablet 

func (wr *Wrangler) InitTablet(ctx context.Context, tablet *topodatapb.Tablet, allowMasterOverride, createShardAndKeyspace, allowUpdate bool) error

InitTablet creates or updates a tablet. If no parent is specified in the tablet, and the tablet has a replica type, we will find the appropriate parent. If createShardAndKeyspace is true and the parent keyspace or shard don't exist, they will be created. If allowUpdate is true, and a tablet with the same ID exists, just update it. If a tablet is created as master, and there is already a different master in the shard, allowMasterOverride must be set.

func (*Wrangler) ListActiveWorkflows added in v0.9.0 

func (wr *Wrangler) ListActiveWorkflows(ctx context.Context, keyspace string) ([]string, error)

ListActiveWorkflows will return a list of all active workflows for the given keyspace.

func (*Wrangler) ListAllWorkflows added in v0.8.0 

func (wr *Wrangler) ListAllWorkflows(ctx context.Context, keyspace string, active bool) ([]string, error)

ListAllWorkflows will return a list of all workflows (Running and Stopped) for the given keyspace.

func (*Wrangler) Logger 

func (wr *Wrangler) Logger() logutil.Logger

Logger returns the logger associated with this wrangler.

func (*Wrangler) Materialize 

func (wr *Wrangler) Materialize(ctx context.Context, ms *vtctldatapb.MaterializeSettings) error

Materialize performs the steps needed to materialize a list of tables based on the materialization specs.

func (*Wrangler) MigrateServedFrom 

func (wr *Wrangler) MigrateServedFrom(ctx context.Context, keyspace, shard string, servedType topodatapb.TabletType, cells []string, reverse bool, filteredReplicationWaitTime time.Duration) (err error)

MigrateServedFrom is used during vertical splits to migrate a served type from a keyspace to another.

func (*Wrangler) MigrateServedTypes 

func (wr *Wrangler) MigrateServedTypes(ctx context.Context, keyspace, shard string, cells []string, servedType topodatapb.TabletType, reverse, skipReFreshState bool, filteredReplicationWaitTime time.Duration, reverseReplication bool) (err error)

MigrateServedTypes is used during horizontal splits to migrate a served type from a list of shards to another.

func (*Wrangler) MountExternalVitessCluster added in v0.10.0 

func (wr *Wrangler) MountExternalVitessCluster(ctx context.Context, clusterName, topoType, topoServer, topoRoot string) error

MountExternalVitessCluster adds a topo record for cluster with specified parameters so that it is available to a Migrate command

func (*Wrangler) MoveTables 

func (wr *Wrangler) MoveTables(ctx context.Context, workflow, sourceKeyspace, targetKeyspace, tableSpecs,
	cell, tabletTypes string, allTables bool, excludeTables string, autoStart, stopAfterCopy bool,
	externalCluster string) error

MoveTables initiates moving table(s) over to another keyspace

func (*Wrangler) NewVReplicationWorkflow added in v0.9.0 

func (wr *Wrangler) NewVReplicationWorkflow(ctx context.Context, workflowType VReplicationWorkflowType,
	params *VReplicationWorkflowParams) (*VReplicationWorkflow, error)

NewVReplicationWorkflow sets up a MoveTables or Reshard workflow based on options provided, deduces the state of the workflow from the persistent state stored in the vreplication table and the topo

func (*Wrangler) PlannedReparentShard 

func (wr *Wrangler) PlannedReparentShard(ctx context.Context, keyspace, shard string, masterElectTabletAlias, avoidMasterAlias *topodatapb.TabletAlias, waitReplicasTimeout time.Duration) (err error)

PlannedReparentShard will make the provided tablet the master for the shard, when both the current and new master are reachable and in good shape.

func (*Wrangler) PreflightSchema 

func (wr *Wrangler) PreflightSchema(ctx context.Context, tabletAlias *topodatapb.TabletAlias, changes []string) ([]*tabletmanagerdatapb.SchemaChangeResult, error)

PreflightSchema will try a schema change on the remote tablet.

func (*Wrangler) QueryResultForRowsAffected added in v0.8.0 

func (wr *Wrangler) QueryResultForRowsAffected(results map[*topo.TabletInfo]*sqltypes.Result) *sqltypes.Result

QueryResultForRowsAffected aggregates results into row-type results (fields + values)

func (*Wrangler) QueryResultForTabletResults added in v0.8.0 

func (wr *Wrangler) QueryResultForTabletResults(results map[*topo.TabletInfo]*sqltypes.Result) *sqltypes.Result

QueryResultForTabletResults aggregates given results into a "rows-affected" type result (no row data)

func (*Wrangler) RebuildKeyspaceGraph 

func (wr *Wrangler) RebuildKeyspaceGraph(ctx context.Context, keyspace string, cells []string, allowPartial bool) error

RebuildKeyspaceGraph rebuilds the serving graph data while locking out other changes.

func (*Wrangler) RefreshTabletState 

func (wr *Wrangler) RefreshTabletState(ctx context.Context, tabletAlias *topodatapb.TabletAlias) error

RefreshTabletState refreshes tablet state

func (*Wrangler) RefreshTabletsByShard 

func (wr *Wrangler) RefreshTabletsByShard(ctx context.Context, si *topo.ShardInfo, cells []string) error

RefreshTabletsByShard calls RefreshState on all the tablets in a given shard.

func (*Wrangler) ReloadSchema 

func (wr *Wrangler) ReloadSchema(ctx context.Context, tabletAlias *topodatapb.TabletAlias) error

ReloadSchema forces the remote tablet to reload its schema.

func (*Wrangler) ReloadSchemaKeyspace 

func (wr *Wrangler) ReloadSchemaKeyspace(ctx context.Context, keyspace string, concurrency *sync2.Semaphore, includeMaster bool) error

ReloadSchemaKeyspace reloads the schema in all shards in a keyspace. The concurrency is shared across all shards (only that many tablets will be reloaded at once).

func (*Wrangler) ReloadSchemaShard 

func (wr *Wrangler) ReloadSchemaShard(ctx context.Context, keyspace, shard, replicationPos string, concurrency *sync2.Semaphore, includeMaster bool)

ReloadSchemaShard reloads the schema for all replica tablets in a shard, after they reach a given replication position (empty pos means immediate). In general, we don't always expect all replicas to be ready to reload, and the periodic schema reload makes them self-healing anyway. So we do this on a best-effort basis, and log warnings for any tablets that fail to reload within the context deadline.

func (*Wrangler) RemoveKeyspaceCell 

func (wr *Wrangler) RemoveKeyspaceCell(ctx context.Context, keyspace, cell string, force, recursive bool) error

RemoveKeyspaceCell will remove a cell from the Cells list in all shards of a keyspace (by calling RemoveShardCell on every shard). It will also remove the SrvKeyspace for that keyspace/cell.

func (*Wrangler) RemoveShardCell 

func (wr *Wrangler) RemoveShardCell(ctx context.Context, keyspace, shard, cell string, force, recursive bool) error

RemoveShardCell will remove a cell from the Cells list in a shard.

It will first check the shard has no tablets there. If 'force' is specified, it will remove the cell even when the tablet map cannot be retrieved. This is intended to be used when a cell is completely down and its topology server cannot even be reached.

If 'recursive' is specified, it will delete any tablets in the cell/shard, with the assumption that the tablet processes have already been terminated.

func (*Wrangler) ReparentTablet 

func (wr *Wrangler) ReparentTablet(ctx context.Context, tabletAlias *topodatapb.TabletAlias) error

ReparentTablet tells a tablet to reparent this tablet to the current master, based on the current replication position. If there is no match, it will fail.

func (*Wrangler) Reshard 

func (wr *Wrangler) Reshard(ctx context.Context, keyspace, workflow string, sources, targets []string,
	skipSchemaCopy bool, cell, tabletTypes string, autoStart, stopAfterCopy bool) error

Reshard initiates a resharding workflow.

func (*Wrangler) SetKeyspaceServedFrom 

func (wr *Wrangler) SetKeyspaceServedFrom(ctx context.Context, keyspace string, servedType topodatapb.TabletType, cells []string, sourceKeyspace string, remove bool) (err error)

SetKeyspaceServedFrom locks a keyspace and changes its ServerFromMap

func (*Wrangler) SetKeyspaceShardingInfo 

func (wr *Wrangler) SetKeyspaceShardingInfo(ctx context.Context, keyspace, shardingColumnName string, shardingColumnType topodatapb.KeyspaceIdType, force bool) (err error)

SetKeyspaceShardingInfo locks a keyspace and sets its ShardingColumnName and ShardingColumnType

func (*Wrangler) SetLogger 

func (wr *Wrangler) SetLogger(logger logutil.Logger)

SetLogger can be used to change the current logger. Not synchronized, no calls to this wrangler should be in progress.

func (*Wrangler) SetShardIsMasterServing 

func (wr *Wrangler) SetShardIsMasterServing(ctx context.Context, keyspace, shard string, isMasterServing bool) (err error)

SetShardIsMasterServing changes the IsMasterServing parameter of a shard. It does not rebuild any serving graph or do any consistency check. This is an emergency manual operation.

func (*Wrangler) SetShardTabletControl 

func (wr *Wrangler) SetShardTabletControl(ctx context.Context, keyspace, shard string, tabletType topodatapb.TabletType, cells []string, remove bool, blacklistedTables []string) (err error)

SetShardTabletControl changes the TabletControl records for a shard. It does not rebuild any serving graph or do cross-shard consistency check. - sets black listed tables in tablet control record

This takes the keyspace lock as to not interfere with resharding operations.

func (*Wrangler) SetSourceShards 

func (wr *Wrangler) SetSourceShards(ctx context.Context, keyspace, shard string, sources []*topodatapb.TabletAlias, tables []string) error

SetSourceShards is a utility function to override the SourceShards fields on a Shard.

func (*Wrangler) ShardReplicationStatuses 

func (wr *Wrangler) ShardReplicationStatuses(ctx context.Context, keyspace, shard string) ([]*topo.TabletInfo, []*replicationdatapb.Status, error)

ShardReplicationStatuses returns the ReplicationStatus for each tablet in a shard.

func (*Wrangler) ShowResharding 

func (wr *Wrangler) ShowResharding(ctx context.Context, keyspace, shard string) (err error)

ShowResharding shows all resharding related metadata for the keyspace/shard.

func (*Wrangler) ShowWorkflow added in v0.8.0 

func (wr *Wrangler) ShowWorkflow(ctx context.Context, workflow, keyspace string) (*ReplicationStatusResult, error)

ShowWorkflow will return all of the relevant replication related information for the given workflow.

func (*Wrangler) SourceShardAdd 

func (wr *Wrangler) SourceShardAdd(ctx context.Context, keyspace, shard string, uid uint32, skeyspace, sshard string, keyRange *topodatapb.KeyRange, tables []string) (err error)

SourceShardAdd will add a new SourceShard inside a shard.

func (*Wrangler) SourceShardDelete 

func (wr *Wrangler) SourceShardDelete(ctx context.Context, keyspace, shard string, uid uint32) (err error)

SourceShardDelete will delete a SourceShard inside a shard, by index.

This takes the keyspace lock as not to interfere with resharding operations.

func (*Wrangler) SplitClone 

func (wr *Wrangler) SplitClone(ctx context.Context, keyspace string, from, to []string) error

SplitClone initiates a SplitClone workflow.

func (*Wrangler) SwitchReads 

func (wr *Wrangler) SwitchReads(ctx context.Context, targetKeyspace, workflowName string, servedTypes []topodatapb.TabletType,
	cells []string, direction workflow.TrafficSwitchDirection, dryRun bool) (*[]string, error)

SwitchReads is a generic way of switching read traffic for a resharding workflow.

func (*Wrangler) SwitchWrites 

func (wr *Wrangler) SwitchWrites(ctx context.Context, targetKeyspace, workflowName string, timeout time.Duration,
	cancel, reverse, reverseReplication bool, dryRun bool) (journalID int64, dryRunResults *[]string, err error)

SwitchWrites is a generic way of migrating write traffic for a resharding workflow.

func (*Wrangler) TabletExternallyReparented 

func (wr *Wrangler) TabletExternallyReparented(ctx context.Context, newMasterAlias *topodatapb.TabletAlias) error

TabletExternallyReparented changes the type of new master for this shard to MASTER and updates it's tablet record in the topo. Updating the shard record is handled by the new master tablet

func (*Wrangler) TabletManagerClient 

func (wr *Wrangler) TabletManagerClient() tmclient.TabletManagerClient

TabletManagerClient returns the tmclient.TabletManagerClient this wrangler is using.

func (*Wrangler) TopoServer 

func (wr *Wrangler) TopoServer() *topo.Server

TopoServer returns the topo.Server this wrangler is using.

func (*Wrangler) UnmountExternalVitessCluster added in v0.10.0 

func (wr *Wrangler) UnmountExternalVitessCluster(ctx context.Context, clusterName string) error

UnmountExternalVitessCluster deletes a mounted cluster from the topo

func (*Wrangler) UpdateDisableQueryService 

func (wr *Wrangler) UpdateDisableQueryService(ctx context.Context, keyspace, shard string, tabletType topodatapb.TabletType, cells []string, disableQueryService bool) (err error)

UpdateDisableQueryService changes the TabletControl records for a shard. It updates serving graph

This takes the keyspace lock as to not interfere with resharding operations.

func (*Wrangler) UpdateSrvKeyspacePartitions 

func (wr *Wrangler) UpdateSrvKeyspacePartitions(ctx context.Context, keyspace, shard string, tabletType topodatapb.TabletType, cells []string, remove bool) (err error)

UpdateSrvKeyspacePartitions changes the SrvKeyspaceGraph for a shard. It updates serving graph

This takes the keyspace lock as to not interfere with resharding operations.

func (*Wrangler) VDiff 

func (wr *Wrangler) VDiff(ctx context.Context, targetKeyspace, workflowName, sourceCell, targetCell, tabletTypesStr string,
	filteredReplicationWaitTime time.Duration, format string, maxRows int64, tables string, debug, onlyPks bool) (map[string]*DiffReport, error)

VDiff reports differences between the sources and targets of a vreplication workflow.

func (*Wrangler) VExec 

func (wr *Wrangler) VExec(ctx context.Context, workflow, keyspace, query string, dryRun bool) (map[*topo.TabletInfo]*sqltypes.Result, error)

VExec executes queries on a table on all masters in the target keyspace of the workflow

func (*Wrangler) VExecResult added in v0.8.0 

func (wr *Wrangler) VExecResult(ctx context.Context, workflow, keyspace, query string, dryRun bool) (qr *sqltypes.Result, err error)

VExecResult runs VExec and the naggregates the results into a single *sqltypes.Result

func (*Wrangler) VReplicationExec 

func (wr *Wrangler) VReplicationExec(ctx context.Context, tabletAlias *topodatapb.TabletAlias, query string) (*querypb.QueryResult, error)

VReplicationExec executes a query remotely using the DBA pool

func (*Wrangler) Validate 

func (wr *Wrangler) Validate(ctx context.Context, pingTablets bool) error

Validate a whole TopologyServer tree

func (*Wrangler) ValidateKeyspace 

func (wr *Wrangler) ValidateKeyspace(ctx context.Context, keyspace string, pingTablets bool) error

ValidateKeyspace will validate a bunch of information in a keyspace is correct.

func (*Wrangler) ValidatePermissionsKeyspace 

func (wr *Wrangler) ValidatePermissionsKeyspace(ctx context.Context, keyspace string) error

ValidatePermissionsKeyspace validates all the permissions are the same in a keyspace

func (*Wrangler) ValidatePermissionsShard 

func (wr *Wrangler) ValidatePermissionsShard(ctx context.Context, keyspace, shard string) error

ValidatePermissionsShard validates all the permissions are the same in a shard

func (*Wrangler) ValidateSchemaKeyspace 

func (wr *Wrangler) ValidateSchemaKeyspace(ctx context.Context, keyspace string, excludeTables []string, includeViews, skipNoMaster bool, includeVSchema bool) error

ValidateSchemaKeyspace will diff the schema from all the tablets in the keyspace.

func (*Wrangler) ValidateSchemaShard 

func (wr *Wrangler) ValidateSchemaShard(ctx context.Context, keyspace, shard string, excludeTables []string, includeViews bool, includeVSchema bool) error

ValidateSchemaShard will diff the schema from all the tablets in the shard.

func (*Wrangler) ValidateShard 

func (wr *Wrangler) ValidateShard(ctx context.Context, keyspace, shard string, pingTablets bool) error

ValidateShard will validate a bunch of information in a shard is correct.

func (*Wrangler) ValidateVSchema added in v0.11.0 

func (wr *Wrangler) ValidateVSchema(ctx context.Context, keyspace string, shards []string, excludeTables []string, includeViews bool) error

ValidateVSchema compares the schema of each primary tablet in "keyspace/shards..." to the vschema and errs if there are differences

func (*Wrangler) ValidateVersionKeyspace 

func (wr *Wrangler) ValidateVersionKeyspace(ctx context.Context, keyspace string) error

ValidateVersionKeyspace validates all versions are the same in all tablets in a keyspace

func (*Wrangler) ValidateVersionShard 

func (wr *Wrangler) ValidateVersionShard(ctx context.Context, keyspace, shard string) error

ValidateVersionShard validates all versions are the same in all tablets in a shard

func (*Wrangler) VerticalSplitClone 

func (wr *Wrangler) VerticalSplitClone(ctx context.Context, fromKeyspace, toKeyspace string, tables []string) error

VerticalSplitClone initiates a VerticalSplitClone workflow.

func (*Wrangler) VtctldServer added in v0.9.0 

func (wr *Wrangler) VtctldServer() vtctlservicepb.VtctldServer

VtctldServer returns the vtctlservicepb.VtctldServer implementation this wrangler is using.

func (*Wrangler) WaitForDrain 

func (wr *Wrangler) WaitForDrain(ctx context.Context, cells []string, keyspace, shard string, servedType topodatapb.TabletType,
	retryDelay, healthCheckTopologyRefresh, healthcheckRetryDelay, healthCheckTimeout, initialWait time.Duration) error

WaitForDrain blocks until the selected tablets (cells/keyspace/shard/tablet_type) have reported a QPS rate of 0.0. NOTE: This is just an observation of one point in time and no guarantee that the tablet was actually drained. At later times, a QPS rate > 0.0 could still be observed.

func (*Wrangler) WaitForFilteredReplication 

func (wr *Wrangler) WaitForFilteredReplication(ctx context.Context, keyspace, shard string, maxDelay time.Duration) error

WaitForFilteredReplication will wait until the Filtered Replication process has finished.

func (*Wrangler) WorkflowAction 

func (wr *Wrangler) WorkflowAction(ctx context.Context, workflow, keyspace, action string, dryRun bool) (map[*topo.TabletInfo]*sqltypes.Result, error)

WorkflowAction can start/stop/delete or list streams in _vt.vreplication on all masters in the target keyspace of the workflow.

Source Files

View all Source files

Directories

Path Synopsis
Package testlib contains utility methods to include in unit tests to deal with topology common tasks, like fake tablets and action loops.
 Package testlib contains utility methods to include in unit tests to deal with topology common tasks, like fake tablets and action loops.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.