topotools

package
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 27, 2021 License: Apache-2.0 Imports: 24 Imported by: 8

Documentation

Overview

Package topotools contains high level functions based on vt/topo and vt/actionnode. It should not depend on anything else that's higher level. In particular, it cannot depend on:

  • vt/wrangler: much higher level, wrangler depends on topotools.
  • vt/tabletmanager/initiator: we don't want the various remote protocol dependencies here.

topotools is used by wrangler, so it ends up in all tools using wrangler (vtctl, vtctld, ...). It is also included by vttablet, so it contains:

  • most of the logic to create a shard / keyspace (tablet's init code)
  • some of the logic to perform a TabletExternallyReparented (RPC call to master vttablet to let it know it's the master).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ApplyVSchemaDDL 

func ApplyVSchemaDDL(ksName string, ks *vschemapb.Keyspace, alterVschema *sqlparser.AlterVschema) (*vschemapb.Keyspace, error)

ApplyVSchemaDDL applies the given DDL statement to the vschema keyspace definition and returns the modified keyspace object.

func ChangeType 

func ChangeType(ctx context.Context, ts *topo.Server, tabletAlias *topodatapb.TabletAlias, newType topodatapb.TabletType, masterTermStartTime *vttime.Time) (*topodatapb.Tablet, error)

ChangeType changes the type of the tablet. Make this external, since these transitions need to be forced from time to time.

If successful, the updated tablet record is returned.

func CheckOwnership 

func CheckOwnership(oldTablet, newTablet *topodatapb.Tablet) error

CheckOwnership returns nil iff the Hostname and port match on oldTablet and newTablet, which implies that no other tablet process has taken over the record.

func ConfigureTabletHook 

func ConfigureTabletHook(hk *hook.Hook, tabletAlias *topodatapb.TabletAlias)

ConfigureTabletHook configures the right parameters for a hook running locally on a tablet.

func CopyMapKeys 

func CopyMapKeys(m interface{}, typeHint interface{}) interface{}

CopyMapKeys copies keys from map m into a new slice with the type specified by typeHint. Reflection can't make a new slice type just based on the key type AFAICT.

func CopyMapValues 

func CopyMapValues(m interface{}, typeHint interface{}) interface{}

CopyMapValues copies values from map m into a new slice with the type specified by typeHint. Reflection can't make a new slice type just based on the key type AFAICT.

func DeleteTablet 

func DeleteTablet(ctx context.Context, ts *topo.Server, tablet *topodatapb.Tablet) error

DeleteTablet removes a tablet record from the topology: - the replication data record if any - the tablet record

func FindTabletByHostAndPort 

func FindTabletByHostAndPort(tabletMap map[string]*topo.TabletInfo, addr, portName string, port int32) (*topodatapb.TabletAlias, error)

FindTabletByHostAndPort searches within a tablet map for tablets.

func GetAllTablets 

func GetAllTablets(ctx context.Context, ts *topo.Server, cell string) ([]*topo.TabletInfo, error)

GetAllTablets returns a sorted list of tablets.

func GetAllTabletsAcrossCells 

func GetAllTabletsAcrossCells(ctx context.Context, ts *topo.Server) ([]*topo.TabletInfo, error)

GetAllTabletsAcrossCells returns all tablets from known cells. If it returns topo.ErrPartialResult, then the list is valid, but partial.

func GetRoutingRules added in v0.11.0 

func GetRoutingRules(ctx context.Context, ts *topo.Server) (map[string][]string, error)

GetRoutingRules fetches routing rules from the topology server and returns a mapping of fromTable=>[]toTables.

func GetTabletMapForCell added in v0.8.0 

func GetTabletMapForCell(ctx context.Context, ts *topo.Server, cell string) (map[string]*topo.TabletInfo, error)

GetTabletMapForCell returns a map of TabletInfo keyed by alias as string

func IsPrimaryTablet added in v0.10.0 

func IsPrimaryTablet(ctx context.Context, ts *topo.Server, ti *topo.TabletInfo) (bool, error)

IsPrimaryTablet is a helper function to determine whether the current tablet is a primary before we allow its tablet record to be deleted. The canonical way to determine the only true primary in a shard is to list all the tablets and find the one with the highest MasterTermStartTime among the ones that claim to be master.

We err on the side of caution here, i.e. we should never return false for a true primary tablet, but it is okay to return true for a tablet that isn't the true primary. This can occur if someone issues a DeleteTablet while the system is in transition (a reparenting event is in progress and parts of the topo have not yet been updated).

func MapKeys 

func MapKeys(m interface{}) []interface{}

MapKeys returns an array with th provided map keys.

func RebuildKeyspace 

func RebuildKeyspace(ctx context.Context, log logutil.Logger, ts *topo.Server, keyspace string, cells []string, allowPartial bool) (err error)

RebuildKeyspace rebuilds the serving graph data while locking out other changes.

func RebuildKeyspaceLocked 

func RebuildKeyspaceLocked(ctx context.Context, log logutil.Logger, ts *topo.Server, keyspace string, cells []string, allowPartial bool) error

RebuildKeyspaceLocked should only be used with an action lock on the keyspace - otherwise the consistency of the serving graph data can't be guaranteed.

Take data from the global keyspace and rebuild the local serving copies in each cell.

func RefreshTabletsByShard added in v0.11.0 

func RefreshTabletsByShard(ctx context.Context, ts *topo.Server, tmc tmclient.TabletManagerClient, si *topo.ShardInfo, cells []string, logger logutil.Logger) (isPartialRefresh bool, err error)

RefreshTabletsByShard calls RefreshState on all the tablets in a given shard.

It only returns errors from looking up the tablet map from the topology; errors returned from any RefreshState RPCs are logged and then ignored. Also, any tablets without a .Hostname set in the topology are skipped.

However, on partial errors from the topology, or errors from a RefreshState RPC will cause a boolean flag to be returned indicating only partial success.

func SaveRoutingRules added in v0.11.0 

func SaveRoutingRules(ctx context.Context, ts *topo.Server, rules map[string][]string) error

SaveRoutingRules converts a mapping of fromTable=>[]toTables into a vschemapb.RoutingRules protobuf message and saves it in the topology.

func SortedTabletMap 

func SortedTabletMap(tabletMap map[string]*topo.TabletInfo) (map[string]*topo.TabletInfo, map[string]*topo.TabletInfo)

SortedTabletMap returns two maps:

  • The replicaMap contains all the non-master non-scrapped hosts. This can be used as a list of replicas to fix up for reparenting
  • The masterMap contains all the tablets without parents (scrapped or not). This can be used to special case the old master, and any tablet in a weird state, left over, ...

func TabletIdent 

func TabletIdent(tablet *topodatapb.Tablet) string

TabletIdent returns a concise string representation of this tablet.

func TargetIdent 

func TargetIdent(target *querypb.Target) string

TargetIdent returns a concise string representation of a query target

func UpdateShardRecords added in v0.11.0 

func UpdateShardRecords(
	ctx context.Context,
	ts *topo.Server,
	tmc tmclient.TabletManagerClient,
	keyspace string,
	shards []*topo.ShardInfo,
	cells []string,
	servedType topodatapb.TabletType,
	isFrom bool,
	clearSourceShards bool,
	logger logutil.Logger,
) error

UpdateShardRecords updates the shard records based on 'from' or 'to' direction.

func ValidateForReshard 

func ValidateForReshard(sourceShards, targetShards []*topo.ShardInfo) error

ValidateForReshard returns an error if sourceShards cannot reshard into targetShards.

Types

type MaxReplicationPositionSearcher added in v0.10.0 

type MaxReplicationPositionSearcher struct {
	// contains filtered or unexported fields
}

MaxReplicationPositionSearcher provides a threadsafe way to find a tablet with the most advanced replication position.

A typical usage will look like: 

var (
	searcher = NewMaxReplicationPositionSearcher(tmc, logger, waitTimeout)
	wg sync.WaitGroup
)
for _, tablet := range tablets {
	wg.Add(1)
	go func(t *topodatapb.Tablet) {
		defer wg.Done()
		searcher.ProcessTablet(ctx, t)
	}(tablet)
}
wg.Wait()
maxPosTablet := searcher.MaxPositionTablet()

func NewMaxReplicationPositionSearcher added in v0.10.0 

func NewMaxReplicationPositionSearcher(tmc tmclient.TabletManagerClient, logger logutil.Logger, waitTimeout time.Duration) *MaxReplicationPositionSearcher

NewMaxReplicationPositionSearcher returns a new MaxReplicationPositionSearcher instance, ready to begin processing tablets. To reuse an existing instance, first call Reset().

func (*MaxReplicationPositionSearcher) MaxPositionTablet added in v0.10.0 

func (searcher *MaxReplicationPositionSearcher) MaxPositionTablet() *topodatapb.Tablet

MaxPositionTablet returns the most advanced-positioned tablet the searcher has seen so far.

func (*MaxReplicationPositionSearcher) ProcessTablet added in v0.10.0 

func (searcher *MaxReplicationPositionSearcher) ProcessTablet(ctx context.Context, tablet *topodatapb.Tablet)

ProcessTablet processes the replication position for a single tablet and updates the state of the searcher. It is safe to call from multiple goroutines.

func (*MaxReplicationPositionSearcher) Reset added in v0.10.0 

func (searcher *MaxReplicationPositionSearcher) Reset()

Reset clears any tracked position or tablet from the searcher, making this instance ready to begin a new search.

type OverlappingShards 

type OverlappingShards struct {
	Left  []*topo.ShardInfo
	Right []*topo.ShardInfo
}

OverlappingShards contains sets of shards that overlap which each-other. With this library, there is no guarantee of which set will be left or right.

func FindOverlappingShards 

func FindOverlappingShards(ctx context.Context, ts *topo.Server, keyspace string) ([]*OverlappingShards, error)

FindOverlappingShards will return an array of OverlappingShards for the provided keyspace. We do not support more than two overlapping shards (for instance, having 40-80, 40-60 and 40-50 in the same keyspace is not supported and will return an error). If shards don't perfectly overlap, they are not returned.

func OverlappingShardsForShard 

func OverlappingShardsForShard(os []*OverlappingShards, shardName string) *OverlappingShards

OverlappingShardsForShard returns the OverlappingShards object from the list that has he provided shard, or nil

func (*OverlappingShards) ContainsShard 

func (os *OverlappingShards) ContainsShard(shardName string) bool

ContainsShard returns true if either Left or Right lists contain the provided Shard.

Source Files

View all Source files

Directories

Path Synopsis
Package events defines the structures used for events dispatched from the wrangler package.
 Package events defines the structures used for events dispatched from the wrangler package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.