mysqlctl

package
Version: v0.0.0-...-275105e Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 20, 2017 License: Apache-2.0 Imports: 46 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// SQLStartSlave is the SQl command issued to start MySQL replication
	SQLStartSlave = "START SLAVE"

	// SQLStopSlave is the SQl command issued to stop MySQL replication
	SQLStopSlave = "STOP SLAVE"
)

Variables

View Source
var (
	// ErrNoBackup is returned when there is no backup.
	ErrNoBackup = errors.New("no available backup")

	// ErrExistingDB is returned when there's already an active DB.
	ErrExistingDB = errors.New("skipping restore due to existing database")
)
View Source
var (
	// ErrNotSlave means there is no slave status
	ErrNotSlave = errors.New("no slave status")
	// ErrNotMaster means there is no master status
	ErrNotMaster = errors.New("no master status")
)
View Source
var (
	// ErrBinlogUnavailable is returned by this library when we
	// cannot find a suitable binlog to satisfy the request.
	ErrBinlogUnavailable = fmt.Errorf("cannot find relevant binlogs on this server")
)

Functions

func Backup

func Backup(ctx context.Context, mysqld MysqlDaemon, logger logutil.Logger, dir, name string, backupConcurrency int, hookExtraEnv map[string]string) error

Backup is the main entry point for a backup: - uses the BackupStorage service to store a new backup - shuts down Mysqld during the backup - remember if we were replicating, restore the exact same state

func ConcurrentMap

func ConcurrentMap(concurrency, n int, fun MapFunc) error

ConcurrentMap applies fun in a concurrent manner on integers from 0 to n-1 (they are assumed to be indexes of some slice containing items to be processed). The first error returned by a fun application will returned (subsequent errors will only be logged). It will use concurrency goroutines.

func CreateReparentJournal

func CreateReparentJournal() []string

CreateReparentJournal returns the commands to execute to create the _vt.reparent_journal table. It is safe to run these commands even if the table already exists.

If the table was created by Vitess version 2.0, the following command may need to be run: ALTER TABLE _vt.reparent_journal MODIFY COLUMN replication_position VARBINARY(64000);

func FindSlaves

func FindSlaves(mysqld MysqlDaemon) ([]string, error)

FindSlaves gets IP addresses for all currently connected slaves.

func GetPermissions

func GetPermissions(mysqld MysqlDaemon) (*tabletmanagerdatapb.Permissions, error)

GetPermissions lists the permissions on the mysqld

func MycnfFile

func MycnfFile(uid uint32) string

MycnfFile returns the default location of the my.cnf file.

func PopulateReparentJournal

func PopulateReparentJournal(timeCreatedNS int64, actionName, masterAlias string, pos mysql.Position) string

PopulateReparentJournal returns the SQL command to use to populate the _vt.reparent_journal table, as well as the time_created_ns value used.

func RegisterFlags

func RegisterFlags()

RegisterFlags registers the command line flags for specifying the values of a mycnf config file. See NewMycnfFromFlags to get the supported modes.

func ResolveTables

func ResolveTables(mysqld MysqlDaemon, dbName string, tables []string) ([]string, error)

ResolveTables returns a list of actual tables+views matching a list of regexps

func Restore

func Restore(
	ctx context.Context,
	mysqld MysqlDaemon,
	dir string,
	restoreConcurrency int,
	hookExtraEnv map[string]string,
	localMetadata map[string]string,
	logger logutil.Logger,
	deleteBeforeRestore bool,
	dbName string) (mysql.Position, error)

Restore is the main entry point for backup restore. If there is no appropriate backup on the BackupStorage, Restore logs an error and returns ErrNoBackup. Any other error is returned.

func StartSlave

func StartSlave(md MysqlDaemon, hookExtraEnv map[string]string) error

StartSlave starts a slave on the provided MysqldDaemon

func StatusToProto

func StatusToProto(r Status) *replicationdatapb.Status

StatusToProto translates a Status to proto3

func StopSlave

func StopSlave(md MysqlDaemon, hookExtraEnv map[string]string) error

StopSlave stops a slave on the provided MysqldDaemon

func TabletDir

func TabletDir(uid uint32) string

TabletDir returns the default directory for a tablet

func TopLevelDirs

func TopLevelDirs() []string

TopLevelDirs returns the list of directories in the toplevel tablet directory that might be located in a different place.

func WaitBlpPosition

func WaitBlpPosition(ctx context.Context, mysqld MysqlDaemon, sql string, replicationPosition string) error

WaitBlpPosition will wait for the filtered replication to reach at least the provided position.

func WaitForSlaveStart

func WaitForSlaveStart(mysqld MysqlDaemon, slaveStartDeadline int) error

WaitForSlaveStart waits until the deadline for replication to start. This validates the current master is correct and can be connected to.

Types

type BackupManifest

type BackupManifest struct {
	// FileEntries contains all the files in the backup
	FileEntries []FileEntry

	// Position is the position at which the backup was taken
	Position mysql.Position

	// TransformHook that was used on the files, if any.
	TransformHook string

	// SkipCompress can be set if the backup files were not run
	// through gzip. It is the negative of the flag, so old
	// backups that don't have this flag are assumed to be
	// compressed.
	SkipCompress bool
}

BackupManifest represents the backup. It lists all the files, the Position that the backup was taken at, and the transform hook used, if any.

type FakeMysqlDaemon

type FakeMysqlDaemon struct {

	// Mycnf will be returned by Cnf()
	Mycnf *Mycnf

	// Running is used by Start / Shutdown
	Running bool

	// MysqlPort will be returned by GetMysqlPort(). Set to -1 to
	// return an error.
	MysqlPort int32

	// Replicating is updated when calling StartSlave / StopSlave
	// (it is not used at all when calling SlaveStatus, it is the
	// test owner responsability to have these two match)
	Replicating bool

	// ResetReplicationResult is returned by ResetReplication
	ResetReplicationResult []string

	// ResetReplicationError is returned by ResetReplication
	ResetReplicationError error

	// ResetSlaveResult is returned by ResetSlave
	ResetSlaveResult []string

	// ResetSlaveError is returned by ResetSlave
	ResetSlaveError error

	// CurrentMasterPosition is returned by MasterPosition
	// and SlaveStatus
	CurrentMasterPosition mysql.Position

	// SlaveStatusError is used by SlaveStatus
	SlaveStatusError error

	// CurrentMasterHost is returned by SlaveStatus
	CurrentMasterHost string

	// CurrentMasterport is returned by SlaveStatus
	CurrentMasterPort int

	// SecondsBehindMaster is returned by SlaveStatus
	SecondsBehindMaster uint

	// ReadOnly is the current value of the flag
	ReadOnly bool

	// SetSlavePositionCommandsPos is matched against the input
	// of SetSlavePositionCommands. If it doesn't match,
	// SetSlavePositionCommands will return an error.
	SetSlavePositionCommandsPos mysql.Position

	// SetSlavePositionCommandsResult is what
	// SetSlavePositionCommands will return
	SetSlavePositionCommandsResult []string

	// SetMasterCommandsInput is matched against the input
	// of SetMasterCommands (as "%v:%v"). If it doesn't match,
	// SetMasterCommands will return an error.
	SetMasterCommandsInput string

	// SetMasterCommandsResult is what
	// SetMasterCommands will return
	SetMasterCommandsResult []string

	// DemoteMasterPosition is returned by DemoteMaster
	DemoteMasterPosition mysql.Position

	// WaitMasterPosition is checked by WaitMasterPos, if the
	// same it returns nil, if different it returns an error
	WaitMasterPosition mysql.Position

	// PromoteSlaveResult is returned by PromoteSlave
	PromoteSlaveResult mysql.Position

	// SchemaFunc provides the return value for GetSchema.
	// If not defined, the "Schema" field will be used instead, see below.
	SchemaFunc func() (*tabletmanagerdatapb.SchemaDefinition, error)

	// Schema will be returned by GetSchema. If nil we'll
	// return an error.
	Schema *tabletmanagerdatapb.SchemaDefinition

	// PreflightSchemaChangeResult will be returned by PreflightSchemaChange.
	// If nil we'll return an error.
	PreflightSchemaChangeResult []*tabletmanagerdatapb.SchemaChangeResult

	// ApplySchemaChangeResult will be returned by ApplySchemaChange.
	// If nil we'll return an error.
	ApplySchemaChangeResult *tabletmanagerdatapb.SchemaChangeResult

	// DbAppConnectionFactory is the factory for making fake db app connections
	DbAppConnectionFactory func() (dbconnpool.PoolConnection, error)

	// ExpectedExecuteSuperQueryList is what we expect
	// ExecuteSuperQueryList to be called with. If it doesn't
	// match, ExecuteSuperQueryList will return an error.
	// Note each string is just a substring if it begins with SUB,
	// so we support partial queries (usefull when queries contain
	// data fields like timestamps)
	ExpectedExecuteSuperQueryList []string

	// ExpectedExecuteSuperQueryCurrent is the current index of the queries
	// we expect
	ExpectedExecuteSuperQueryCurrent int

	// FetchSuperQueryResults is used by FetchSuperQuery
	FetchSuperQueryMap map[string]*sqltypes.Result

	// BinlogPlayerEnabled is used by {Enable,Disable}BinlogPlayer
	BinlogPlayerEnabled bool

	// SemiSyncMasterEnabled represents the state of rpl_semi_sync_master_enabled.
	SemiSyncMasterEnabled bool
	// SemiSyncSlaveEnabled represents the state of rpl_semi_sync_slave_enabled.
	SemiSyncSlaveEnabled bool
	// contains filtered or unexported fields
}

FakeMysqlDaemon implements MysqlDaemon and allows the user to fake everything.

func NewFakeMysqlDaemon

func NewFakeMysqlDaemon(db *fakesqldb.DB) *FakeMysqlDaemon

NewFakeMysqlDaemon returns a FakeMysqlDaemon where mysqld appears to be running, based on a fakesqldb.DB. 'db' can be nil if the test doesn't use a database at all.

func (*FakeMysqlDaemon) ApplySchemaChange

func (fmd *FakeMysqlDaemon) ApplySchemaChange(dbName string, change *tmutils.SchemaChange) (*tabletmanagerdatapb.SchemaChangeResult, error)

ApplySchemaChange is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) CheckSuperQueryList

func (fmd *FakeMysqlDaemon) CheckSuperQueryList() error

CheckSuperQueryList returns an error if all the queries we expected haven't been seen.

func (*FakeMysqlDaemon) Close

func (fmd *FakeMysqlDaemon) Close()

Close is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) Cnf

func (fmd *FakeMysqlDaemon) Cnf() *Mycnf

Cnf is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) DemoteMaster

func (fmd *FakeMysqlDaemon) DemoteMaster() (mysql.Position, error)

DemoteMaster is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) DisableBinlogPlayback

func (fmd *FakeMysqlDaemon) DisableBinlogPlayback() error

DisableBinlogPlayback disable playback of binlog events

func (*FakeMysqlDaemon) EnableBinlogPlayback

func (fmd *FakeMysqlDaemon) EnableBinlogPlayback() error

EnableBinlogPlayback is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) ExecuteSuperQueryList

func (fmd *FakeMysqlDaemon) ExecuteSuperQueryList(ctx context.Context, queryList []string) error

ExecuteSuperQueryList is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) FetchSuperQuery

func (fmd *FakeMysqlDaemon) FetchSuperQuery(ctx context.Context, query string) (*sqltypes.Result, error)

FetchSuperQuery returns the results from the map, if any

func (*FakeMysqlDaemon) GetAllPrivsConnection

func (fmd *FakeMysqlDaemon) GetAllPrivsConnection() (*dbconnpool.DBConnection, error)

GetAllPrivsConnection is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) GetAppConnection

func (fmd *FakeMysqlDaemon) GetAppConnection(ctx context.Context) (dbconnpool.PoolConnection, error)

GetAppConnection is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) GetDbaConnection

func (fmd *FakeMysqlDaemon) GetDbaConnection() (*dbconnpool.DBConnection, error)

GetDbaConnection is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) GetMysqlPort

func (fmd *FakeMysqlDaemon) GetMysqlPort() (int32, error)

GetMysqlPort is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) GetSchema

func (fmd *FakeMysqlDaemon) GetSchema(dbName string, tables, excludeTables []string, includeViews bool) (*tabletmanagerdatapb.SchemaDefinition, error)

GetSchema is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) IsReadOnly

func (fmd *FakeMysqlDaemon) IsReadOnly() (bool, error)

IsReadOnly is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) MasterPosition

func (fmd *FakeMysqlDaemon) MasterPosition() (mysql.Position, error)

MasterPosition is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) NewSlaveConnection

func (fmd *FakeMysqlDaemon) NewSlaveConnection() (*SlaveConnection, error)

NewSlaveConnection is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) PreflightSchemaChange

func (fmd *FakeMysqlDaemon) PreflightSchemaChange(dbName string, changes []string) ([]*tabletmanagerdatapb.SchemaChangeResult, error)

PreflightSchemaChange is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) PromoteSlave

func (fmd *FakeMysqlDaemon) PromoteSlave(hookExtraEnv map[string]string) (mysql.Position, error)

PromoteSlave is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) ReinitConfig

func (fmd *FakeMysqlDaemon) ReinitConfig(ctx context.Context) error

ReinitConfig is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) ResetReplicationCommands

func (fmd *FakeMysqlDaemon) ResetReplicationCommands() ([]string, error)

ResetReplicationCommands is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) ResetSlaveCommands

func (fmd *FakeMysqlDaemon) ResetSlaveCommands() ([]string, error)

ResetSlaveCommands is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) RunMysqlUpgrade

func (fmd *FakeMysqlDaemon) RunMysqlUpgrade() error

RunMysqlUpgrade is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) SemiSyncEnabled

func (fmd *FakeMysqlDaemon) SemiSyncEnabled() (master, slave bool)

SemiSyncEnabled is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) SemiSyncSlaveStatus

func (fmd *FakeMysqlDaemon) SemiSyncSlaveStatus() (bool, error)

SemiSyncSlaveStatus is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) SetMasterCommands

func (fmd *FakeMysqlDaemon) SetMasterCommands(masterHost string, masterPort int) ([]string, error)

SetMasterCommands is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) SetReadOnly

func (fmd *FakeMysqlDaemon) SetReadOnly(on bool) error

SetReadOnly is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) SetSemiSyncEnabled

func (fmd *FakeMysqlDaemon) SetSemiSyncEnabled(master, slave bool) error

SetSemiSyncEnabled is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) SetSlavePositionCommands

func (fmd *FakeMysqlDaemon) SetSlavePositionCommands(pos mysql.Position) ([]string, error)

SetSlavePositionCommands is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) Shutdown

func (fmd *FakeMysqlDaemon) Shutdown(ctx context.Context, waitForMysqld bool) error

Shutdown is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) SlaveStatus

func (fmd *FakeMysqlDaemon) SlaveStatus() (Status, error)

SlaveStatus is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) Start

func (fmd *FakeMysqlDaemon) Start(ctx context.Context, mysqldArgs ...string) error

Start is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) TabletDir

func (fmd *FakeMysqlDaemon) TabletDir() string

TabletDir is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) Wait

func (fmd *FakeMysqlDaemon) Wait(ctx context.Context) error

Wait is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) WaitForReparentJournal

func (fmd *FakeMysqlDaemon) WaitForReparentJournal(ctx context.Context, timeCreatedNS int64) error

WaitForReparentJournal is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) WaitMasterPos

func (fmd *FakeMysqlDaemon) WaitMasterPos(_ context.Context, pos mysql.Position) error

WaitMasterPos is part of the MysqlDaemon interface

type FileEntry

type FileEntry struct {
	// Base is one of:
	// - backupInnodbDataHomeDir for files that go into Mycnf.InnodbDataHomeDir
	// - backupInnodbLogGroupHomeDir for files that go into Mycnf.InnodbLogGroupHomeDir
	// - backupData for files that go into Mycnf.DataDir
	Base string

	// Name is the file name, relative to Base
	Name string

	// Hash is the hash of the final data (transformed and
	// compressed if specified) stored in the BackupStorage.
	Hash string
}

FileEntry is one file to backup

type MapFunc

type MapFunc func(index int) error

type Mycnf

type Mycnf struct {
	// ServerID is the unique id for this server.
	// Used to create a bunch of named directories.
	ServerID uint32

	// MysqlPort is the port for the MySQL server running on this machine.
	// It is mainly used to communicate with topology server.
	MysqlPort int32

	// DataDir is where the table files are
	// (used by vt software for Clone)
	DataDir string

	// InnodbDataHomeDir is the data directory for innodb.
	// (used by vt software for Clone)
	InnodbDataHomeDir string

	// InnodbLogGroupHomeDir is the logs directory for innodb.
	// (used by vt software for Clone)
	InnodbLogGroupHomeDir string

	// SocketFile is the path to the local mysql.sock file.
	// (used by vt software to check server is running)
	SocketFile string

	// GeneralLogPath is the path to store general logs at,
	// if general-log is enabled.
	// (unused by vt software for now)
	GeneralLogPath string

	// ErrorLogPath is the path to store error logs at.
	// (unused by vt software for now)
	ErrorLogPath string

	// SlowLogPath is the slow query log path
	// (unused by vt software for now)
	SlowLogPath string

	// RelayLogPath is the path of the relay logs
	// (unused by vt software for now)
	RelayLogPath string

	// RelayLogIndexPath is the file name for the relay log index
	// (unused by vt software for now)
	RelayLogIndexPath string

	// RelayLogInfoPath is the file name for the relay log info file
	// (unused by vt software for now)
	RelayLogInfoPath string

	// BinLogPath is the base path for binlogs
	// (used by vt software for binlog streaming)
	BinLogPath string

	// MasterInfoFile is the master.info file location.
	// (unused by vt software for now)
	MasterInfoFile string

	// PidFile is the mysql.pid file location
	// (used by vt software to check server is running)
	PidFile string

	// TmpDir is where to create temporary tables
	// (unused by vt software for now)
	TmpDir string

	// SlaveLoadTmpDir is where to create tmp files for replication
	// (unused by vt software for now)
	SlaveLoadTmpDir string
	// contains filtered or unexported fields
}

Mycnf is a memory structure that contains a bunch of interesting parameters to start mysqld. It can be used to generate standard my.cnf files from a server id and mysql port. It can also be populated from an existing my.cnf, or by command line parameters.

func NewMycnf

func NewMycnf(tabletUID uint32, mysqlPort int32) *Mycnf

NewMycnf fills the Mycnf structure with vt root paths and derived values. This is used to fill out the cnfTemplate values and generate my.cnf. uid is a unique id for a particular tablet - it must be unique within the tabletservers deployed within a keyspace, lest there be collisions on disk. mysqldPort needs to be unique per instance per machine.

func NewMycnfFromFlags

func NewMycnfFromFlags(uid uint32) (mycnf *Mycnf, err error)

NewMycnfFromFlags creates a Mycnf object from the command line flags.

Multiple modes are supported: - at least mycnf_server_id is set on the command line

--> then we read all parameters from the command line, and not from
    any my.cnf file.

- mycnf_server_id is not passed in, but mycnf-file is passed in

--> then we read that mycnf file

- mycnf_server_id and mycnf-file are not passed in:

--> then we use the default location of the my.cnf file for the
    provided uid and read that my.cnf file.

RegisterCommandLineFlags should have been called before calling this, otherwise we'll panic.

func ReadMycnf

func ReadMycnf(mycnf *Mycnf) (*Mycnf, error)

ReadMycnf will read an existing my.cnf from disk, and update the passed in Mycnf object with values from the my.cnf on disk.

func (*Mycnf) RandomizeMysqlServerID

func (cnf *Mycnf) RandomizeMysqlServerID() error

RandomizeMysqlServerID generates a random MySQL server_id.

The value assigned to ServerID will be in the range [100, 2^31): - It avoids 0 because that's reserved for mysqlbinlog dumps. - It also avoids 1-99 because low numbers are used for fake slave connections. See NewSlaveConnection() in slave_connection.go for more on that. - It avoids the 2^31 - 2^32-1 range, as there seems to be some confusion there. The main MySQL documentation at: http://dev.mysql.com/doc/refman/5.7/en/replication-options.html implies serverID is a full 32 bits number. The semi-sync log line at startup '[Note] Start semi-sync binlog_dump to slave ...' interprets the server_id as signed 32-bit (shows negative numbers for that range). Such an ID may also be responsible for a mysqld crash in semi-sync code, although we haven't been able to verify that yet. The issue for that is: https://github.com/youtube/vitess/issues/2280

type MysqlDaemon

type MysqlDaemon interface {
	// Cnf returns the underlying mycnf
	Cnf() *Mycnf
	// TabletDir returns the tablet directory.
	TabletDir() string

	// methods related to mysql running or not
	Start(ctx context.Context, mysqldArgs ...string) error
	Shutdown(ctx context.Context, waitForMysqld bool) error
	RunMysqlUpgrade() error
	ReinitConfig(ctx context.Context) error
	Wait(ctx context.Context) error

	// GetMysqlPort returns the current port mysql is listening on.
	GetMysqlPort() (int32, error)

	// replication related methods
	SlaveStatus() (Status, error)
	SetSemiSyncEnabled(master, slave bool) error
	SemiSyncEnabled() (master, slave bool)
	SemiSyncSlaveStatus() (bool, error)

	// reparenting related methods
	ResetReplicationCommands() ([]string, error)
	MasterPosition() (mysql.Position, error)
	IsReadOnly() (bool, error)
	SetReadOnly(on bool) error
	SetSlavePositionCommands(pos mysql.Position) ([]string, error)
	SetMasterCommands(masterHost string, masterPort int) ([]string, error)
	WaitForReparentJournal(ctx context.Context, timeCreatedNS int64) error

	// Used for backup restoration, to ensure we have a clean slate
	ResetSlaveCommands() ([]string, error)

	// DemoteMaster waits for all current transactions to finish,
	// and returns the current replication position. It will not
	// change the read_only state of the server.
	DemoteMaster() (mysql.Position, error)

	WaitMasterPos(context.Context, mysql.Position) error

	// PromoteSlave makes the slave the new master. It will not change
	// the read_only state of the server.
	PromoteSlave(map[string]string) (mysql.Position, error)

	// Schema related methods
	GetSchema(dbName string, tables, excludeTables []string, includeViews bool) (*tabletmanagerdatapb.SchemaDefinition, error)
	PreflightSchemaChange(dbName string, changes []string) ([]*tabletmanagerdatapb.SchemaChangeResult, error)
	ApplySchemaChange(dbName string, change *tmutils.SchemaChange) (*tabletmanagerdatapb.SchemaChangeResult, error)

	// GetAppConnection returns a app connection to be able to talk to the database.
	GetAppConnection(ctx context.Context) (dbconnpool.PoolConnection, error)
	// GetDbaConnection returns a dba connection.
	GetDbaConnection() (*dbconnpool.DBConnection, error)
	// GetAllPrivsConnection returns an allprivs connection (for user with all privileges except SUPER).
	GetAllPrivsConnection() (*dbconnpool.DBConnection, error)

	// ExecuteSuperQueryList executes a list of queries, no result
	ExecuteSuperQueryList(ctx context.Context, queryList []string) error

	// FetchSuperQuery executes one query, returns the result
	FetchSuperQuery(ctx context.Context, query string) (*sqltypes.Result, error)

	// NewSlaveConnection returns a SlaveConnection to the database.
	NewSlaveConnection() (*SlaveConnection, error)

	// EnableBinlogPlayback enables playback of binlog events
	EnableBinlogPlayback() error

	// DisableBinlogPlayback disable playback of binlog events
	DisableBinlogPlayback() error

	// Close will close this instance of Mysqld. It will wait for all dba
	// queries to be finished.
	Close()
}

MysqlDaemon is the interface we use for abstracting Mysqld.

type MysqlFlavor

type MysqlFlavor interface {
	// VersionMatch returns true if the version string (from
	// SELECT VERSION()) represents a server that this flavor
	// knows how to talk to.
	VersionMatch(version string) bool

	// MasterPosition returns the ReplicationPosition of a master.
	MasterPosition(mysqld *Mysqld) (mysql.Position, error)

	// SlaveStatus returns the ReplicationStatus of a slave.
	SlaveStatus(mysqld *Mysqld) (Status, error)

	// ResetReplicationCommands returns the commands to completely reset
	// replication on the host.
	ResetReplicationCommands() []string

	// ResetSlaveCommands returns the commands to reset just slave
	// settings on the host.
	ResetSlaveCommands() []string

	// PromoteSlaveCommands returns the commands to run to change
	// a slave into a master.
	PromoteSlaveCommands() []string

	// SetSlavePositionCommands returns the commands to set the
	// replication position at which the slave will resume
	// when it is later reparented with SetMasterCommands.
	SetSlavePositionCommands(pos mysql.Position) ([]string, error)

	// SetMasterCommands returns the commands to use the provided master
	// as the new master (without changing any GTID position).
	// It is guaranteed to be called with replication stopped.
	// It should not start or stop replication.
	SetMasterCommands(params *mysql.ConnParams, masterHost string, masterPort int, masterConnectRetry int) ([]string, error)

	// ParseGTID parses a GTID in the canonical format of this
	// MySQL flavor into a replication.GTID interface value.
	ParseGTID(string) (mysql.GTID, error)

	// ParseReplicationPosition parses a replication position in
	// the canonical format of this MySQL flavor into a
	// mysql.Position struct.
	ParseReplicationPosition(string) (mysql.Position, error)

	// SendBinlogDumpCommand sends the flavor-specific version of
	// the COM_BINLOG_DUMP command to start dumping raw binlog
	// events over a slave connection, starting at a given GTID.
	SendBinlogDumpCommand(conn *SlaveConnection, startPos mysql.Position) error

	// MakeBinlogEvent takes a raw packet from the MySQL binlog
	// stream connection and returns a BinlogEvent through which
	// the packet can be examined.
	MakeBinlogEvent(buf []byte) mysql.BinlogEvent

	// WaitMasterPos waits until slave replication reaches at
	// least targetPos.
	WaitMasterPos(ctx context.Context, mysqld *Mysqld, targetPos mysql.Position) error

	// EnableBinlogPlayback prepares the server to play back
	// events from a binlog stream.  Whatever it does for a given
	// flavor, it must be idempotent.
	EnableBinlogPlayback(mysqld *Mysqld) error

	// DisableBinlogPlayback returns the server to the normal
	// state after playback is done.  Whatever it does for a given
	// flavor, it must be idempotent.
	DisableBinlogPlayback(mysqld *Mysqld) error
}

MysqlFlavor is the abstract interface for a flavor.

type Mysqld

type Mysqld struct {
	// contains filtered or unexported fields
}

Mysqld is the object that represents a mysqld daemon running on this server.

func CreateMysqld

func CreateMysqld(tabletUID uint32, mysqlSocket string, mysqlPort int32, dbconfigFlags dbconfigs.DBConfigFlag) (*Mysqld, error)

CreateMysqld returns a Mysqld object to use for working with a MySQL installation that hasn't been set up yet. This will generate a new my.cnf from scratch and use that to call NewMysqld().

func NewMysqld

func NewMysqld(config *Mycnf, dbcfgs *dbconfigs.DBConfigs, dbconfigsFlags dbconfigs.DBConfigFlag) *Mysqld

NewMysqld creates a Mysqld object based on the provided configuration and connection parameters.

func OpenMysqld

func OpenMysqld(tabletUID uint32, dbconfigFlags dbconfigs.DBConfigFlag) (*Mysqld, error)

OpenMysqld returns a Mysqld object to use for working with a MySQL installation that already exists. This will look for an existing my.cnf file and use that to call NewMysqld().

func (*Mysqld) ApplySchemaChange

func (mysqld *Mysqld) ApplySchemaChange(dbName string, change *tmutils.SchemaChange) (*tabletmanagerdatapb.SchemaChangeResult, error)

ApplySchemaChange will apply the schema change to the given database.

func (*Mysqld) Close

func (mysqld *Mysqld) Close()

Close will close this instance of Mysqld. It will wait for all dba queries to be finished.

func (*Mysqld) Cnf

func (mysqld *Mysqld) Cnf() *Mycnf

Cnf returns the mysql config for the daemon

func (*Mysqld) DemoteMaster

func (mysqld *Mysqld) DemoteMaster() (rp mysql.Position, err error)

DemoteMaster will gracefully demote a master mysql instance to read only. If the master is still alive, then we need to demote it gracefully make it read-only, flush the writes and get the position

func (*Mysqld) DisableBinlogPlayback

func (mysqld *Mysqld) DisableBinlogPlayback() error

DisableBinlogPlayback returns the server to the normal state after streaming. Whatever it does for a given flavor, it must be idempotent.

func (*Mysqld) EnableBinlogPlayback

func (mysqld *Mysqld) EnableBinlogPlayback() error

EnableBinlogPlayback prepares the server to play back events from a binlog stream. Whatever it does for a given flavor, it must be idempotent.

func (*Mysqld) ExecuteSuperQuery

func (mysqld *Mysqld) ExecuteSuperQuery(ctx context.Context, query string) error

ExecuteSuperQuery allows the user to execute a query as a super user.

func (*Mysqld) ExecuteSuperQueryList

func (mysqld *Mysqld) ExecuteSuperQueryList(ctx context.Context, queryList []string) error

ExecuteSuperQueryList alows the user to execute queries as a super user.

func (*Mysqld) FetchSuperQuery

func (mysqld *Mysqld) FetchSuperQuery(ctx context.Context, query string) (*sqltypes.Result, error)

FetchSuperQuery returns the results of executing a query as a super user.

func (*Mysqld) GetAllPrivsConnection

func (mysqld *Mysqld) GetAllPrivsConnection() (*dbconnpool.DBConnection, error)

GetAllPrivsConnection creates a new DBConnection.

func (*Mysqld) GetAppConnection

func (mysqld *Mysqld) GetAppConnection(ctx context.Context) (dbconnpool.PoolConnection, error)

GetAppConnection returns a connection from the app pool. Recycle needs to be called on the result.

func (*Mysqld) GetColumns

func (mysqld *Mysqld) GetColumns(dbName, table string) ([]string, error)

GetColumns returns the columns of table.

func (*Mysqld) GetDbaConnection

func (mysqld *Mysqld) GetDbaConnection() (*dbconnpool.DBConnection, error)

GetDbaConnection creates a new DBConnection.

func (*Mysqld) GetMysqlPort

func (mysqld *Mysqld) GetMysqlPort() (int32, error)

GetMysqlPort returns mysql port

func (*Mysqld) GetPrimaryKeyColumns

func (mysqld *Mysqld) GetPrimaryKeyColumns(dbName, table string) ([]string, error)

GetPrimaryKeyColumns returns the primary key columns of table.

func (*Mysqld) GetSchema

func (mysqld *Mysqld) GetSchema(dbName string, tables, excludeTables []string, includeViews bool) (*tabletmanagerdatapb.SchemaDefinition, error)

GetSchema returns the schema for database for tables listed in tables. If tables is empty, return the schema for all tables.

func (*Mysqld) Init

func (mysqld *Mysqld) Init(ctx context.Context, initDBSQLFile string) error

Init will create the default directory structure for the mysqld process, generate / configure a my.cnf file, install a skeleton database, and apply the provided initial SQL file.

func (*Mysqld) IsReadOnly

func (mysqld *Mysqld) IsReadOnly() (bool, error)

IsReadOnly return true if the instance is read only

func (*Mysqld) MasterPosition

func (mysqld *Mysqld) MasterPosition() (rp mysql.Position, err error)

MasterPosition returns master replication position

func (*Mysqld) NewSlaveConnection

func (mysqld *Mysqld) NewSlaveConnection() (*SlaveConnection, error)

NewSlaveConnection creates a new slave connection to the mysqld instance. It uses a pools.IDPool to ensure that the server IDs used to connect are unique within this process. This is done with the assumptions that:

1) No other processes are making fake slave connections to our mysqld. 2) No real slave servers will have IDs in the range 1-N where N is the peak

number of concurrent fake slave connections we will ever make.

func (*Mysqld) OnTerm

func (mysqld *Mysqld) OnTerm(f func())

OnTerm registers a function to be called if mysqld terminates for any reason other than a call to Mysqld.Shutdown(). This only works if mysqld was actually started by calling Start() on this Mysqld instance.

func (*Mysqld) PreflightSchemaChange

func (mysqld *Mysqld) PreflightSchemaChange(dbName string, changes []string) ([]*tabletmanagerdatapb.SchemaChangeResult, error)

PreflightSchemaChange checks the schema changes in "changes" by applying them to an intermediate database that has the same schema as the target database.

func (*Mysqld) PromoteSlave

func (mysqld *Mysqld) PromoteSlave(hookExtraEnv map[string]string) (mysql.Position, error)

PromoteSlave will promote a slave to be the new master.

func (*Mysqld) RefreshConfig

func (mysqld *Mysqld) RefreshConfig() error

RefreshConfig attempts to recreate the my.cnf from templates, and log and swap in to place if it's updated. It keeps a copy of the last version in case fallback is required. Should be called from a stable replica, server_id is not regenerated.

func (*Mysqld) ReinitConfig

func (mysqld *Mysqld) ReinitConfig(ctx context.Context) error

ReinitConfig updates the config file as if Mysqld is initializing. At the moment it only randomizes ServerID because it's not safe to restore a replica from a backup and then give it the same ServerID as before, MySQL can then skip transactions in the replication stream with the same server_id.

func (*Mysqld) ResetReplicationCommands

func (mysqld *Mysqld) ResetReplicationCommands() ([]string, error)

ResetReplicationCommands returns the commands to run to reset all replication for this host.

func (*Mysqld) ResetSlaveCommands

func (mysqld *Mysqld) ResetSlaveCommands() ([]string, error)

ResetSlaveCommands returns the commands to run to reset just slave settings for this host

func (*Mysqld) RunMysqlUpgrade

func (mysqld *Mysqld) RunMysqlUpgrade() error

RunMysqlUpgrade will run the mysql_upgrade program on the current install. Will be called only when mysqld is running with no network and no grant tables.

func (*Mysqld) SemiSyncEnabled

func (mysqld *Mysqld) SemiSyncEnabled() (master, slave bool)

SemiSyncEnabled returns whether semi-sync is enabled for master or slave. If the semi-sync plugin is not loaded, we assume semi-sync is disabled.

func (*Mysqld) SemiSyncSlaveStatus

func (mysqld *Mysqld) SemiSyncSlaveStatus() (bool, error)

SemiSyncSlaveStatus returns whether semi-sync is currently used by replication.

func (*Mysqld) SetMasterCommands

func (mysqld *Mysqld) SetMasterCommands(masterHost string, masterPort int) ([]string, error)

SetMasterCommands returns the commands to run to make the provided host / port the master.

func (*Mysqld) SetReadOnly

func (mysqld *Mysqld) SetReadOnly(on bool) error

SetReadOnly set/unset the read_only flag

func (*Mysqld) SetSemiSyncEnabled

func (mysqld *Mysqld) SetSemiSyncEnabled(master, slave bool) error

SetSemiSyncEnabled enables or disables semi-sync replication for master and/or slave mode.

func (*Mysqld) SetSlavePositionCommands

func (mysqld *Mysqld) SetSlavePositionCommands(pos mysql.Position) ([]string, error)

SetSlavePositionCommands returns the commands to set the replication position at which the slave will resume when it is later reparented with SetMasterCommands.

func (*Mysqld) Shutdown

func (mysqld *Mysqld) Shutdown(ctx context.Context, waitForMysqld bool) error

Shutdown will stop the mysqld daemon that is running in the background.

waitForMysqld: should the function block until mysqld has stopped? This can actually take a *long* time if the buffer cache needs to be fully flushed - on the order of 20-30 minutes.

If a mysqlctld address is provided in a flag, Shutdown will run remotely.

func (*Mysqld) SlaveStatus

func (mysqld *Mysqld) SlaveStatus() (Status, error)

SlaveStatus returns the slave replication statuses

func (*Mysqld) Start

func (mysqld *Mysqld) Start(ctx context.Context, mysqldArgs ...string) error

Start will start the mysql daemon, either by running the 'mysqld_start' hook, or by running mysqld_safe in the background. If a mysqlctld address is provided in a flag, Start will run remotely. When waiting for mysqld to start, we will use the dba user.

func (*Mysqld) TabletDir

func (mysqld *Mysqld) TabletDir() string

TabletDir returns the main tablet directory. It's a method so it can be accessed through the MysqlDaemon interface.

func (*Mysqld) Teardown

func (mysqld *Mysqld) Teardown(ctx context.Context, force bool) error

Teardown will shutdown the running daemon, and delete the root directory.

func (*Mysqld) Wait

func (mysqld *Mysqld) Wait(ctx context.Context) error

Wait returns nil when mysqld is up and accepting connections. It will use the dba credentials to try to connect. Use wait() with different credentials if needed.

func (*Mysqld) WaitForReparentJournal

func (mysqld *Mysqld) WaitForReparentJournal(ctx context.Context, timeCreatedNS int64) error

WaitForReparentJournal will wait until the context is done for the row in the reparent_journal table.

func (*Mysqld) WaitMasterPos

func (mysqld *Mysqld) WaitMasterPos(ctx context.Context, targetPos mysql.Position) error

WaitMasterPos lets slaves wait to given replication position

type SlaveConnection

type SlaveConnection struct {
	*mysql.Conn
	// contains filtered or unexported fields
}

SlaveConnection represents a connection to mysqld that pretends to be a slave connecting for replication. Each such connection must identify itself to mysqld with a server ID that is unique both among other SlaveConnections and among actual slaves in the topology.

func (*SlaveConnection) Close

func (sc *SlaveConnection) Close()

Close closes the slave connection, which also signals an ongoing dump started with StartBinlogDump() to stop and close its BinlogEvent channel. The ID for the slave connection is recycled back into the pool.

func (*SlaveConnection) StartBinlogDumpFromBinlogBeforeTimestamp

func (sc *SlaveConnection) StartBinlogDumpFromBinlogBeforeTimestamp(ctx context.Context, timestamp int64) (<-chan mysql.BinlogEvent, error)

StartBinlogDumpFromBinlogBeforeTimestamp requests a replication binlog dump from the master mysqld starting with a file that has timestamps smaller than the provided timestamp, and then sends binlog events to the provided channel.

The startup phase will list all the binary logs, and find the one that has events starting strictly before the provided timestamp. It will then start from there, and stream all events. It is the responsability of the calling site to filter the events more.

MySQL 5.6+ note: we need to do it that way because of the way the GTIDSet works. In the previous two streaming functions, we pass in the full GTIDSet (that has the list of all transactions seen in the replication stream). In this case, we don't know it, all we have is the binlog file names. We depend on parsing the first PREVIOUS_GTIDS_EVENT event in the logs to get it. So we need the caller to parse that event, and it can't be skipped because its timestamp is lower. Then, for each subsequent event, the caller also needs to add the event GTID to its GTIDSet. Otherwise it won't be correct ever. So the caller really needs to build up its GTIDSet along the entire file, not just for events whose timestamp is in a given range.

The stream will continue in the background, waiting for new events if necessary, until the connection is closed, either by the master or by canceling the context.

Note the context is valid and used until eventChan is closed.

func (*SlaveConnection) StartBinlogDumpFromCurrent

func (sc *SlaveConnection) StartBinlogDumpFromCurrent(ctx context.Context) (mysql.Position, <-chan mysql.BinlogEvent, error)

StartBinlogDumpFromCurrent requests a replication binlog dump from the current position.

func (*SlaveConnection) StartBinlogDumpFromPosition

func (sc *SlaveConnection) StartBinlogDumpFromPosition(ctx context.Context, startPos mysql.Position) (<-chan mysql.BinlogEvent, error)

StartBinlogDumpFromPosition requests a replication binlog dump from the master mysqld at the given Position and then sends binlog events to the provided channel. The stream will continue in the background, waiting for new events if necessary, until the connection is closed, either by the master or by canceling the context.

Note the context is valid and used until eventChan is closed.

type Status

type Status struct {
	Position            mysql.Position
	SlaveIORunning      bool
	SlaveSQLRunning     bool
	SecondsBehindMaster uint
	MasterHost          string
	MasterPort          int
	MasterConnectRetry  int
}

Status holds replication information from SHOW SLAVE STATUS.

func NewStatus

func NewStatus(masterAddr string) (*Status, error)

NewStatus creates a Status pointing to masterAddr.

func ProtoToStatus

func ProtoToStatus(r *replicationdatapb.Status) Status

ProtoToStatus translates a proto Status, or panics

func (*Status) MasterAddr

func (rs *Status) MasterAddr() string

MasterAddr returns the host:port address of the master.

func (*Status) SlaveRunning

func (rs *Status) SlaveRunning() bool

SlaveRunning returns true iff both the Slave IO and Slave SQL threads are running.

Directories

Path Synopsis
Package backupstorage contains the interface and file system implementation of the backup system.
Package backupstorage contains the interface and file system implementation of the backup system.
Package cephbackupstorage implements the BackupStorage interface for Ceph Cloud Storage.
Package cephbackupstorage implements the BackupStorage interface for Ceph Cloud Storage.
Package filebackupstorage implements the BackupStorage interface for a local filesystem (which can be an NFS mount).
Package filebackupstorage implements the BackupStorage interface for a local filesystem (which can be an NFS mount).
Package gcsbackupstorage implements the BackupStorage interface for Google Cloud Storage.
Package gcsbackupstorage implements the BackupStorage interface for Google Cloud Storage.
Package grpcmysqlctlclient contains the gRPC1 version of the mysqlctl client protocol.
Package grpcmysqlctlclient contains the gRPC1 version of the mysqlctl client protocol.
Package grpcmysqlctlserver contains the gRPC implementation of the server side of the remote execution of mysqlctl commands.
Package grpcmysqlctlserver contains the gRPC implementation of the server side of the remote execution of mysqlctl commands.
Package mysqlctlclient contains the generic client side of the remote mysqlctl protocol.
Package mysqlctlclient contains the generic client side of the remote mysqlctl protocol.
Package s3backupstorage implements the BackupStorage interface for AWS S3.
Package s3backupstorage implements the BackupStorage interface for AWS S3.
Package tmutils contains helper methods to deal with the tabletmanagerdata proto3 structures.
Package tmutils contains helper methods to deal with the tabletmanagerdata proto3 structures.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL