tabletserver

package
v2.0.0-rc.1+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 7, 2016 License: BSD-3-Clause Imports: 58 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// QuerySourceConsolidator means query result is found in consolidator.
	QuerySourceConsolidator = 1 << iota
	// QuerySourceMySQL means query result is returned from MySQL.
	QuerySourceMySQL
)
View Source
const (
	QRContinue = Action(iota)
	QRFail
	QRFailRetry
)

These are actions.

View Source
const (
	QRNoOp = Operator(iota)
	QREqual
	QRNotEqual
	QRLessThan
	QRGreaterEqual
	QRGreaterThan
	QRLessEqual
	QRMatch
	QRNoMatch
	QRIn
	QRNotIn
	QRNumOp
)

These are comparison operators.

View Source
const (
	QROK = iota
	QRMismatch
	QROutOfRange
)

These are return statii.

View Source
const (
	// StateNotConnected is the state where tabletserver is not
	// connected to an underlying mysql instance.
	StateNotConnected = iota
	// StateNotServing is the state where tabletserver is connected
	// to an underlying mysql instance, but is not serving queries.
	StateNotServing
	// StateServing is where queries are allowed.
	StateServing
	// StateTransitioning is a transient state indicating that
	// the tabletserver is tranisitioning to a new state.
	StateTransitioning
	// StateShuttingDown is a transient state indicating that
	// the tabletserver is shutting down. This state differs from
	// StateTransitioning because we allow queries for transactions
	// that are still in flight.
	StateShuttingDown
)
View Source
const (
	TxClose    = "close"
	TxCommit   = "commit"
	TxRollback = "rollback"
	TxKill     = "kill"
)

These consts identify how a transaction was resolved.

Variables

View Source
var DefaultQsConfig = Config{
	PoolSize:             16,
	StreamPoolSize:       750,
	TransactionCap:       20,
	TransactionTimeout:   30,
	MaxResultSize:        10000,
	MaxDMLRows:           500,
	QueryCacheSize:       5000,
	SchemaReloadTime:     30 * 60,
	QueryTimeout:         0,
	TxPoolTimeout:        1,
	IdleTimeout:          30 * 60,
	StreamBufferSize:     32 * 1024,
	StrictMode:           true,
	StrictTableAcl:       false,
	TerseErrors:          false,
	EnablePublishStats:   true,
	EnableAutoCommit:     false,
	EnableTableAclDryRun: false,
	StatsPrefix:          "",
	DebugURLPrefix:       "/debug",
	PoolNamePrefix:       "",
	TableAclExemptACL:    "",
}

DefaultQsConfig is the default value for the query service config. The value for StreamBufferSize was chosen after trying out a few of them. Too small buffers force too many packets to be sent. Too big buffers force the clients to read them in multiple chunks and make memory copies. so with the encoding overhead, this seems to work great (the overhead makes the final packets on the wire about twice bigger than this).

View Source
var ErrConnPoolClosed = NewTabletError(

	vtrpcpb.ErrorCode_INTERNAL_ERROR,
	"connection pool is closed")

ErrConnPoolClosed is returned / panicked when the connection pool is closed.

View Source
var RegisterFunctions []RegisterFunction

RegisterFunctions is a list of all the RegisterFunction that will be called upon Register() on a TabletServer

View Source
var StatsLogger = streamlog.New("TabletServer", 50)

StatsLogger is the main stream logger object

View Source
var TxLogger = streamlog.New("TxLog", 10)

TxLogger can be used to enable logging of transactions. Call TxLogger.ServeLogs in your main program to enable logging. The log format can be inferred by looking at TxConnection.Format.

Functions

func Init

func Init()

Init must be called after flag.Parse, and before doing any other operations.

func IsConnErr

func IsConnErr(err error) bool

IsConnErr returns true if the error is a connection error. If the error is of type TabletError or hasNumber, it checks the error code. Otherwise, it parses the string looking for (errno xxxx) and uses the extracted value to determine if it's a conn error.

func PrefixTabletError

func PrefixTabletError(errCode vtrpcpb.ErrorCode, err error, prefix string) error

PrefixTabletError attempts to add a string prefix to a TabletError, while preserving its ErrorCode. If the given error is not a TabletError, a new TabletError is returned with the desired ErrorCode.

func Rand

func Rand() int64

Rand generates a pseudo-random int64 number.

func ToGRPCError

func ToGRPCError(err error) error

ToGRPCError returns a TabletError as a grpc error, with the appropriate error code. This function lives here, instead of in vterrors, so that the vterrors package doesn't have to import tabletserver.

func ToRPCError

func ToRPCError(err error) *vtrpcpb.RPCError

ToRPCError returns a vtrpcpb.RPCError (that can be packed across RPC boundaries) for the TabletError.

Types

type Action

type Action int

Action speficies the list of actions to perform when a QueryRule is triggered.

func (Action) MarshalJSON

func (act Action) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

type BindVarCond

type BindVarCond struct {
	// contains filtered or unexported fields
}

BindVarCond represents a bind var condition.

func (BindVarCond) MarshalJSON

func (bvc BindVarCond) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

type Config

type Config struct {
	PoolSize             int
	StreamPoolSize       int
	TransactionCap       int
	TransactionTimeout   float64
	MaxResultSize        int
	MaxDMLRows           int
	StreamBufferSize     int
	QueryCacheSize       int
	SchemaReloadTime     float64
	QueryTimeout         float64
	TxPoolTimeout        float64
	IdleTimeout          float64
	StrictMode           bool
	StrictTableAcl       bool
	TerseErrors          bool
	EnablePublishStats   bool
	EnableAutoCommit     bool
	EnableTableAclDryRun bool
	StatsPrefix          string
	DebugURLPrefix       string
	PoolNamePrefix       string
	TableAclExemptACL    string
}

Config contains all the configuration for query service

type ConnPool

type ConnPool struct {
	// contains filtered or unexported fields
}

ConnPool implements a custom connection pool for tabletserver. It's similar to dbconnpool.ConnPool, but the connections it creates come with built-in ability to kill in-flight queries. These connections also trigger a CheckMySQL call if we fail to connect to MySQL. Other than the connection type, ConnPool maintains an additional pool of dba connections that are used to kill connections.

func NewConnPool

func NewConnPool(
	name string,
	capacity int,
	idleTimeout time.Duration,
	enablePublishStats bool,
	queryServiceStats *QueryServiceStats,
	checker MySQLChecker) *ConnPool

NewConnPool creates a new ConnPool. The name is used to publish stats only.

func (*ConnPool) Available

func (cp *ConnPool) Available() int64

Available returns the number of available connections in the pool

func (*ConnPool) Capacity

func (cp *ConnPool) Capacity() int64

Capacity returns the pool capacity.

func (*ConnPool) Close

func (cp *ConnPool) Close()

Close will close the pool and wait for connections to be returned before exiting.

func (*ConnPool) Get

func (cp *ConnPool) Get(ctx context.Context) (*DBConn, error)

Get returns a connection. You must call Recycle on DBConn once done.

func (*ConnPool) IdleTimeout

func (cp *ConnPool) IdleTimeout() time.Duration

IdleTimeout returns the idle timeout for the pool.

func (*ConnPool) MaxCap

func (cp *ConnPool) MaxCap() int64

MaxCap returns the maximum size of the pool

func (*ConnPool) Open

func (cp *ConnPool) Open(appParams, dbaParams *sqldb.ConnParams)

Open must be called before starting to use the pool.

func (*ConnPool) Put

func (cp *ConnPool) Put(conn *DBConn)

Put puts a connection into the pool.

func (*ConnPool) SetCapacity

func (cp *ConnPool) SetCapacity(capacity int) (err error)

SetCapacity alters the size of the pool at runtime.

func (*ConnPool) SetIdleTimeout

func (cp *ConnPool) SetIdleTimeout(idleTimeout time.Duration)

SetIdleTimeout sets the idleTimeout on the pool.

func (*ConnPool) StatsJSON

func (cp *ConnPool) StatsJSON() string

StatsJSON returns the pool stats as a JSOn object.

func (*ConnPool) WaitCount

func (cp *ConnPool) WaitCount() int64

WaitCount returns how many clients are waiting for a connection

func (*ConnPool) WaitTime

func (cp *ConnPool) WaitTime() time.Duration

WaitTime return the pool WaitTime.

type Controller

type Controller interface {
	// Register registers this query service with the RPC layer.
	Register()

	// AddStatusPart adds the status part to the status page
	AddStatusPart()

	// InitDBConfig sets up the db config vars.
	InitDBConfig(querypb.Target, dbconfigs.DBConfigs, mysqlctl.MysqlDaemon) error

	// SetServingType transitions the query service to the required serving type.
	// Returns true if the state of QueryService or the tablet type changed.
	SetServingType(tabletType topodatapb.TabletType, serving bool, alsoAllow []topodatapb.TabletType) (bool, error)

	// EnterLameduck causes tabletserver to enter the lameduck state.
	EnterLameduck()

	// IsServing returns true if the query service is running
	IsServing() bool

	// IsHealthy returns the health status of the QueryService
	IsHealthy() error

	// ClearQueryPlanCache clears internal query plan cache
	ClearQueryPlanCache()

	// ReloadSchema makes the quey service reload its schema cache
	ReloadSchema()

	// RegisterQueryRuleSource adds a query rule source
	RegisterQueryRuleSource(ruleSource string)

	// RegisterQueryRuleSource removes a query rule source
	UnRegisterQueryRuleSource(ruleSource string)

	// SetQueryRules sets the query rules for this QueryService
	SetQueryRules(ruleSource string, qrs *QueryRules) error

	// QueryService returns the QueryService object used by this Controller
	QueryService() queryservice.QueryService

	QueryServiceStats() *QueryServiceStats

	// BroadcastHealth sends the current health to all listeners
	BroadcastHealth(terTimestamp int64, stats *querypb.RealtimeStats)
}

Controller defines the control interface for TabletServer.

type DBConn

type DBConn struct {
	// contains filtered or unexported fields
}

DBConn is a db connection for tabletserver. It performs automatic reconnects as needed. Its Execute function has a timeout that can kill its own queries and the underlying connection. It will also trigger a CheckMySQL whenever applicable.

func NewDBConn

func NewDBConn(
	cp *ConnPool,
	appParams,
	dbaParams *sqldb.ConnParams,
	qStats *QueryServiceStats) (*DBConn, error)

NewDBConn creates a new DBConn. It triggers a CheckMySQL if creation fails.

func (*DBConn) Close

func (dbc *DBConn) Close()

Close closes the DBConn.

func (*DBConn) Current

func (dbc *DBConn) Current() string

Current returns the currently executing query.

func (*DBConn) Exec

func (dbc *DBConn) Exec(ctx context.Context, query string, maxrows int, wantfields bool) (*sqltypes.Result, error)

Exec executes the specified query. If there is a connection error, it will reconnect and retry. A failed reconnect will trigger a CheckMySQL.

func (*DBConn) ExecOnce

func (dbc *DBConn) ExecOnce(ctx context.Context, query string, maxrows int, wantfields bool) (*sqltypes.Result, error)

ExecOnce executes the specified query, but does not retry on connection errors.

func (*DBConn) ID

func (dbc *DBConn) ID() int64

ID returns the connection id.

func (*DBConn) IsClosed

func (dbc *DBConn) IsClosed() bool

IsClosed returns true if DBConn is closed.

func (*DBConn) Kill

func (dbc *DBConn) Kill(reason string) error

Kill kills the currently executing query both on MySQL side and on the connection side. If no query is executing, it's a no-op. Kill will also not kill a query more than once.

func (*DBConn) Recycle

func (dbc *DBConn) Recycle()

Recycle returns the DBConn to the pool.

func (*DBConn) Stream

func (dbc *DBConn) Stream(ctx context.Context, query string, callback func(*sqltypes.Result) error, streamBufferSize int) error

Stream executes the query and streams the results.

func (*DBConn) VerifyStrict

func (dbc *DBConn) VerifyStrict() bool

VerifyStrict returns true if MySQL is in STRICT mode.

type ExecPlan

type ExecPlan struct {
	*planbuilder.ExecPlan
	TableInfo  *TableInfo
	Fields     []*querypb.Field
	Rules      *QueryRules
	Authorized *tableacl.ACLResult

	QueryCount int64
	Time       time.Duration
	RowCount   int64
	ErrorCount int64
	// contains filtered or unexported fields
}

ExecPlan wraps the planbuilder's exec plan to enforce additional rules and track stats.

func (*ExecPlan) AddStats

func (ep *ExecPlan) AddStats(queryCount int64, duration time.Duration, rowCount, errorCount int64)

AddStats updates the stats for the current ExecPlan.

func (*ExecPlan) Size

func (*ExecPlan) Size() int

Size allows ExecPlan to be in cache.LRUCache.

func (*ExecPlan) Stats

func (ep *ExecPlan) Stats() (queryCount int64, duration time.Duration, rowCount, errorCount int64)

Stats returns the current stats of ExecPlan.

type LogStats

type LogStats struct {
	Method        string
	PlanType      string
	OriginalSQL   string
	BindVariables map[string]interface{}

	RowsAffected         int
	NumberOfQueries      int
	StartTime            time.Time
	EndTime              time.Time
	MysqlResponseTime    time.Duration
	WaitingForConnection time.Duration
	QuerySources         byte
	Rows                 [][]sqltypes.Value
	TransactionID        int64

	Error *TabletError
	// contains filtered or unexported fields
}

LogStats records the stats for a single query

func (*LogStats) AddRewrittenSQL

func (stats *LogStats) AddRewrittenSQL(sql string, start time.Time)

AddRewrittenSQL adds a single sql statement to the rewritten list

func (*LogStats) ContextHTML

func (stats *LogStats) ContextHTML() template.HTML

ContextHTML returns the HTML version of the context that was used, or "". This is a method on LogStats instead of a field so that it doesn't need to be passed by value everywhere.

func (*LogStats) EffectiveCaller

func (stats *LogStats) EffectiveCaller() string

EffectiveCaller returns the effective caller stored in LogStats.ctx

func (*LogStats) ErrorStr

func (stats *LogStats) ErrorStr() string

ErrorStr returns the error string or ""

func (*LogStats) EventTime

func (stats *LogStats) EventTime() time.Time

EventTime returns the time the event was created.

func (*LogStats) FmtBindVariables

func (stats *LogStats) FmtBindVariables(full bool) string

FmtBindVariables returns the map of bind variables as JSON. For values that are strings or byte slices it only reports their type and length.

func (*LogStats) FmtQuerySources

func (stats *LogStats) FmtQuerySources() string

FmtQuerySources returns a comma separated list of query sources. If there were no query sources, it returns the string "none".

func (*LogStats) Format

func (stats *LogStats) Format(params url.Values) string

Format returns a tab separated list of logged fields.

func (*LogStats) ImmediateCaller

func (stats *LogStats) ImmediateCaller() string

ImmediateCaller returns the immediate caller stored in LogStats.ctx

func (*LogStats) RemoteAddrUsername

func (stats *LogStats) RemoteAddrUsername() (string, string)

RemoteAddrUsername returns some parts of CallInfo if set

func (*LogStats) RewrittenSQL

func (stats *LogStats) RewrittenSQL() string

RewrittenSQL returns a semicolon separated list of SQL statements that were executed.

func (*LogStats) Send

func (stats *LogStats) Send()

Send finalizes a record and sends it

func (*LogStats) SizeOfResponse

func (stats *LogStats) SizeOfResponse() int

SizeOfResponse returns the approximate size of the response in bytes (this does not take in account protocol encoding). It will return 0 for streaming requests.

func (*LogStats) TotalTime

func (stats *LogStats) TotalTime() time.Duration

TotalTime returns how long this query has been running

type MySQLChecker

type MySQLChecker interface {
	CheckMySQL()
}

MySQLChecker defines the CheckMySQL interface that lower level objects can use to call back into TabletServer.

type Operator

type Operator int

Operator represents the list of operators.

func MapStrOperator

func MapStrOperator(strop string) (op Operator, err error)

MapStrOperator maps a string representation to an Operator.

func (Operator) MarshalJSON

func (op Operator) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

type QueryDetail

type QueryDetail struct {
	// contains filtered or unexported fields
}

QueryDetail is a simple wrapper for Query, Context and a killable conn.

func NewQueryDetail

func NewQueryDetail(ctx context.Context, conn killable) *QueryDetail

NewQueryDetail creates a new QueryDetail

type QueryDetailzRow

type QueryDetailzRow struct {
	Query             string
	ContextHTML       template.HTML
	Start             time.Time
	Duration          time.Duration
	ConnID            int64
	State             string
	ShowTerminateLink bool
}

QueryDetailzRow is used for rendering QueryDetail in a template

type QueryEngine

type QueryEngine struct {
	// contains filtered or unexported fields
}

QueryEngine implements the core functionality of tabletserver. It assumes that no requests will be sent to it before Open is called and succeeds. Shutdown is done in the following order:

WaitForTxEmpty: There should be no more new calls to Begin once this function is called. This will return when there are no more pending transactions.

Close: There should be no more pending queries when this function is called.

Functions of QueryEngine do not return errors. They instead panic with NewTabletError as the error type. TODO(sougou): Switch to error return scheme.

func NewQueryEngine

func NewQueryEngine(checker MySQLChecker, config Config) *QueryEngine

NewQueryEngine creates a new QueryEngine. This is a singleton class. You must call this only once.

func (*QueryEngine) Close

func (qe *QueryEngine) Close()

Close must be called to shut down QueryEngine. You must ensure that no more queries will be sent before calling Close.

func (*QueryEngine) IsMySQLReachable

func (qe *QueryEngine) IsMySQLReachable() bool

IsMySQLReachable returns true if we can connect to MySQL.

func (*QueryEngine) Open

func (qe *QueryEngine) Open(dbconfigs dbconfigs.DBConfigs)

Open must be called before sending requests to QueryEngine.

func (*QueryEngine) WaitForTxEmpty

func (qe *QueryEngine) WaitForTxEmpty()

WaitForTxEmpty must be called before calling Close. Before calling WaitForTxEmpty, you must ensure that there will be no more calls to Begin.

type QueryExecutor

type QueryExecutor struct {
	// contains filtered or unexported fields
}

QueryExecutor is used for executing a query request.

func (*QueryExecutor) Execute

func (qre *QueryExecutor) Execute() (reply *sqltypes.Result, err error)

Execute performs a non-streaming query execution.

func (*QueryExecutor) Stream

func (qre *QueryExecutor) Stream(sendReply func(*sqltypes.Result) error) error

Stream performs a streaming query execution.

type QueryList

type QueryList struct {
	// contains filtered or unexported fields
}

QueryList holds a thread safe list of QueryDetails

func NewQueryList

func NewQueryList() *QueryList

NewQueryList creates a new QueryList

func (*QueryList) Add

func (ql *QueryList) Add(qd *QueryDetail)

Add adds a QueryDetail to QueryList

func (*QueryList) GetQueryzRows

func (ql *QueryList) GetQueryzRows() []QueryDetailzRow

GetQueryzRows returns a list of QueryDetailzRow sorted by start time

func (*QueryList) Remove

func (ql *QueryList) Remove(qd *QueryDetail)

Remove removes a QueryDetail from QueryList

func (*QueryList) Terminate

func (ql *QueryList) Terminate(connID int64) error

Terminate updates the query status and kills the connection

func (*QueryList) TerminateAll

func (ql *QueryList) TerminateAll()

TerminateAll terminates all queries and kills the MySQL connections

type QueryRule

type QueryRule struct {
	Description string
	Name        string
	// contains filtered or unexported fields
}

QueryRule represents one rule (conditions-action). Name is meant to uniquely identify a rule. Description is a human readable comment that describes the rule. For a QueryRule to fire, all conditions of the QueryRule have to match. For example, an empty QueryRule will match all requests. Every QueryRule has an associated Action. If all the conditions of the QueryRule are met, then the Action is triggerred.

func BuildQueryRule

func BuildQueryRule(ruleInfo map[string]interface{}) (qr *QueryRule, err error)

BuildQueryRule builds a query rule from a ruleInfo.

func NewQueryRule

func NewQueryRule(description, name string, act Action) (qr *QueryRule)

NewQueryRule creates a new QueryRule.

func (*QueryRule) AddBindVarCond

func (qr *QueryRule) AddBindVarCond(name string, onAbsent, onMismatch bool, op Operator, value interface{}) error

AddBindVarCond adds a bind variable restriction to the QueryRule. All bind var conditions have to be satisfied for the QueryRule to be a match. name represents the name (not regexp) of the bind variable. onAbsent specifies the value of the condition if the bind variable is absent. onMismatch specifies the value of the condition if there's a type mismatch on the condition. For inequalities, the bindvar is the left operand and the value in the condition is the right operand: bindVar Operator value. Value & operator rules Type Operators Bindvar nil "" any type uint64 ==, !=, <, >=, >, <= whole numbers int64 ==, !=, <, >=, >, <= whole numbers string ==, !=, <, >=, >, <=, MATCH, NOMATCH []byte, string KeyRange IN, NOTIN whole numbers whole numbers can be: int, int8, int16, int32, int64, uint64

func (*QueryRule) AddPlanCond

func (qr *QueryRule) AddPlanCond(planType planbuilder.PlanType)

AddPlanCond adds to the list of plans that can be matched for the rule to fire. This function acts as an OR: Any plan id match is considered a match.

func (*QueryRule) AddTableCond

func (qr *QueryRule) AddTableCond(tableName string)

AddTableCond adds to the list of tableNames that can be matched for the rule to fire. This function acts as an OR: Any tableName match is considered a match.

func (*QueryRule) Copy

func (qr *QueryRule) Copy() (newqr *QueryRule)

Copy performs a deep copy of a QueryRule.

func (*QueryRule) MarshalJSON

func (qr *QueryRule) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

func (*QueryRule) SetIPCond

func (qr *QueryRule) SetIPCond(pattern string) (err error)

SetIPCond adds a regular expression condition for the client IP. It has to be a full match (not substring).

func (*QueryRule) SetQueryCond

func (qr *QueryRule) SetQueryCond(pattern string) (err error)

SetQueryCond adds a regular expression condition for the query.

func (*QueryRule) SetUserCond

func (qr *QueryRule) SetUserCond(pattern string) (err error)

SetUserCond adds a regular expression condition for the user name used by the client.

type QueryRuleInfo

type QueryRuleInfo struct {
	// contains filtered or unexported fields
}

QueryRuleInfo is the maintainer of QueryRules from multiple sources

func NewQueryRuleInfo

func NewQueryRuleInfo() *QueryRuleInfo

NewQueryRuleInfo returns an empty QueryRuleInfo object for use

func (*QueryRuleInfo) GetRules

func (qri *QueryRuleInfo) GetRules(ruleSource string) (*QueryRules, error)

GetRules returns the corresponding QueryRules as designated by ruleSource parameter

func (*QueryRuleInfo) MarshalJSON

func (qri *QueryRuleInfo) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

func (*QueryRuleInfo) RegisterQueryRuleSource

func (qri *QueryRuleInfo) RegisterQueryRuleSource(ruleSource string)

RegisterQueryRuleSource registers a query rule source name with QueryRuleInfo

func (*QueryRuleInfo) SetRules

func (qri *QueryRuleInfo) SetRules(ruleSource string, newRules *QueryRules) error

SetRules takes an external QueryRules structure and overwrite one of the internal QueryRules as designated by ruleSource parameter

func (*QueryRuleInfo) UnRegisterQueryRuleSource

func (qri *QueryRuleInfo) UnRegisterQueryRuleSource(ruleSource string)

UnRegisterQueryRuleSource removes a registered query rule source name

type QueryRules

type QueryRules struct {
	// contains filtered or unexported fields
}

QueryRules is used to store and execute rules for the tabletserver.

func NewQueryRules

func NewQueryRules() *QueryRules

NewQueryRules creates a new QueryRules.

func (*QueryRules) Add

func (qrs *QueryRules) Add(qr *QueryRule)

Add adds a QueryRule to QueryRules. It does not check for duplicates.

func (*QueryRules) Append

func (qrs *QueryRules) Append(otherqrs *QueryRules)

Append merges the rules from another QueryRules into the receiver

func (*QueryRules) Copy

func (qrs *QueryRules) Copy() (newqrs *QueryRules)

Copy performs a deep copy of QueryRules. A nil input produces a nil output.

func (*QueryRules) Delete

func (qrs *QueryRules) Delete(name string) (qr *QueryRule)

Delete deletes a QueryRule by name and returns the rule that was deleted. It returns nil if the rule was not found.

func (*QueryRules) Find

func (qrs *QueryRules) Find(name string) (qr *QueryRule)

Find finds the first occurrence of a QueryRule by matching the Name field. It returns nil if the rule was not found.

func (*QueryRules) MarshalJSON

func (qrs *QueryRules) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

func (*QueryRules) UnmarshalJSON

func (qrs *QueryRules) UnmarshalJSON(data []byte) (err error)

UnmarshalJSON unmarshals QueryRules.

type QueryServiceStats

type QueryServiceStats struct {
	// MySQLStats shows the time histogram for operations spent on mysql side.
	MySQLStats *stats.Timings
	// QueryStats shows the time histogram for each type of queries.
	QueryStats *stats.Timings
	// WaitStats shows the time histogram for wait operations
	WaitStats *stats.Timings
	// KillStats shows number of connections being killed.
	KillStats *stats.Counters
	// InfoErrors shows number of various non critical errors happened.
	InfoErrors *stats.Counters
	// ErrorStats shows number of critial erros happened.
	ErrorStats *stats.Counters
	// InternalErros shows number of errors from internal components.
	InternalErrors *stats.Counters
	// UserTableQueryCount shows number of queries received for each CallerID/table combination.
	UserTableQueryCount *stats.MultiCounters
	// UserTableQueryTimesNs shows total latency for each CallerID/table combination.
	UserTableQueryTimesNs *stats.MultiCounters
	// UserTransactionCount shows number of transactions received for each CallerID.
	UserTransactionCount *stats.MultiCounters
	// UserTransactionTimesNs shows total transaction latency for each CallerID.
	UserTransactionTimesNs *stats.MultiCounters
	// QPSRates shows the qps.
	QPSRates *stats.Rates
	// ResultStats shows the histogram of number of rows returned.
	ResultStats *stats.Histogram
}

QueryServiceStats contains stats that used in queryservice level.

func NewQueryServiceStats

func NewQueryServiceStats(statsPrefix string, enablePublishStats bool) *QueryServiceStats

NewQueryServiceStats returns a new QueryServiceStats instance.

type QuerySplitter

type QuerySplitter struct {
	// contains filtered or unexported fields
}

QuerySplitter splits a BoundQuery into equally sized smaller queries. QuerySplits are generated by adding primary key range clauses to the original query. Only a limited set of queries are supported, see QuerySplitter.validateQuery() for details. Also, the table must have at least one primary key and the leading primary key must be numeric, see QuerySplitter.splitBoundaries()

func NewQuerySplitter

func NewQuerySplitter(
	sql string,
	bindVariables map[string]interface{},
	splitColumn string,
	splitCount int64,
	schemaInfo *SchemaInfo) *QuerySplitter

NewQuerySplitter creates a new QuerySplitter. query is the original query to split and splitCount is the desired number of splits. splitCount must be a positive int, if not it will be set to 1.

type RegisterFunction

type RegisterFunction func(Controller)

RegisterFunction is a callback type to be called when we Register() a TabletServer

type SchemaInfo

type SchemaInfo struct {
	// contains filtered or unexported fields
}

SchemaInfo stores the schema info and performs operations that keep itself up-to-date.

func NewSchemaInfo

func NewSchemaInfo(
	statsPrefix string,
	checker MySQLChecker,
	queryCacheSize int,
	reloadTime time.Duration,
	idleTimeout time.Duration,
	endpoints map[string]string,
	enablePublishStats bool,
	queryServiceStats *QueryServiceStats) *SchemaInfo

NewSchemaInfo creates a new SchemaInfo.

func (*SchemaInfo) ClearQueryPlanCache

func (si *SchemaInfo) ClearQueryPlanCache()

ClearQueryPlanCache should be called if query plan cache is potentially obsolete

func (*SchemaInfo) Close

func (si *SchemaInfo) Close()

Close shuts down SchemaInfo. It can be re-opened after Close.

func (*SchemaInfo) CreateOrUpdateTable

func (si *SchemaInfo) CreateOrUpdateTable(ctx context.Context, tableName string)

CreateOrUpdateTable must be called if a DDL was applied to that table.

func (*SchemaInfo) DropTable

func (si *SchemaInfo) DropTable(tableName string)

DropTable must be called if a table was dropped.

func (*SchemaInfo) GetPlan

func (si *SchemaInfo) GetPlan(ctx context.Context, logStats *LogStats, sql string) *ExecPlan

GetPlan returns the ExecPlan that for the query. Plans are cached in a cache.LRUCache.

func (*SchemaInfo) GetSchema

func (si *SchemaInfo) GetSchema() []*schema.Table

GetSchema returns a copy of the schema.

func (*SchemaInfo) GetStreamPlan

func (si *SchemaInfo) GetStreamPlan(sql string) *ExecPlan

GetStreamPlan is similar to GetPlan, but doesn't use the cache and doesn't enforce a limit. It just returns the parsed query.

func (*SchemaInfo) GetTable

func (si *SchemaInfo) GetTable(tableName string) *TableInfo

GetTable returns the TableInfo for a table.

func (*SchemaInfo) Open

func (si *SchemaInfo) Open(dbaParams *sqldb.ConnParams, strictMode bool)

Open initializes the current SchemaInfo for service by loading the necessary info from the specified database.

func (*SchemaInfo) QueryCacheCap

func (si *SchemaInfo) QueryCacheCap() int

QueryCacheCap returns the capacity of the query cache.

func (*SchemaInfo) Reload

func (si *SchemaInfo) Reload()

Reload reloads the schema info from the db. Any tables that have changed since the last load are updated.

func (*SchemaInfo) ReloadTime

func (si *SchemaInfo) ReloadTime() time.Duration

ReloadTime returns schema info reload time.

func (*SchemaInfo) ServeHTTP

func (si *SchemaInfo) ServeHTTP(response http.ResponseWriter, request *http.Request)

func (*SchemaInfo) SetQueryCacheCap

func (si *SchemaInfo) SetQueryCacheCap(size int)

SetQueryCacheCap sets the query cache capacity.

func (*SchemaInfo) SetReloadTime

func (si *SchemaInfo) SetReloadTime(reloadTime time.Duration)

SetReloadTime changes how often the schema is reloaded. This call also triggers an immediate reload.

type TableInfo

type TableInfo struct {
	*schema.Table

	// Seq must be locked before accessing the sequence vars.
	// If CurVal==LastVal, we have to cache new values.
	Seq       sync.Mutex
	NextVal   int64
	Increment int64
	LastVal   int64
}

TableInfo contains the tabletserver related info for a table. It's a superset of schema.Table.

func NewTableInfo

func NewTableInfo(conn *DBConn, tableName string, tableType string, comment string) (ti *TableInfo, err error)

NewTableInfo creates a new TableInfo.

func (*TableInfo) SetPK

func (ti *TableInfo) SetPK(colnames []string) error

SetPK sets the pk columns for a TableInfo.

type TabletError

type TabletError struct {
	Message  string
	SQLError int
	SQLState string
	// ErrorCode will be used to transmit the error across RPC boundaries
	ErrorCode vtrpcpb.ErrorCode
}

TabletError is the error type we use in this library

func NewTabletError

func NewTabletError(errCode vtrpcpb.ErrorCode, format string, args ...interface{}) *TabletError

NewTabletError returns a TabletError of the given type

func NewTabletErrorSQL

func NewTabletErrorSQL(errCode vtrpcpb.ErrorCode, err error) *TabletError

NewTabletErrorSQL returns a TabletError based on the error

func (*TabletError) Error

func (te *TabletError) Error() string

func (*TabletError) Prefix

func (te *TabletError) Prefix() string

Prefix returns the prefix for the error, like error, fatal, etc.

func (*TabletError) RecordStats

func (te *TabletError) RecordStats(queryServiceStats *QueryServiceStats)

RecordStats will record the error in the proper stat bucket

func (*TabletError) VtErrorCode

func (te *TabletError) VtErrorCode() vtrpcpb.ErrorCode

VtErrorCode returns the underlying Vitess error code

type TabletServer

type TabletServer struct {
	QueryTimeout sync2.AtomicDuration
	BeginTimeout sync2.AtomicDuration
	// contains filtered or unexported fields
}

TabletServer implements the RPC interface for the query service.

func NewServer

func NewServer() *TabletServer

NewServer creates a new TabletServer based on the command line flags.

func NewTabletServer

func NewTabletServer(config Config) *TabletServer

NewTabletServer creates an instance of TabletServer. Only one instance of TabletServer can be created per process.

func (*TabletServer) AddStatusPart

func (tsv *TabletServer) AddStatusPart()

AddStatusPart registers the status part for the status page.

func (*TabletServer) Begin

func (tsv *TabletServer) Begin(ctx context.Context, target *querypb.Target) (transactionID int64, err error)

Begin starts a new transaction. This is allowed only if the state is StateServing.

func (*TabletServer) BroadcastHealth

func (tsv *TabletServer) BroadcastHealth(terTimestamp int64, stats *querypb.RealtimeStats)

BroadcastHealth will broadcast the current health to all listeners

func (*TabletServer) CheckMySQL

func (tsv *TabletServer) CheckMySQL()

CheckMySQL initiates a check to see if MySQL is reachable. If not, it shuts down the query service. The check is rate-limited to no more than once per second.

func (*TabletServer) ClearQueryPlanCache

func (tsv *TabletServer) ClearQueryPlanCache()

ClearQueryPlanCache clears internal query plan cache

func (*TabletServer) Commit

func (tsv *TabletServer) Commit(ctx context.Context, target *querypb.Target, transactionID int64) (err error)

Commit commits the specified transaction.

func (*TabletServer) EnterLameduck

func (tsv *TabletServer) EnterLameduck()

EnterLameduck causes tabletserver to enter the lameduck state. This state causes health checks to fail, but the behavior of tabletserver otherwise remains the same. Any subsequent calls to SetServingType will cause the tabletserver to exit this mode.

func (*TabletServer) Execute

func (tsv *TabletServer) Execute(ctx context.Context, target *querypb.Target, sql string, bindVariables map[string]interface{}, transactionID int64) (result *sqltypes.Result, err error)

Execute executes the query and returns the result as response.

func (*TabletServer) ExecuteBatch

func (tsv *TabletServer) ExecuteBatch(ctx context.Context, target *querypb.Target, queries []querytypes.BoundQuery, asTransaction bool, transactionID int64) (results []sqltypes.Result, err error)

ExecuteBatch executes a group of queries and returns their results as a list. ExecuteBatch can be called for an existing transaction, or it can be called with the AsTransaction flag which will execute all statements inside an independent transaction. If AsTransaction is true, TransactionId must be 0.

func (*TabletServer) ExitLameduck

func (tsv *TabletServer) ExitLameduck()

ExitLameduck causes the tabletserver to exit the lameduck mode.

func (*TabletServer) GetState

func (tsv *TabletServer) GetState() string

GetState returns the name of the current TabletServer state.

func (*TabletServer) HandlePanic

func (tsv *TabletServer) HandlePanic(err *error)

HandlePanic is part of the queryservice.QueryService interface

func (*TabletServer) InitDBConfig

func (tsv *TabletServer) InitDBConfig(target querypb.Target, dbconfigs dbconfigs.DBConfigs, mysqld mysqlctl.MysqlDaemon) error

InitDBConfig inititalizes the db config variables for TabletServer. You must call this function before calling StartService or SetServingType.

func (*TabletServer) IsHealthy

func (tsv *TabletServer) IsHealthy() error

IsHealthy returns nil if the query service is healthy (able to connect to the database and serving traffic) or an error explaining the unhealthiness otherwise.

func (*TabletServer) IsServing

func (tsv *TabletServer) IsServing() bool

IsServing returns true if TabletServer is in SERVING state.

func (*TabletServer) MaxDMLRows

func (tsv *TabletServer) MaxDMLRows() int

MaxDMLRows returns the max result size.

func (*TabletServer) MaxResultSize

func (tsv *TabletServer) MaxResultSize() int

MaxResultSize returns the max result size.

func (*TabletServer) PoolSize

func (tsv *TabletServer) PoolSize() int

PoolSize returns the pool size.

func (*TabletServer) QueryCacheCap

func (tsv *TabletServer) QueryCacheCap() int

QueryCacheCap returns the pool size.

func (*TabletServer) QueryService

func (tsv *TabletServer) QueryService() queryservice.QueryService

QueryService returns the QueryService part of TabletServer.

func (*TabletServer) QueryServiceStats

func (tsv *TabletServer) QueryServiceStats() *QueryServiceStats

QueryServiceStats returns the QueryServiceStats instance of the TabletServer's QueryEngine.

func (*TabletServer) Register

func (tsv *TabletServer) Register()

Register prepares TabletServer for serving by calling all the registrations functions.

func (*TabletServer) RegisterQueryRuleSource

func (tsv *TabletServer) RegisterQueryRuleSource(ruleSource string)

RegisterQueryRuleSource registers ruleSource for setting query rules.

func (*TabletServer) ReloadSchema

func (tsv *TabletServer) ReloadSchema()

ReloadSchema reloads the schema. If the query service is not running, it's a no-op.

func (*TabletServer) Rollback

func (tsv *TabletServer) Rollback(ctx context.Context, target *querypb.Target, transactionID int64) (err error)

Rollback rollsback the specified transaction.

func (*TabletServer) SetAutoCommit

func (tsv *TabletServer) SetAutoCommit(auto bool)

SetAutoCommit sets autocommit on or off.

func (*TabletServer) SetMaxDMLRows

func (tsv *TabletServer) SetMaxDMLRows(val int)

SetMaxDMLRows changes the max result size to the specified value.

func (*TabletServer) SetMaxResultSize

func (tsv *TabletServer) SetMaxResultSize(val int)

SetMaxResultSize changes the max result size to the specified value.

func (*TabletServer) SetPoolSize

func (tsv *TabletServer) SetPoolSize(val int)

SetPoolSize changes the pool size to the specified value.

func (*TabletServer) SetQueryCacheCap

func (tsv *TabletServer) SetQueryCacheCap(val int)

SetQueryCacheCap changes the pool size to the specified value.

func (*TabletServer) SetQueryRules

func (tsv *TabletServer) SetQueryRules(ruleSource string, qrs *QueryRules) error

SetQueryRules sets the query rules for a registered ruleSource.

func (*TabletServer) SetServingType

func (tsv *TabletServer) SetServingType(tabletType topodatapb.TabletType, serving bool, alsoAllow []topodatapb.TabletType) (bool, error)

SetServingType changes the serving type of the tabletserver. It starts or stops internal services as deemed necessary. The tabletType determines the primary serving type, while alsoAllow specifies other tablet types that should also be honored for serving. Returns true if the state of QueryService or the tablet type changed.

func (*TabletServer) SetStreamPoolSize

func (tsv *TabletServer) SetStreamPoolSize(val int)

SetStreamPoolSize changes the pool size to the specified value.

func (*TabletServer) SetStrictMode

func (tsv *TabletServer) SetStrictMode(strict bool)

SetStrictMode sets strict mode on or off.

func (*TabletServer) SetTxPoolSize

func (tsv *TabletServer) SetTxPoolSize(val int)

SetTxPoolSize changes the tx pool size to the specified value.

func (*TabletServer) SetTxTimeout

func (tsv *TabletServer) SetTxTimeout(val time.Duration)

SetTxTimeout changes the transaction timeout to the specified value.

func (*TabletServer) SplitQuery

func (tsv *TabletServer) SplitQuery(ctx context.Context, target *querypb.Target, sql string, bindVariables map[string]interface{}, splitColumn string, splitCount int64) (splits []querytypes.QuerySplit, err error)

SplitQuery splits a query + bind variables into smaller queries that return a subset of rows from the original query. TODO(erez): Remove this method and rename SplitQueryV2 to SplitQuery once we migrate to SplitQuery V2.

func (*TabletServer) SplitQueryV2

func (tsv *TabletServer) SplitQueryV2(
	ctx context.Context,
	target *querypb.Target,
	sql string,
	bindVariables map[string]interface{},
	splitColumns []string,
	splitCount int64,
	numRowsPerQueryPart int64,
	algorithm querypb.SplitQueryRequest_Algorithm,
) (splits []querytypes.QuerySplit, err error)

SplitQueryV2 splits a query + bind variables into smaller queries that return a subset of rows from the original query. This is the new version that supports multiple split columns and multiple split algortihms. See the documentation of SplitQueryRequest in proto/vtgate.proto for more details.

func (*TabletServer) StartService

func (tsv *TabletServer) StartService(target querypb.Target, dbconfigs dbconfigs.DBConfigs, mysqld mysqlctl.MysqlDaemon) (err error)

StartService is a convenience function for InitDBConfig->SetServingType with serving=true.

func (*TabletServer) StopService

func (tsv *TabletServer) StopService()

StopService shuts down the tabletserver to the uninitialized state. It first transitions to StateShuttingDown, then waits for existing transactions to complete. Once all transactions are resolved, it shuts down the rest of the services and transitions to StateNotConnected.

func (*TabletServer) StreamExecute

func (tsv *TabletServer) StreamExecute(ctx context.Context, target *querypb.Target, sql string, bindVariables map[string]interface{}, sendReply func(*sqltypes.Result) error) (err error)

StreamExecute executes the query and streams the result. The first QueryResult will have Fields set (and Rows nil). The subsequent QueryResult will have Rows set (and Fields nil).

func (*TabletServer) StreamHealthRegister

func (tsv *TabletServer) StreamHealthRegister(c chan<- *querypb.StreamHealthResponse) (int, error)

StreamHealthRegister is part of queryservice.QueryService interface

func (*TabletServer) StreamHealthUnregister

func (tsv *TabletServer) StreamHealthUnregister(id int) error

StreamHealthUnregister is part of queryservice.QueryService interface

func (*TabletServer) StreamPoolSize

func (tsv *TabletServer) StreamPoolSize() int

StreamPoolSize returns the pool size.

func (*TabletServer) TxPoolSize

func (tsv *TabletServer) TxPoolSize() int

TxPoolSize returns the tx pool size.

func (*TabletServer) TxTimeout

func (tsv *TabletServer) TxTimeout() time.Duration

TxTimeout returns the transaction timeout.

func (*TabletServer) UnRegisterQueryRuleSource

func (tsv *TabletServer) UnRegisterQueryRuleSource(ruleSource string)

UnRegisterQueryRuleSource unregisters ruleSource from query rules.

type TxConnection

type TxConnection struct {
	*DBConn
	TransactionID int64

	StartTime         time.Time
	EndTime           time.Time
	Queries           []string
	Conclusion        string
	LogToFile         sync2.AtomicInt32
	ImmediateCallerID *querypb.VTGateCallerID
	EffectiveCallerID *vtrpcpb.CallerID
	// contains filtered or unexported fields
}

TxConnection is meant for executing transactions. It can return itself to the tx pool correctly. It also does not retry statements if there are failures.

func (*TxConnection) EventTime

func (txc *TxConnection) EventTime() time.Time

EventTime returns the time the event was created.

func (*TxConnection) Exec

func (txc *TxConnection) Exec(ctx context.Context, query string, maxrows int, wantfields bool) (*sqltypes.Result, error)

Exec executes the statement for the current transaction.

func (*TxConnection) Format

func (txc *TxConnection) Format(params url.Values) string

Format returns a printable version of the connection info.

func (*TxConnection) RecordQuery

func (txc *TxConnection) RecordQuery(query string)

RecordQuery records the query against this transaction.

func (*TxConnection) Recycle

func (txc *TxConnection) Recycle()

Recycle returns the connection to the pool. The transaction remains active.

type TxPool

type TxPool struct {
	// contains filtered or unexported fields
}

TxPool is the transaction pool for the query service.

func NewTxPool

func NewTxPool(
	name string,
	txStatsPrefix string,
	capacity int,
	timeout time.Duration,
	idleTimeout time.Duration,
	enablePublishStats bool,
	qStats *QueryServiceStats,
	checker MySQLChecker) *TxPool

NewTxPool creates a new TxPool. It's not operational until it's Open'd.

func (*TxPool) Begin

func (axp *TxPool) Begin(ctx context.Context) int64

Begin begins a transaction, and returns the associated transaction id. Subsequent statements can access the connection through the transaction id.

func (*TxPool) Close

func (axp *TxPool) Close()

Close closes the TxPool. A closed pool can be reopened.

func (*TxPool) Commit

func (axp *TxPool) Commit(ctx context.Context, transactionID int64)

Commit commits the specified transaction.

func (*TxPool) Get

func (axp *TxPool) Get(transactionID int64) (conn *TxConnection)

Get fetches the connection associated to the transactionID. You must call Recycle on TxConnection once done.

func (*TxPool) LogActive

func (axp *TxPool) LogActive()

LogActive causes all existing transactions to be logged when they complete. The logging is throttled to no more than once every txLogInterval.

func (*TxPool) Open

func (axp *TxPool) Open(appParams, dbaParams *sqldb.ConnParams)

Open makes the TxPool operational. This also starts the transaction killer that will kill long-running transactions.

func (*TxPool) Rollback

func (axp *TxPool) Rollback(ctx context.Context, transactionID int64)

Rollback rolls back the specified transaction.

func (*TxPool) SetTimeout

func (axp *TxPool) SetTimeout(timeout time.Duration)

SetTimeout sets the transaction timeout.

func (*TxPool) Timeout

func (axp *TxPool) Timeout() time.Duration

Timeout returns the transaction timeout.

func (*TxPool) WaitForEmpty

func (axp *TxPool) WaitForEmpty()

WaitForEmpty waits until all active transactions are completed.

Directories

Path Synopsis
customrule
filecustomrule
Package filecustomrule implements static custom rule from a config file
Package filecustomrule implements static custom rule from a config file
Package endtoend is a test-only package.
Package endtoend is a test-only package.
Package queryservice contains the interface for the service definition of the Query Service.
Package queryservice contains the interface for the service definition of the Query Service.
Package querytypes defines internal types used in the APIs to deal with queries.
Package querytypes defines internal types used in the APIs to deal with queries.
Package splitquery contains the logic needed for implementing the tabletserver's SplitQuery RPC.
Package splitquery contains the logic needed for implementing the tabletserver's SplitQuery RPC.
Package tabletconntest provides the test methods to make sure a tabletconn/queryservice pair over RPC works correctly.
Package tabletconntest provides the test methods to make sure a tabletconn/queryservice pair over RPC works correctly.
Package tabletservermock provides mock interfaces for tabletserver.
Package tabletservermock provides mock interfaces for tabletserver.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL