tabletserver

package
v2.0.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 12, 2016 License: BSD-3-Clause Imports: 59 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// QuerySourceConsolidator means query result is found in consolidator.
	QuerySourceConsolidator = 1 << iota
	// QuerySourceMySQL means query result is returned from MySQL.
	QuerySourceMySQL
)
View Source 
const (
	QRContinue = Action(iota)
	QRFail
	QRFailRetry
)

These are actions.

View Source 
const (
	QRNoOp = Operator(iota)
	QREqual
	QRNotEqual
	QRLessThan
	QRGreaterEqual
	QRGreaterThan
	QRLessEqual
	QRMatch
	QRNoMatch
	QRIn
	QRNotIn
	QRNumOp
)

These are comparison operators.

View Source 
const (
	QROK = iota
	QRMismatch
	QROutOfRange
)

These are return statii.

View Source 
const (
	// StateNotConnected is the state where tabletserver is not
	// connected to an underlying mysql instance.
	StateNotConnected = iota
	// StateNotServing is the state where tabletserver is connected
	// to an underlying mysql instance, but is not serving queries.
	StateNotServing
	// StateServing is where queries are allowed.
	StateServing
	// StateTransitioning is a transient state indicating that
	// the tabletserver is tranisitioning to a new state.
	StateTransitioning
	// StateShuttingDown is a transient state indicating that
	// the tabletserver is shutting down. This state differs from
	// StateTransitioning because we allow queries for transactions
	// that are still in flight.
	StateShuttingDown
)
View Source 
const (
	TxClose    = "close"
	TxCommit   = "commit"
	TxRollback = "rollback"
	TxKill     = "kill"
)

These consts identify how a transaction was resolved.

Variables

View Source 
var DefaultQsConfig = Config{
	PoolSize:             16,
	StreamPoolSize:       750,
	TransactionCap:       20,
	TransactionTimeout:   30,
	MaxResultSize:        10000,
	MaxDMLRows:           500,
	QueryCacheSize:       5000,
	SchemaReloadTime:     30 * 60,
	QueryTimeout:         0,
	TxPoolTimeout:        1,
	IdleTimeout:          30 * 60,
	StreamBufferSize:     32 * 1024,
	StrictMode:           true,
	StrictTableAcl:       false,
	TerseErrors:          false,
	EnablePublishStats:   true,
	EnableAutoCommit:     false,
	EnableTableAclDryRun: false,
	StatsPrefix:          "",
	DebugURLPrefix:       "/debug",
	PoolNamePrefix:       "",
	TableAclExemptACL:    "",
}

DefaultQsConfig is the default value for the query service config. The value for StreamBufferSize was chosen after trying out a few of them. Too small buffers force too many packets to be sent. Too big buffers force the clients to read them in multiple chunks and make memory copies. so with the encoding overhead, this seems to work great (the overhead makes the final packets on the wire about twice bigger than this).

View Source 
var ErrConnPoolClosed = NewTabletError(

	vtrpcpb.ErrorCode_INTERNAL_ERROR,
	"connection pool is closed")

ErrConnPoolClosed is returned / panicked when the connection pool is closed.

View Source 
var RegisterFunctions []RegisterFunction

RegisterFunctions is a list of all the RegisterFunction that will be called upon Register() on a TabletServer

View Source 
var StatsLogger = streamlog.New("TabletServer", 50)

StatsLogger is the main stream logger object

View Source 
var TxLogger = streamlog.New("TxLog", 10)

TxLogger can be used to enable logging of transactions. Call TxLogger.ServeLogs in your main program to enable logging. The log format can be inferred by looking at TxConnection.Format.

Functions

func Init 

func Init()

Init must be called after flag.Parse, and before doing any other operations.

func IsConnErr 

func IsConnErr(err error) bool

IsConnErr returns true if the error is a connection error. If the error is of type TabletError or hasNumber, it checks the error code. Otherwise, it parses the string looking for (errno xxxx) and uses the extracted value to determine if it's a conn error.

func PrefixTabletError 

func PrefixTabletError(errCode vtrpcpb.ErrorCode, err error, prefix string) error

PrefixTabletError attempts to add a string prefix to a TabletError, while preserving its ErrorCode. If the given error is not a TabletError, a new TabletError is returned with the desired ErrorCode.

func Rand 

func Rand() int64

Rand generates a pseudo-random int64 number.

func ToGRPCError 

func ToGRPCError(err error) error

ToGRPCError returns a TabletError as a grpc error, with the appropriate error code. This function lives here, instead of in vterrors, so that the vterrors package doesn't have to import tabletserver.

func ToRPCError 

func ToRPCError(err error) *vtrpcpb.RPCError

ToRPCError returns a vtrpcpb.RPCError (that can be packed across RPC boundaries) for the TabletError.

Types

type Action 

type Action int

Action speficies the list of actions to perform when a QueryRule is triggered.

func (Action) MarshalJSON 

func (act Action) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

type BindVarCond 

type BindVarCond struct {
	// contains filtered or unexported fields
}

BindVarCond represents a bind var condition.

func (BindVarCond) MarshalJSON 

func (bvc BindVarCond) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

type Config 

type Config struct {
	PoolSize             int
	StreamPoolSize       int
	TransactionCap       int
	TransactionTimeout   float64
	MaxResultSize        int
	MaxDMLRows           int
	StreamBufferSize     int
	QueryCacheSize       int
	SchemaReloadTime     float64
	QueryTimeout         float64
	TxPoolTimeout        float64
	IdleTimeout          float64
	StrictMode           bool
	StrictTableAcl       bool
	TerseErrors          bool
	EnablePublishStats   bool
	EnableAutoCommit     bool
	EnableTableAclDryRun bool
	StatsPrefix          string
	DebugURLPrefix       string
	PoolNamePrefix       string
	TableAclExemptACL    string
}

Config contains all the configuration for query service

type ConnPool 

type ConnPool struct {
	// contains filtered or unexported fields
}

ConnPool implements a custom connection pool for tabletserver. It's similar to dbconnpool.ConnPool, but the connections it creates come with built-in ability to kill in-flight queries. These connections also trigger a CheckMySQL call if we fail to connect to MySQL. Other than the connection type, ConnPool maintains an additional pool of dba connections that are used to kill connections.

func NewConnPool 

func NewConnPool(
	name string,
	capacity int,
	idleTimeout time.Duration,
	enablePublishStats bool,
	queryServiceStats *QueryServiceStats,
	checker MySQLChecker) *ConnPool

NewConnPool creates a new ConnPool. The name is used to publish stats only.

func (*ConnPool) Available 

func (cp *ConnPool) Available() int64

Available returns the number of available connections in the pool

func (*ConnPool) Capacity 

func (cp *ConnPool) Capacity() int64

Capacity returns the pool capacity.

func (*ConnPool) Close 

func (cp *ConnPool) Close()

Close will close the pool and wait for connections to be returned before exiting.

func (*ConnPool) Get 

func (cp *ConnPool) Get(ctx context.Context) (*DBConn, error)

Get returns a connection. You must call Recycle on DBConn once done.

func (*ConnPool) IdleTimeout 

func (cp *ConnPool) IdleTimeout() time.Duration

IdleTimeout returns the idle timeout for the pool.

func (*ConnPool) MaxCap 

func (cp *ConnPool) MaxCap() int64

MaxCap returns the maximum size of the pool

func (*ConnPool) Open 

func (cp *ConnPool) Open(appParams, dbaParams *sqldb.ConnParams)

Open must be called before starting to use the pool.

func (*ConnPool) Put 

func (cp *ConnPool) Put(conn *DBConn)

Put puts a connection into the pool.

func (*ConnPool) SetCapacity 

func (cp *ConnPool) SetCapacity(capacity int) (err error)

SetCapacity alters the size of the pool at runtime.

func (*ConnPool) SetIdleTimeout 

func (cp *ConnPool) SetIdleTimeout(idleTimeout time.Duration)

SetIdleTimeout sets the idleTimeout on the pool.

func (*ConnPool) StatsJSON 

func (cp *ConnPool) StatsJSON() string

StatsJSON returns the pool stats as a JSON object.

func (*ConnPool) WaitCount 

func (cp *ConnPool) WaitCount() int64

WaitCount returns how many clients are waiting for a connection

func (*ConnPool) WaitTime 

func (cp *ConnPool) WaitTime() time.Duration

WaitTime return the pool WaitTime.

type Controller 

type Controller interface {
	// Register registers this query service with the RPC layer.
	Register()

	// AddStatusPart adds the status part to the status page
	AddStatusPart()

	// InitDBConfig sets up the db config vars.
	InitDBConfig(querypb.Target, dbconfigs.DBConfigs, mysqlctl.MysqlDaemon) error

	// SetServingType transitions the query service to the required serving type.
	// Returns true if the state of QueryService or the tablet type changed.
	SetServingType(tabletType topodatapb.TabletType, serving bool, alsoAllow []topodatapb.TabletType) (bool, error)

	// EnterLameduck causes tabletserver to enter the lameduck state.
	EnterLameduck()

	// IsServing returns true if the query service is running
	IsServing() bool

	// IsHealthy returns the health status of the QueryService
	IsHealthy() error

	// ClearQueryPlanCache clears internal query plan cache
	ClearQueryPlanCache()

	// ReloadSchema makes the quey service reload its schema cache
	ReloadSchema(ctx context.Context) error

	// RegisterQueryRuleSource adds a query rule source
	RegisterQueryRuleSource(ruleSource string)

	// RegisterQueryRuleSource removes a query rule source
	UnRegisterQueryRuleSource(ruleSource string)

	// SetQueryRules sets the query rules for this QueryService
	SetQueryRules(ruleSource string, qrs *QueryRules) error

	// QueryService returns the QueryService object used by this Controller
	QueryService() queryservice.QueryService

	QueryServiceStats() *QueryServiceStats

	// BroadcastHealth sends the current health to all listeners
	BroadcastHealth(terTimestamp int64, stats *querypb.RealtimeStats)
}

Controller defines the control interface for TabletServer.

type DBConn 

type DBConn struct {
	// contains filtered or unexported fields
}

DBConn is a db connection for tabletserver. It performs automatic reconnects as needed. Its Execute function has a timeout that can kill its own queries and the underlying connection. It will also trigger a CheckMySQL whenever applicable.

func NewDBConn 

func NewDBConn(
	cp *ConnPool,
	appParams,
	dbaParams *sqldb.ConnParams,
	qStats *QueryServiceStats) (*DBConn, error)

NewDBConn creates a new DBConn. It triggers a CheckMySQL if creation fails.

func (*DBConn) Close 

func (dbc *DBConn) Close()

Close closes the DBConn.

func (*DBConn) Current 

func (dbc *DBConn) Current() string

Current returns the currently executing query.

func (*DBConn) Exec 

func (dbc *DBConn) Exec(ctx context.Context, query string, maxrows int, wantfields bool) (*sqltypes.Result, error)

Exec executes the specified query. If there is a connection error, it will reconnect and retry. A failed reconnect will trigger a CheckMySQL.

func (*DBConn) ExecOnce 

func (dbc *DBConn) ExecOnce(ctx context.Context, query string, maxrows int, wantfields bool) (*sqltypes.Result, error)

ExecOnce executes the specified query, but does not retry on connection errors.

func (*DBConn) ID 

func (dbc *DBConn) ID() int64

ID returns the connection id.

func (*DBConn) IsClosed 

func (dbc *DBConn) IsClosed() bool

IsClosed returns true if DBConn is closed.

func (*DBConn) Kill 

func (dbc *DBConn) Kill(reason string) error

Kill kills the currently executing query both on MySQL side and on the connection side. If no query is executing, it's a no-op. Kill will also not kill a query more than once.

func (*DBConn) Recycle 

func (dbc *DBConn) Recycle()

Recycle returns the DBConn to the pool.

func (*DBConn) Stream 

func (dbc *DBConn) Stream(ctx context.Context, query string, callback func(*sqltypes.Result) error, streamBufferSize int) error

Stream executes the query and streams the results.

func (*DBConn) VerifyMode 

func (dbc *DBConn) VerifyMode() error

VerifyMode returns an error if the connection mode is incorrect.

type ExecPlan 

type ExecPlan struct {
	*planbuilder.ExecPlan
	TableInfo  *TableInfo
	Fields     []*querypb.Field
	Rules      *QueryRules
	Authorized *tableacl.ACLResult

	QueryCount int64
	Time       time.Duration
	RowCount   int64
	ErrorCount int64
	// contains filtered or unexported fields
}

ExecPlan wraps the planbuilder's exec plan to enforce additional rules and track stats.

func (*ExecPlan) AddStats 

func (ep *ExecPlan) AddStats(queryCount int64, duration time.Duration, rowCount, errorCount int64)

AddStats updates the stats for the current ExecPlan.

func (*ExecPlan) Size 

func (*ExecPlan) Size() int

Size allows ExecPlan to be in cache.LRUCache.

func (*ExecPlan) Stats 

func (ep *ExecPlan) Stats() (queryCount int64, duration time.Duration, rowCount, errorCount int64)

Stats returns the current stats of ExecPlan.

type LogStats 

type LogStats struct {
	Method        string
	PlanType      string
	OriginalSQL   string
	BindVariables map[string]interface{}

	RowsAffected         int
	NumberOfQueries      int
	StartTime            time.Time
	EndTime              time.Time
	MysqlResponseTime    time.Duration
	WaitingForConnection time.Duration
	QuerySources         byte
	Rows                 [][]sqltypes.Value
	TransactionID        int64

	Error *TabletError
	// contains filtered or unexported fields
}

LogStats records the stats for a single query

func (*LogStats) AddRewrittenSQL 

func (stats *LogStats) AddRewrittenSQL(sql string, start time.Time)

AddRewrittenSQL adds a single sql statement to the rewritten list

func (*LogStats) ContextHTML 

func (stats *LogStats) ContextHTML() template.HTML

ContextHTML returns the HTML version of the context that was used, or "". This is a method on LogStats instead of a field so that it doesn't need to be passed by value everywhere.

func (*LogStats) EffectiveCaller 

func (stats *LogStats) EffectiveCaller() string

EffectiveCaller returns the effective caller stored in LogStats.ctx

func (*LogStats) ErrorStr 

func (stats *LogStats) ErrorStr() string

ErrorStr returns the error string or ""

func (*LogStats) EventTime 

func (stats *LogStats) EventTime() time.Time

EventTime returns the time the event was created.

func (*LogStats) FmtBindVariables 

func (stats *LogStats) FmtBindVariables(full bool) string

FmtBindVariables returns the map of bind variables as JSON. For values that are strings or byte slices it only reports their type and length.

func (*LogStats) FmtQuerySources 

func (stats *LogStats) FmtQuerySources() string

FmtQuerySources returns a comma separated list of query sources. If there were no query sources, it returns the string "none".

func (*LogStats) Format 

func (stats *LogStats) Format(params url.Values) string

Format returns a tab separated list of logged fields.

func (*LogStats) ImmediateCaller 

func (stats *LogStats) ImmediateCaller() string

ImmediateCaller returns the immediate caller stored in LogStats.ctx

func (*LogStats) RemoteAddrUsername 

func (stats *LogStats) RemoteAddrUsername() (string, string)

RemoteAddrUsername returns some parts of CallInfo if set

func (*LogStats) RewrittenSQL 

func (stats *LogStats) RewrittenSQL() string

RewrittenSQL returns a semicolon separated list of SQL statements that were executed.

func (*LogStats) Send 

func (stats *LogStats) Send()

Send finalizes a record and sends it

func (*LogStats) SizeOfResponse 

func (stats *LogStats) SizeOfResponse() int

SizeOfResponse returns the approximate size of the response in bytes (this does not take in account protocol encoding). It will return 0 for streaming requests.

func (*LogStats) TotalTime 

func (stats *LogStats) TotalTime() time.Duration

TotalTime returns how long this query has been running

type MySQLChecker 

type MySQLChecker interface {
	CheckMySQL()
}

MySQLChecker defines the CheckMySQL interface that lower level objects can use to call back into TabletServer.

type Operator 

type Operator int

Operator represents the list of operators.

func MapStrOperator 

func MapStrOperator(strop string) (op Operator, err error)

MapStrOperator maps a string representation to an Operator.

func (Operator) MarshalJSON 

func (op Operator) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

type QueryDetail 

type QueryDetail struct {
	// contains filtered or unexported fields
}

QueryDetail is a simple wrapper for Query, Context and a killable conn.

func NewQueryDetail 

func NewQueryDetail(ctx context.Context, conn killable) *QueryDetail

NewQueryDetail creates a new QueryDetail

type QueryDetailzRow 

type QueryDetailzRow struct {
	Query             string
	ContextHTML       template.HTML
	Start             time.Time
	Duration          time.Duration
	ConnID            int64
	State             string
	ShowTerminateLink bool
}

QueryDetailzRow is used for rendering QueryDetail in a template

type QueryEngine 

type QueryEngine struct {
	// contains filtered or unexported fields
}

QueryEngine implements the core functionality of tabletserver. It assumes that no requests will be sent to it before Open is called and succeeds. Shutdown is done in the following order:

WaitForTxEmpty: There should be no more new calls to Begin once this function is called. This will return when there are no more pending transactions.

Close: There should be no more pending queries when this function is called.

Functions of QueryEngine do not return errors. They instead panic with NewTabletError as the error type. TODO(sougou): Switch to error return scheme.

func NewQueryEngine 

func NewQueryEngine(checker MySQLChecker, config Config) *QueryEngine

NewQueryEngine creates a new QueryEngine. This is a singleton class. You must call this only once.

func (*QueryEngine) Close 

func (qe *QueryEngine) Close()

Close must be called to shut down QueryEngine. You must ensure that no more queries will be sent before calling Close.

func (*QueryEngine) IsMySQLReachable 

func (qe *QueryEngine) IsMySQLReachable() bool

IsMySQLReachable returns true if we can connect to MySQL.

func (*QueryEngine) Open 

func (qe *QueryEngine) Open(dbconfigs dbconfigs.DBConfigs)

Open must be called before sending requests to QueryEngine.

func (*QueryEngine) WaitForTxEmpty 

func (qe *QueryEngine) WaitForTxEmpty()

WaitForTxEmpty must be called before calling Close. Before calling WaitForTxEmpty, you must ensure that there will be no more calls to Begin.

type QueryExecutor 

type QueryExecutor struct {
	// contains filtered or unexported fields
}

QueryExecutor is used for executing a query request.

func (*QueryExecutor) Execute 

func (qre *QueryExecutor) Execute() (reply *sqltypes.Result, err error)

Execute performs a non-streaming query execution.

func (*QueryExecutor) Stream 

func (qre *QueryExecutor) Stream(sendReply func(*sqltypes.Result) error) error

Stream performs a streaming query execution.

type QueryList 

type QueryList struct {
	// contains filtered or unexported fields
}

QueryList holds a thread safe list of QueryDetails

func NewQueryList 

func NewQueryList() *QueryList

NewQueryList creates a new QueryList

func (*QueryList) Add 

func (ql *QueryList) Add(qd *QueryDetail)

Add adds a QueryDetail to QueryList

func (*QueryList) GetQueryzRows 

func (ql *QueryList) GetQueryzRows() []QueryDetailzRow

GetQueryzRows returns a list of QueryDetailzRow sorted by start time

func (*QueryList) Remove 

func (ql *QueryList) Remove(qd *QueryDetail)

Remove removes a QueryDetail from QueryList

func (*QueryList) Terminate 

func (ql *QueryList) Terminate(connID int64) error

Terminate updates the query status and kills the connection

func (*QueryList) TerminateAll 

func (ql *QueryList) TerminateAll()

TerminateAll terminates all queries and kills the MySQL connections

type QueryRule 

type QueryRule struct {
	Description string
	Name        string
	// contains filtered or unexported fields
}

QueryRule represents one rule (conditions-action). Name is meant to uniquely identify a rule. Description is a human readable comment that describes the rule. For a QueryRule to fire, all conditions of the QueryRule have to match. For example, an empty QueryRule will match all requests. Every QueryRule has an associated Action. If all the conditions of the QueryRule are met, then the Action is triggerred.

func BuildQueryRule 

func BuildQueryRule(ruleInfo map[string]interface{}) (qr *QueryRule, err error)

BuildQueryRule builds a query rule from a ruleInfo.

func NewQueryRule 

func NewQueryRule(description, name string, act Action) (qr *QueryRule)

NewQueryRule creates a new QueryRule.

func (*QueryRule) AddBindVarCond 

func (qr *QueryRule) AddBindVarCond(name string, onAbsent, onMismatch bool, op Operator, value interface{}) error

AddBindVarCond adds a bind variable restriction to the QueryRule. All bind var conditions have to be satisfied for the QueryRule to be a match. name represents the name (not regexp) of the bind variable. onAbsent specifies the value of the condition if the bind variable is absent. onMismatch specifies the value of the condition if there's a type mismatch on the condition. For inequalities, the bindvar is the left operand and the value in the condition is the right operand: bindVar Operator value. Value & operator rules Type Operators Bindvar nil "" any type uint64 ==, !=, <, >=, >, <= whole numbers int64 ==, !=, <, >=, >, <= whole numbers string ==, !=, <, >=, >, <=, MATCH, NOMATCH []byte, string KeyRange IN, NOTIN whole numbers whole numbers can be: int, int8, int16, int32, int64, uint64

func (*QueryRule) AddPlanCond 

func (qr *QueryRule) AddPlanCond(planType planbuilder.PlanType)

AddPlanCond adds to the list of plans that can be matched for the rule to fire. This function acts as an OR: Any plan id match is considered a match.

func (*QueryRule) AddTableCond 

func (qr *QueryRule) AddTableCond(tableName string)

AddTableCond adds to the list of tableNames that can be matched for the rule to fire. This function acts as an OR: Any tableName match is considered a match.

func (*QueryRule) Copy 

func (qr *QueryRule) Copy() (newqr *QueryRule)

Copy performs a deep copy of a QueryRule.

func (*QueryRule) MarshalJSON 

func (qr *QueryRule) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

func (*QueryRule) SetIPCond 

func (qr *QueryRule) SetIPCond(pattern string) (err error)

SetIPCond adds a regular expression condition for the client IP. It has to be a full match (not substring).

func (*QueryRule) SetQueryCond 

func (qr *QueryRule) SetQueryCond(pattern string) (err error)

SetQueryCond adds a regular expression condition for the query.

func (*QueryRule) SetUserCond 

func (qr *QueryRule) SetUserCond(pattern string) (err error)

SetUserCond adds a regular expression condition for the user name used by the client.

type QueryRuleInfo 

type QueryRuleInfo struct {
	// contains filtered or unexported fields
}

QueryRuleInfo is the maintainer of QueryRules from multiple sources

func NewQueryRuleInfo 

func NewQueryRuleInfo() *QueryRuleInfo

NewQueryRuleInfo returns an empty QueryRuleInfo object for use

func (*QueryRuleInfo) GetRules 

func (qri *QueryRuleInfo) GetRules(ruleSource string) (*QueryRules, error)

GetRules returns the corresponding QueryRules as designated by ruleSource parameter

func (*QueryRuleInfo) MarshalJSON 

func (qri *QueryRuleInfo) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

func (*QueryRuleInfo) RegisterQueryRuleSource 

func (qri *QueryRuleInfo) RegisterQueryRuleSource(ruleSource string)

RegisterQueryRuleSource registers a query rule source name with QueryRuleInfo

func (*QueryRuleInfo) SetRules 

func (qri *QueryRuleInfo) SetRules(ruleSource string, newRules *QueryRules) error

SetRules takes an external QueryRules structure and overwrite one of the internal QueryRules as designated by ruleSource parameter

func (*QueryRuleInfo) UnRegisterQueryRuleSource 

func (qri *QueryRuleInfo) UnRegisterQueryRuleSource(ruleSource string)

UnRegisterQueryRuleSource removes a registered query rule source name

type QueryRules 

type QueryRules struct {
	// contains filtered or unexported fields
}

QueryRules is used to store and execute rules for the tabletserver.

func NewQueryRules 

func NewQueryRules() *QueryRules

NewQueryRules creates a new QueryRules.

func (*QueryRules) Add 

func (qrs *QueryRules) Add(qr *QueryRule)

Add adds a QueryRule to QueryRules. It does not check for duplicates.

func (*QueryRules) Append 

func (qrs *QueryRules) Append(otherqrs *QueryRules)

Append merges the rules from another QueryRules into the receiver

func (*QueryRules) Copy 

func (qrs *QueryRules) Copy() (newqrs *QueryRules)

Copy performs a deep copy of QueryRules. A nil input produces a nil output.

func (*QueryRules) Delete 

func (qrs *QueryRules) Delete(name string) (qr *QueryRule)

Delete deletes a QueryRule by name and returns the rule that was deleted. It returns nil if the rule was not found.

func (*QueryRules) Find 

func (qrs *QueryRules) Find(name string) (qr *QueryRule)

Find finds the first occurrence of a QueryRule by matching the Name field. It returns nil if the rule was not found.

func (*QueryRules) MarshalJSON 

func (qrs *QueryRules) MarshalJSON() ([]byte, error)

MarshalJSON marshals to JSON.

func (*QueryRules) UnmarshalJSON 

func (qrs *QueryRules) UnmarshalJSON(data []byte) (err error)

UnmarshalJSON unmarshals QueryRules.

type QueryServiceStats 

type QueryServiceStats struct {
	// MySQLStats shows the time histogram for operations spent on mysql side.
	MySQLStats *stats.Timings
	// QueryStats shows the time histogram for each type of queries.
	QueryStats *stats.Timings
	// WaitStats shows the time histogram for wait operations
	WaitStats *stats.Timings
	// KillStats shows number of connections being killed.
	KillStats *stats.Counters
	// InfoErrors shows number of various non critical errors happened.
	InfoErrors *stats.Counters
	// ErrorStats shows number of critial erros happened.
	ErrorStats *stats.Counters
	// InternalErros shows number of errors from internal components.
	InternalErrors *stats.Counters
	// UserTableQueryCount shows number of queries received for each CallerID/table combination.
	UserTableQueryCount *stats.MultiCounters
	// UserTableQueryTimesNs shows total latency for each CallerID/table combination.
	UserTableQueryTimesNs *stats.MultiCounters
	// UserTransactionCount shows number of transactions received for each CallerID.
	UserTransactionCount *stats.MultiCounters
	// UserTransactionTimesNs shows total transaction latency for each CallerID.
	UserTransactionTimesNs *stats.MultiCounters
	// QPSRates shows the qps.
	QPSRates *stats.Rates
	// ResultStats shows the histogram of number of rows returned.
	ResultStats *stats.Histogram
}

QueryServiceStats contains stats that used in queryservice level.

func NewQueryServiceStats 

func NewQueryServiceStats(statsPrefix string, enablePublishStats bool) *QueryServiceStats

NewQueryServiceStats returns a new QueryServiceStats instance.

type QuerySplitter 

type QuerySplitter struct {
	// contains filtered or unexported fields
}

QuerySplitter splits a BoundQuery into equally sized smaller queries. QuerySplits are generated by adding primary key range clauses to the original query. Only a limited set of queries are supported, see QuerySplitter.validateQuery() for details. Also, the table must have at least one primary key and the leading primary key must be numeric, see QuerySplitter.splitBoundaries()

func NewQuerySplitter 

func NewQuerySplitter(
	sql string,
	bindVariables map[string]interface{},
	splitColumn string,
	splitCount int64,
	schemaInfo *SchemaInfo) *QuerySplitter

NewQuerySplitter creates a new QuerySplitter. query is the original query to split and splitCount is the desired number of splits. splitCount must be a positive int, if not it will be set to 1.

type RegisterFunction 

type RegisterFunction func(Controller)

RegisterFunction is a callback type to be called when we Register() a TabletServer

type SchemaInfo 

type SchemaInfo struct {
	// contains filtered or unexported fields
}

SchemaInfo stores the schema info and performs operations that keep itself up-to-date.

func NewSchemaInfo 

func NewSchemaInfo(
	statsPrefix string,
	checker MySQLChecker,
	queryCacheSize int,
	reloadTime time.Duration,
	idleTimeout time.Duration,
	endpoints map[string]string,
	enablePublishStats bool,
	queryServiceStats *QueryServiceStats) *SchemaInfo

NewSchemaInfo creates a new SchemaInfo.

func (*SchemaInfo) ClearQueryPlanCache 

func (si *SchemaInfo) ClearQueryPlanCache()

ClearQueryPlanCache should be called if query plan cache is potentially obsolete

func (*SchemaInfo) Close 

func (si *SchemaInfo) Close()

Close shuts down SchemaInfo. It can be re-opened after Close.

func (*SchemaInfo) CreateOrUpdateTable 

func (si *SchemaInfo) CreateOrUpdateTable(ctx context.Context, tableName string) error

CreateOrUpdateTable must be called if a DDL was applied to that table.

func (*SchemaInfo) DropTable 

func (si *SchemaInfo) DropTable(tableName string)

DropTable must be called if a table was dropped.

func (*SchemaInfo) GetPlan 

func (si *SchemaInfo) GetPlan(ctx context.Context, logStats *LogStats, sql string) *ExecPlan

GetPlan returns the ExecPlan that for the query. Plans are cached in a cache.LRUCache.

func (*SchemaInfo) GetSchema 

func (si *SchemaInfo) GetSchema() []*schema.Table

GetSchema returns a copy of the schema.

func (*SchemaInfo) GetStreamPlan 

func (si *SchemaInfo) GetStreamPlan(sql string) *ExecPlan

GetStreamPlan is similar to GetPlan, but doesn't use the cache and doesn't enforce a limit. It just returns the parsed query.

func (*SchemaInfo) GetTable 

func (si *SchemaInfo) GetTable(tableName string) *TableInfo

GetTable returns the TableInfo for a table.

func (*SchemaInfo) IsClosed 

func (si *SchemaInfo) IsClosed() bool

IsClosed returns true if the SchemaInfo is closed.

func (*SchemaInfo) Open 

func (si *SchemaInfo) Open(dbaParams *sqldb.ConnParams, strictMode bool)

Open initializes the current SchemaInfo for service by loading the necessary info from the specified database.

func (*SchemaInfo) QueryCacheCap 

func (si *SchemaInfo) QueryCacheCap() int

QueryCacheCap returns the capacity of the query cache.

func (*SchemaInfo) Reload 

func (si *SchemaInfo) Reload(ctx context.Context) error

Reload reloads the schema info from the db. Any tables that have changed since the last load are updated. This is a no-op if the SchemaInfo is closed.

func (*SchemaInfo) ReloadTime 

func (si *SchemaInfo) ReloadTime() time.Duration

ReloadTime returns schema info reload time.

func (*SchemaInfo) ServeHTTP 

func (si *SchemaInfo) ServeHTTP(response http.ResponseWriter, request *http.Request)

func (*SchemaInfo) SetQueryCacheCap 

func (si *SchemaInfo) SetQueryCacheCap(size int)

SetQueryCacheCap sets the query cache capacity.

func (*SchemaInfo) SetReloadTime 

func (si *SchemaInfo) SetReloadTime(reloadTime time.Duration)

SetReloadTime changes how often the schema is reloaded. This call also triggers an immediate reload.

type TableInfo 

type TableInfo struct {
	*schema.Table

	// Seq must be locked before accessing the sequence vars.
	// If CurVal==LastVal, we have to cache new values.
	Seq       sync.Mutex
	NextVal   int64
	Increment int64
	LastVal   int64
}

TableInfo contains the tabletserver related info for a table. It's a superset of schema.Table.

func NewTableInfo 

func NewTableInfo(conn *DBConn, tableName string, tableType string, comment string) (ti *TableInfo, err error)

NewTableInfo creates a new TableInfo.

func (*TableInfo) SetPK 

func (ti *TableInfo) SetPK(colnames []string) error

SetPK sets the pk columns for a TableInfo.

type TabletError 

type TabletError struct {
	Message  string
	SQLError int
	SQLState string
	// ErrorCode will be used to transmit the error across RPC boundaries
	ErrorCode vtrpcpb.ErrorCode
}

TabletError is the error type we use in this library

func NewTabletError 

func NewTabletError(errCode vtrpcpb.ErrorCode, format string, args ...interface{}) *TabletError

NewTabletError returns a TabletError of the given type

func NewTabletErrorSQL 

func NewTabletErrorSQL(errCode vtrpcpb.ErrorCode, err error) *TabletError

NewTabletErrorSQL returns a TabletError based on the error

func (*TabletError) Error 

func (te *TabletError) Error() string

func (*TabletError) Prefix 

func (te *TabletError) Prefix() string

Prefix returns the prefix for the error, like error, fatal, etc.

func (*TabletError) RecordStats 

func (te *TabletError) RecordStats(queryServiceStats *QueryServiceStats)

RecordStats will record the error in the proper stat bucket

func (*TabletError) VtErrorCode 

func (te *TabletError) VtErrorCode() vtrpcpb.ErrorCode

VtErrorCode returns the underlying Vitess error code

type TabletServer 

type TabletServer struct {
	QueryTimeout sync2.AtomicDuration
	BeginTimeout sync2.AtomicDuration
	// contains filtered or unexported fields
}

TabletServer implements the RPC interface for the query service.

func NewServer 

func NewServer() *TabletServer

NewServer creates a new TabletServer based on the command line flags.

func NewTabletServer 

func NewTabletServer(config Config) *TabletServer

NewTabletServer creates an instance of TabletServer. Only one instance of TabletServer can be created per process.

func (*TabletServer) AddStatusPart 

func (tsv *TabletServer) AddStatusPart()

AddStatusPart registers the status part for the status page.

func (*TabletServer) Begin 

func (tsv *TabletServer) Begin(ctx context.Context, target *querypb.Target) (transactionID int64, err error)

Begin starts a new transaction. This is allowed only if the state is StateServing.

func (*TabletServer) BroadcastHealth 

func (tsv *TabletServer) BroadcastHealth(terTimestamp int64, stats *querypb.RealtimeStats)

BroadcastHealth will broadcast the current health to all listeners

func (*TabletServer) CheckMySQL 

func (tsv *TabletServer) CheckMySQL()

CheckMySQL initiates a check to see if MySQL is reachable. If not, it shuts down the query service. The check is rate-limited to no more than once per second.

func (*TabletServer) ClearQueryPlanCache 

func (tsv *TabletServer) ClearQueryPlanCache()

ClearQueryPlanCache clears internal query plan cache

func (*TabletServer) Commit 

func (tsv *TabletServer) Commit(ctx context.Context, target *querypb.Target, transactionID int64) (err error)

Commit commits the specified transaction.

func (*TabletServer) EnterLameduck 

func (tsv *TabletServer) EnterLameduck()

EnterLameduck causes tabletserver to enter the lameduck state. This state causes health checks to fail, but the behavior of tabletserver otherwise remains the same. Any subsequent calls to SetServingType will cause the tabletserver to exit this mode.

func (*TabletServer) Execute 

func (tsv *TabletServer) Execute(ctx context.Context, target *querypb.Target, sql string, bindVariables map[string]interface{}, transactionID int64) (result *sqltypes.Result, err error)

Execute executes the query and returns the result as response.

func (*TabletServer) ExecuteBatch 

func (tsv *TabletServer) ExecuteBatch(ctx context.Context, target *querypb.Target, queries []querytypes.BoundQuery, asTransaction bool, transactionID int64) (results []sqltypes.Result, err error)

ExecuteBatch executes a group of queries and returns their results as a list. ExecuteBatch can be called for an existing transaction, or it can be called with the AsTransaction flag which will execute all statements inside an independent transaction. If AsTransaction is true, TransactionId must be 0.

func (*TabletServer) ExitLameduck 

func (tsv *TabletServer) ExitLameduck()

ExitLameduck causes the tabletserver to exit the lameduck mode.

func (*TabletServer) GetState 

func (tsv *TabletServer) GetState() string

GetState returns the name of the current TabletServer state.

func (*TabletServer) HandlePanic 

func (tsv *TabletServer) HandlePanic(err *error)

HandlePanic is part of the queryservice.QueryService interface

func (*TabletServer) InitDBConfig 

func (tsv *TabletServer) InitDBConfig(target querypb.Target, dbconfigs dbconfigs.DBConfigs, mysqld mysqlctl.MysqlDaemon) error

InitDBConfig inititalizes the db config variables for TabletServer. You must call this function before calling StartService or SetServingType.

func (*TabletServer) IsHealthy 

func (tsv *TabletServer) IsHealthy() error

IsHealthy returns nil if the query service is healthy (able to connect to the database and serving traffic) or an error explaining the unhealthiness otherwise.

func (*TabletServer) IsServing 

func (tsv *TabletServer) IsServing() bool

IsServing returns true if TabletServer is in SERVING state.

func (*TabletServer) MaxDMLRows 

func (tsv *TabletServer) MaxDMLRows() int

MaxDMLRows returns the max result size.

func (*TabletServer) MaxResultSize 

func (tsv *TabletServer) MaxResultSize() int

MaxResultSize returns the max result size.

func (*TabletServer) PoolSize 

func (tsv *TabletServer) PoolSize() int

PoolSize returns the pool size.

func (*TabletServer) QueryCacheCap 

func (tsv *TabletServer) QueryCacheCap() int

QueryCacheCap returns the pool size.

func (*TabletServer) QueryService 

func (tsv *TabletServer) QueryService() queryservice.QueryService

QueryService returns the QueryService part of TabletServer.

func (*TabletServer) QueryServiceStats 

func (tsv *TabletServer) QueryServiceStats() *QueryServiceStats

QueryServiceStats returns the QueryServiceStats instance of the TabletServer's QueryEngine.

func (*TabletServer) Register 

func (tsv *TabletServer) Register()

Register prepares TabletServer for serving by calling all the registrations functions.

func (*TabletServer) RegisterQueryRuleSource 

func (tsv *TabletServer) RegisterQueryRuleSource(ruleSource string)

RegisterQueryRuleSource registers ruleSource for setting query rules.

func (*TabletServer) ReloadSchema 

func (tsv *TabletServer) ReloadSchema(ctx context.Context) error

ReloadSchema reloads the schema.

func (*TabletServer) Rollback 

func (tsv *TabletServer) Rollback(ctx context.Context, target *querypb.Target, transactionID int64) (err error)

Rollback rollsback the specified transaction.

func (*TabletServer) SetAutoCommit 

func (tsv *TabletServer) SetAutoCommit(auto bool)

SetAutoCommit sets autocommit on or off.

func (*TabletServer) SetMaxDMLRows 

func (tsv *TabletServer) SetMaxDMLRows(val int)

SetMaxDMLRows changes the max result size to the specified value.

func (*TabletServer) SetMaxResultSize 

func (tsv *TabletServer) SetMaxResultSize(val int)

SetMaxResultSize changes the max result size to the specified value.

func (*TabletServer) SetPoolSize 

func (tsv *TabletServer) SetPoolSize(val int)

SetPoolSize changes the pool size to the specified value.

func (*TabletServer) SetQueryCacheCap 

func (tsv *TabletServer) SetQueryCacheCap(val int)

SetQueryCacheCap changes the pool size to the specified value.

func (*TabletServer) SetQueryRules 

func (tsv *TabletServer) SetQueryRules(ruleSource string, qrs *QueryRules) error

SetQueryRules sets the query rules for a registered ruleSource.

func (*TabletServer) SetServingType 

func (tsv *TabletServer) SetServingType(tabletType topodatapb.TabletType, serving bool, alsoAllow []topodatapb.TabletType) (bool, error)

SetServingType changes the serving type of the tabletserver. It starts or stops internal services as deemed necessary. The tabletType determines the primary serving type, while alsoAllow specifies other tablet types that should also be honored for serving. Returns true if the state of QueryService or the tablet type changed.

func (*TabletServer) SetStreamPoolSize 

func (tsv *TabletServer) SetStreamPoolSize(val int)

SetStreamPoolSize changes the pool size to the specified value.

func (*TabletServer) SetStrictMode 

func (tsv *TabletServer) SetStrictMode(strict bool)

SetStrictMode sets strict mode on or off.

func (*TabletServer) SetTxPoolSize 

func (tsv *TabletServer) SetTxPoolSize(val int)

SetTxPoolSize changes the tx pool size to the specified value.

func (*TabletServer) SetTxTimeout 

func (tsv *TabletServer) SetTxTimeout(val time.Duration)

SetTxTimeout changes the transaction timeout to the specified value.

func (*TabletServer) SplitQuery 

func (tsv *TabletServer) SplitQuery(ctx context.Context, target *querypb.Target, sql string, bindVariables map[string]interface{}, splitColumn string, splitCount int64) (splits []querytypes.QuerySplit, err error)

SplitQuery splits a query + bind variables into smaller queries that return a subset of rows from the original query. TODO(erez): Remove this method and rename SplitQueryV2 to SplitQuery once we migrate to SplitQuery V2.

func (*TabletServer) SplitQueryV2 

func (tsv *TabletServer) SplitQueryV2(
	ctx context.Context,
	target *querypb.Target,
	sql string,
	bindVariables map[string]interface{},
	splitColumns []string,
	splitCount int64,
	numRowsPerQueryPart int64,
	algorithm querypb.SplitQueryRequest_Algorithm,
) (splits []querytypes.QuerySplit, err error)

SplitQueryV2 splits a query + bind variables into smaller queries that return a subset of rows from the original query. This is the new version that supports multiple split columns and multiple split algortihms. See the documentation of SplitQueryRequest in proto/vtgate.proto for more details.

func (*TabletServer) StartService 

func (tsv *TabletServer) StartService(target querypb.Target, dbconfigs dbconfigs.DBConfigs, mysqld mysqlctl.MysqlDaemon) (err error)

StartService is a convenience function for InitDBConfig->SetServingType with serving=true.

func (*TabletServer) StopService 

func (tsv *TabletServer) StopService()

StopService shuts down the tabletserver to the uninitialized state. It first transitions to StateShuttingDown, then waits for existing transactions to complete. Once all transactions are resolved, it shuts down the rest of the services and transitions to StateNotConnected.

func (*TabletServer) StreamExecute 

func (tsv *TabletServer) StreamExecute(ctx context.Context, target *querypb.Target, sql string, bindVariables map[string]interface{}, sendReply func(*sqltypes.Result) error) (err error)

StreamExecute executes the query and streams the result. The first QueryResult will have Fields set (and Rows nil). The subsequent QueryResult will have Rows set (and Fields nil).

func (*TabletServer) StreamHealthRegister 

func (tsv *TabletServer) StreamHealthRegister(c chan<- *querypb.StreamHealthResponse) (int, error)

StreamHealthRegister is part of queryservice.QueryService interface

func (*TabletServer) StreamHealthUnregister 

func (tsv *TabletServer) StreamHealthUnregister(id int) error

StreamHealthUnregister is part of queryservice.QueryService interface

func (*TabletServer) StreamPoolSize 

func (tsv *TabletServer) StreamPoolSize() int

StreamPoolSize returns the pool size.

func (*TabletServer) TxPoolSize 

func (tsv *TabletServer) TxPoolSize() int

TxPoolSize returns the tx pool size.

func (*TabletServer) TxTimeout 

func (tsv *TabletServer) TxTimeout() time.Duration

TxTimeout returns the transaction timeout.

func (*TabletServer) UnRegisterQueryRuleSource 

func (tsv *TabletServer) UnRegisterQueryRuleSource(ruleSource string)

UnRegisterQueryRuleSource unregisters ruleSource from query rules.

type TxConnection 

type TxConnection struct {
	*DBConn
	TransactionID int64

	StartTime         time.Time
	EndTime           time.Time
	Queries           []string
	Conclusion        string
	LogToFile         sync2.AtomicInt32
	ImmediateCallerID *querypb.VTGateCallerID
	EffectiveCallerID *vtrpcpb.CallerID
	// contains filtered or unexported fields
}

TxConnection is meant for executing transactions. It can return itself to the tx pool correctly. It also does not retry statements if there are failures.

func (*TxConnection) EventTime 

func (txc *TxConnection) EventTime() time.Time

EventTime returns the time the event was created.

func (*TxConnection) Exec 

func (txc *TxConnection) Exec(ctx context.Context, query string, maxrows int, wantfields bool) (*sqltypes.Result, error)

Exec executes the statement for the current transaction.

func (*TxConnection) Format 

func (txc *TxConnection) Format(params url.Values) string

Format returns a printable version of the connection info.

func (*TxConnection) RecordQuery 

func (txc *TxConnection) RecordQuery(query string)

RecordQuery records the query against this transaction.

func (*TxConnection) Recycle 

func (txc *TxConnection) Recycle()

Recycle returns the connection to the pool. The transaction remains active.

type TxPool 

type TxPool struct {
	// contains filtered or unexported fields
}

TxPool is the transaction pool for the query service.

func NewTxPool 

func NewTxPool(
	name string,
	txStatsPrefix string,
	capacity int,
	timeout time.Duration,
	idleTimeout time.Duration,
	enablePublishStats bool,
	qStats *QueryServiceStats,
	checker MySQLChecker) *TxPool

NewTxPool creates a new TxPool. It's not operational until it's Open'd.

func (*TxPool) Begin 

func (axp *TxPool) Begin(ctx context.Context) int64

Begin begins a transaction, and returns the associated transaction id. Subsequent statements can access the connection through the transaction id.

func (*TxPool) Close 

func (axp *TxPool) Close()

Close closes the TxPool. A closed pool can be reopened.

func (*TxPool) Commit 

func (axp *TxPool) Commit(ctx context.Context, transactionID int64)

Commit commits the specified transaction.

func (*TxPool) Get 

func (axp *TxPool) Get(transactionID int64) (conn *TxConnection)

Get fetches the connection associated to the transactionID. You must call Recycle on TxConnection once done.

func (*TxPool) LogActive 

func (axp *TxPool) LogActive()

LogActive causes all existing transactions to be logged when they complete. The logging is throttled to no more than once every txLogInterval.

func (*TxPool) Open 

func (axp *TxPool) Open(appParams, dbaParams *sqldb.ConnParams)

Open makes the TxPool operational. This also starts the transaction killer that will kill long-running transactions.

func (*TxPool) Rollback 

func (axp *TxPool) Rollback(ctx context.Context, transactionID int64)

Rollback rolls back the specified transaction.

func (*TxPool) SetTimeout 

func (axp *TxPool) SetTimeout(timeout time.Duration)

SetTimeout sets the transaction timeout.

func (*TxPool) Timeout 

func (axp *TxPool) Timeout() time.Duration

Timeout returns the transaction timeout.

func (*TxPool) WaitForEmpty 

func (axp *TxPool) WaitForEmpty()

WaitForEmpty waits until all active transactions are completed.

Source Files

View all Source files

Directories

Path Synopsis
customrule
filecustomrule
Package filecustomrule implements static custom rule from a config file
 Package filecustomrule implements static custom rule from a config file
zkcustomrule
Package endtoend is a test-only package.
 Package endtoend is a test-only package.
framework
Package queryservice contains the interface for the service definition of the Query Service.
 Package queryservice contains the interface for the service definition of the Query Service.
fakes
queryservice_testing
Package querytypes defines internal types used in the APIs to deal with queries.
 Package querytypes defines internal types used in the APIs to deal with queries.
Package sandboxconn provides a fake TabletConn implementation for tests.
 Package sandboxconn provides a fake TabletConn implementation for tests.
Package splitquery contains the logic needed for implementing the tabletserver's SplitQuery RPC.
 Package splitquery contains the logic needed for implementing the tabletserver's SplitQuery RPC.
splitquery_testing
Package tabletconntest provides the test methods to make sure a tabletconn/queryservice pair over RPC works correctly.
 Package tabletconntest provides the test methods to make sure a tabletconn/queryservice pair over RPC works correctly.
Package tabletservermock provides mock interfaces for tabletserver.
 Package tabletservermock provides mock interfaces for tabletserver.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.