sqlds

package module
v5.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 16, 2026 License: Apache-2.0 Imports: 33 Imported by: 0

README

sqlds for Hydrolix

Hydrolix-specific fork of grafana/sqlds v5.1.1.

Fork Changes

Forked on 2026-04-16. Summary of changes from upstream:

Added
  • interpolator.go — SQL interpolation and macro handling using Hydrolix's clickhouse-sql-parser
  • metadata.go — Metadata provider for ClickHouse metadata queries (primary keys, ad-hoc filters) with TTL caching
  • models/settings.go — Hydrolix-specific plugin settings and configuration models
  • driver_round_time_test.go — Round time functionality tests
  • Comprehensive test coverage for interpolator, macros, metadata, and settings
Changed
  • Renamed Go module to github.com/hydrolix/sqlds/v5
  • Rewrote macros.go with Hydrolix-specific macro implementations
  • Modified connector.go, datasource.go, driver.go, and health.go for Hydrolix integration
Removed
  • .github/ CI/CD workflows, codeowners, and issue automation (Grafana-specific)
  • completion.go — generic completion logic
  • driver-mock.go — generic driver mock
  • Generic integration and middleware tests
Dependencies
  • Added github.com/hydrolix/clickhouse-sql-parser v0.3.0
  • Added github.com/jellydator/ttlcache/v3 v3.4.0
  • Added github.com/mithrandie/csvq-driver v1.7.0
  • Added github.com/grafana/dataplane/sdata v0.0.9

Documentation

Index

Constants

View Source 
const (
	SyntheticNull  = "__null__"
	SyntheticEmpty = "__empty__"
	RegexPrefix    = "regex:"
)
View Source 
const (
	StatusOK         Status   = "ok"
	StatusError      Status   = "error"
	EndpointHealth   Endpoint = "health"
	EndpointQuery    Endpoint = "query"
	SourceDownstream Source   = "downstream"
	SourcePlugin     Source   = "plugin"
)
View Source 
const (
	// FormatOptionTimeSeries formats the query results as a timeseries using "LongToWide"
	FormatOptionTimeSeries = sqlutil.FormatOptionTimeSeries
	// FormatOptionTable formats the query results as a table using "LongToWide"
	FormatOptionTable = sqlutil.FormatOptionTable
	// FormatOptionLogs sets the preferred visualization to logs
	FormatOptionLogs = sqlutil.FormatOptionLogs
	// FormatOptionsTrace sets the preferred visualization to trace
	FormatOptionTrace = sqlutil.FormatOptionTrace
	// FormatOptionMulti formats the query results as a timeseries using "LongToMulti"
	FormatOptionMulti = sqlutil.FormatOptionMulti
)

Deprecated: use the values in sqlutil directly instead

View Source 
const MockDataFolder = "/mock-data"

MockDataFolder is the default folder that will contain data files

Variables

View Source 
var (
	HeaderKey                 = "grafana-http-headers"
	ErrorParsingMacroBrackets = errors.New("failed to parse macro arguments (missing close bracket?)")
)
View Source 
var (
	// ErrorBadDatasource is returned if the data source could not be asserted to the correct type (this should basically never happen?)
	ErrorBadDatasource = errors.New("type assertion to datasource failed")
	// ErrorJSON is returned when json.Unmarshal fails
	ErrorJSON = errors.New("error unmarshaling query JSON the Query Model")
	// ErrorQuery is returned when the query could not complete / execute
	ErrorQuery = errors.New("error querying the database")
	// ErrorTimeout is returned if the query has timed out
	ErrorTimeout = errors.New("query timeout exceeded")
	// ErrorNoResults is returned if there were no results returned
	ErrorNoResults = errors.New("no results returned from query")
	// ErrorRowValidation is returned when SQL rows validation fails (e.g., connection issues, corrupt results)
	ErrorRowValidation = errors.New("SQL rows validation failed")
	// ErrorConnectionClosed is returned when the database connection is unexpectedly closed
	ErrorConnectionClosed = errors.New("database connection closed")
)
View Source 
var (
	PRIMARY_KEY_QUERY_STRING    = "SELECT primary_key FROM system.tables WHERE database='%s' AND table ='%s'"
	AD_HOC_KEY_QUERY            = "DESCRIBE %s"
	PRIMARY_KEY_NOT_FOUND_ERROR = backend.PluginError(errors.New("primary key not found"))
	KEYS_NOT_FOUND_ERROR        = backend.PluginError(errors.New("adHocFilter keys not found"))
)
View Source 
var Macros = map[string]MacroFunc{
	"adHocFilter":     AdHocFilterMacro,
	"conditionalAll":  Stub,
	"fromTime":        FromTimeFilter,
	"toTime":          ToTimeFilter,
	"fromTime_ms":     FromTimeFilterMs,
	"toTime_ms":       ToTimeFilterMs,
	"timeFilter":      TimeFilter,
	"timeFilter_ms":   TimeFilterMs,
	"dateFilter":      DateFilter,
	"dateTimeFilter":  DateTimeFilter,
	"dt":              DateTimeFilter,
	"timeInterval":    TimeInterval,
	"timeInterval_ms": TimeIntervalMs,
	"interval_s":      IntervalSeconds,
}

Macros is a map of all macro functions

Functions

func AdHocFilterMacro 

func AdHocFilterMacro(ctx context.Context, query *HDXQuery, params []string, pos parser.Pos, mdProvider *MetaDataProvider) (string, error)

AdHocFilterMacro implements the $__adHocFilter() macro

func CreateMockData 

func CreateMockData(table string, folder string, csvData string) error

CreateData will create a "table" (csv file) in the data folder that can be queried with SQL

func CreateMockTable 

func CreateMockTable(table string, folder string) error

Create will create a "table" (csv file) in the data folder that can be queried with SQL

func DateFilter 

func DateFilter(_ context.Context, query *HDXQuery, args []string, _ parser.Pos, _ *MetaDataProvider) (string, error)

func DateTimeFilter 

func DateTimeFilter(_ context.Context, query *HDXQuery, args []string, _ parser.Pos, _ *MetaDataProvider) (string, error)

func ErrorSource 

func ErrorSource(err error) backend.ErrorSource

func FromTimeFilter 

func FromTimeFilter(_ context.Context, query *HDXQuery, _ []string, _ parser.Pos, _ *MetaDataProvider) (string, error)

FromTimeFilter returns a time filter expression based on grafana's timepicker's "from" time in seconds

func FromTimeFilterMs 

func FromTimeFilterMs(_ context.Context, query *HDXQuery, _ []string, _ parser.Pos, _ *MetaDataProvider) (string, error)

FromTimeFilterMs returns a time filter expression based on grafana's timepicker's "from" time in milliseconds

func GetMacroCTEs 

func GetMacroCTEs(ast []parser.Expr) (map[MacroId]CTE, error)

func IntervalSeconds 

func IntervalSeconds(_ context.Context, query *HDXQuery, _ []string, _ parser.Pos, _ *MetaDataProvider) (string, error)

func RoundTimeRange 

func RoundTimeRange(timeRange backend.TimeRange, interval string) backend.TimeRange

RoundTimeRange rounds the time range to provided time interval

func Stub 

func Stub(_ context.Context, _ *HDXQuery, _ []string, _ parser.Pos, _ *MetaDataProvider) (string, error)

func TimeFilter 

func TimeFilter(context context.Context, query *HDXQuery, args []string, pos parser.Pos, mdProvider *MetaDataProvider) (string, error)

func TimeFilterMs 

func TimeFilterMs(context context.Context, query *HDXQuery, args []string, pos parser.Pos, mdProvider *MetaDataProvider) (string, error)

func TimeInterval 

func TimeInterval(context context.Context, query *HDXQuery, args []string, pos parser.Pos, mdProvider *MetaDataProvider) (string, error)

func TimeIntervalMs 

func TimeIntervalMs(context context.Context, query *HDXQuery, args []string, pos parser.Pos, mdProvider *MetaDataProvider) (string, error)

func ToTimeFilter 

func ToTimeFilter(_ context.Context, query *HDXQuery, _ []string, _ parser.Pos, _ *MetaDataProvider) (string, error)

ToTimeFilter returns a time filter expression based on grafana's timepicker's "to" time in seconds

func ToTimeFilterMs 

func ToTimeFilterMs(_ context.Context, query *HDXQuery, _ []string, _ parser.Pos, _ *MetaDataProvider) (string, error)

ToTimeFilterMs returns a time filter expression based on grafana's timepicker's "to" time in milliseconds

Types

type AdHocFilter 

type AdHocFilter struct {
	Key      string   `json:"key"`
	Operator string   `json:"operator"`
	Value    string   `json:"value"`
	Values   []string `json:"values,omitempty"`
}

type CTE 

type CTE struct {
	Macro    string     `json:"macro"`
	MacroPos parser.Pos `json:"macroPos"`
	CTE      string     `json:"cte"`
	Table    string     `json:"table"`
	Database string     `json:"database"`
	Pos      parser.Pos `json:"pos"`
}

type CheckHealthMutator 

type CheckHealthMutator interface {
	MutateCheckHealth(ctx context.Context, req *backend.CheckHealthRequest) (context.Context, *backend.CheckHealthRequest)
}

CheckHealthMutator is an additional interface that could be implemented by driver. This adds ability to the driver to optionally mutate the CheckHealth before it's run

type Connection 

type Connection interface {
	Close() error
	Ping() error
	PingContext(ctx context.Context) error
	QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
}

Connection represents a SQL connection and is satisfied by the *sql.DB type For now, we only add the functions that we need / actively use. Some other candidates for future use could include the ExecContext and BeginTxContext functions

type Connector 

type Connector interface {
	Connect(ctx context.Context, headers http.Header) (*dbConnection, error)

	Reconnect(ctx context.Context, dbConn dbConnection, q *sqlutil.Query, cacheKey string) (*sql.DB, error)

	Dispose()
	GetConnectionFromQuery(ctx context.Context, q *sqlutil.Query) (string, dbConnection, error)
	GetDriver() Driver
	GetUID() string
	// contains filtered or unexported methods
}

type DBQuery 

type DBQuery struct {
	DB Connection

	Settings backend.DataSourceInstanceSettings

	DSName string
	// contains filtered or unexported fields
}

func NewQuery 

func NewQuery(db Connection, settings backend.DataSourceInstanceSettings, converters []sqlutil.Converter, fillMode *data.FillMissing, rowLimit int64) *DBQuery

func (*DBQuery) Run 

func (q *DBQuery) Run(ctx context.Context, query *Query, queryErrorMutator QueryErrorMutator, args ...interface{}) (data.Frames, error)

Run sends the query to the connection and converts the rows to a dataframe.

type Driver 

type Driver interface {
	// Connect connects to the database. It does not need to call `db.Ping()`
	Connect(context.Context, backend.DataSourceInstanceSettings, json.RawMessage) (*sql.DB, error)
	// Settings are read whenever the plugin is initialized, or after the data source settings are updated
	Settings(context.Context, backend.DataSourceInstanceSettings) DriverSettings
	Converters() []sqlutil.Converter
}

Driver is a simple interface that defines how to connect to a backend SQL datasource Plugin creators will need to implement this in order to create a managed datasource

type DriverSettings 

type DriverSettings struct {
	FillMode       *data.FillMissing
	RetryOn        []string
	Timeout        time.Duration
	Retries        int
	Pause          int
	ForwardHeaders bool
	Errors         bool
	RowLimit       int64
}

type Endpoint 

type Endpoint string

type FormatQueryOption 

type FormatQueryOption = sqlutil.FormatQueryOption

FormatQueryOption defines how the user has chosen to represent the data Deprecated: use sqlutil.FormatQueryOption directly instead

type HDXQuery 

type HDXQuery struct {
	RawSQL        string                `json:"rawSql"`
	Format        int                   `json:"format"`
	Round         string                `json:"round,omitempty"`
	QuerySettings []models.QuerySetting `json:"querySettings,omitempty"`
	Filters       []AdHocFilter         `json:"filters,omitempty"`
	Meta          struct {
		TimeZone string `json:"timezone"`
	} `json:"meta"`
	TimeRange backend.TimeRange `json:"-"`
	Interval  time.Duration     `json:"-"`
	Headers   http.Header       `json:"-"`
}

func GetHdxQuery 

func GetHdxQuery(query backend.DataQuery, headers http.Header, timeRange *backend.TimeRange, interval *time.Duration) (*HDXQuery, error)

func (*HDXQuery) WithSQL 

func (q *HDXQuery) WithSQL(rawSql string) *HDXQuery

type HealthChecker 

type HealthChecker struct {
	Connector       Connector
	Metrics         Metrics
	PreCheckHealth  func(ctx context.Context, req *backend.CheckHealthRequest) *backend.CheckHealthResult
	PostCheckHealth func(ctx context.Context, req *backend.CheckHealthRequest) *backend.CheckHealthResult
}

func (*HealthChecker) Check 

func (hc *HealthChecker) Check(ctx context.Context, req *backend.CheckHealthRequest) (*backend.CheckHealthResult, error)

type HydrolixConnector 

type HydrolixConnector struct {
	UID string

	Driver Driver
	// contains filtered or unexported fields
}

func NewConnector 

func NewConnector(ctx context.Context, driver Driver, settings backend.DataSourceInstanceSettings) (*HydrolixConnector, error)

func (*HydrolixConnector) Connect 

func (c *HydrolixConnector) Connect(ctx context.Context, headers http.Header) (*dbConnection, error)

func (*HydrolixConnector) Dispose 

func (c *HydrolixConnector) Dispose()

Dispose is called when an existing SQLDatasource needs to be replaced

func (*HydrolixConnector) GetConnectionFromQuery 

func (c *HydrolixConnector) GetConnectionFromQuery(ctx context.Context, q *sqlutil.Query) (string, dbConnection, error)

func (*HydrolixConnector) GetDriver 

func (c *HydrolixConnector) GetDriver() Driver

func (*HydrolixConnector) GetUID 

func (c *HydrolixConnector) GetUID() string

func (*HydrolixConnector) Reconnect 

func (c *HydrolixConnector) Reconnect(ctx context.Context, dbConn dbConnection, q *sqlutil.Query, cacheKey string) (*sql.DB, error)

type HydrolixDatasource 

type HydrolixDatasource struct {
	backend.CallResourceHandler
	Connector    Connector
	ID           string
	Interpolator Interpolator

	// EnableRowLimit: enables using the dataproxy.row_limit setting to limit the number of rows returned by the query
	// https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#row_limit
	EnableRowLimit bool
	// contains filtered or unexported fields
}

func (*HydrolixDatasource) CheckHealth 

func (ds *HydrolixDatasource) CheckHealth(ctx context.Context, req *backend.CheckHealthRequest) (*backend.CheckHealthResult, error)

CheckHealth pings the connected SQL database

func (*HydrolixDatasource) Dispose 

func (ds *HydrolixDatasource) Dispose()

Dispose cleans up datasource instance resources. Note: Called when testing and saving a datasource

func (*HydrolixDatasource) DriverSettings 

func (ds *HydrolixDatasource) DriverSettings() DriverSettings

func (*HydrolixDatasource) GetDBFromQuery 

func (ds *HydrolixDatasource) GetDBFromQuery(ctx context.Context, q *sqlutil.Query) (*sql.DB, error)

func (*HydrolixDatasource) GetRowLimit 

func (ds *HydrolixDatasource) GetRowLimit() int64

func (*HydrolixDatasource) NewDatasource 

func (ds *HydrolixDatasource) NewDatasource(ctx context.Context, settings backend.DataSourceInstanceSettings) (instancemgmt.Instance, error)

NewDatasource creates a new `SQLDatasource`. It uses the provided settings argument to call the ds.Driver to connect to the SQL server

func (*HydrolixDatasource) QueryData 

func (ds *HydrolixDatasource) QueryData(ctx context.Context, req *backend.QueryDataRequest) (*backend.QueryDataResponse, error)

QueryData creates the Responses list and executes each query

func (*HydrolixDatasource) RegisterRoutes 

func (ds *HydrolixDatasource) RegisterRoutes(customRoutes map[string]func(http.ResponseWriter, *http.Request))

func (*HydrolixDatasource) SetDefaultRowLimit 

func (ds *HydrolixDatasource) SetDefaultRowLimit(limit int64)

type InterpolatedQueryMutator added in v5.0.1 

type InterpolatedQueryMutator interface {
	MutateInterpolatedQuery(ctx context.Context, sql string) (context.Context, string)
}

InterpolatedQueryMutator is an additional interface that could be implemented by driver. It mutates the fully interpolated SQL — after Interpolator.Interpolate has expanded all macros — before it reaches the underlying database driver. Use this for rewrites that depend on the final SQL shape (e.g. inspecting a user-supplied SETTINGS clause whose presence the driver cannot observe in MutateQuery because the macros have not yet been expanded).

type Interpolator 

type Interpolator struct {
	// contains filtered or unexported fields
}

func NewInterpolator 

func NewInterpolator(ds *HydrolixDatasource) Interpolator

func (Interpolator) Interpolate 

func (i Interpolator) Interpolate(ctx context.Context, query *HDXQuery) (string, error)

Interpolate returns an interpolated query string given a backend.DataQuery

type MacroFunc 

type MacroFunc func(context.Context, *HDXQuery, []string, parser.Pos, *MetaDataProvider) (string, error)

type MacroId 

type MacroId struct {
	Name  string     `json:"name"`
	Index parser.Pos `json:"index"`
}

type MetaDataProvider 

type MetaDataProvider struct {
	// contains filtered or unexported fields
}

func NewMetaDataProvider 

func NewMetaDataProvider(ds *HydrolixDatasource) *MetaDataProvider

func (*MetaDataProvider) GetKeys 

func (p *MetaDataProvider) GetKeys(context context.Context, headers http.Header, cte string) (map[string]string, error)

func (*MetaDataProvider) GetPK 

func (p *MetaDataProvider) GetPK(context context.Context, headers http.Header, database string, table string) (string, error)

func (*MetaDataProvider) GetStringSafe 

func (p *MetaDataProvider) GetStringSafe(v any) (string, error)

func (*MetaDataProvider) QueryKeys 

func (p *MetaDataProvider) QueryKeys(ctx context.Context, headers http.Header, cte string) (map[string]string, error)

func (*MetaDataProvider) QueryPK 

func (p *MetaDataProvider) QueryPK(ctx context.Context, headers http.Header, database string, table string) (string, error)

type Metrics 

type Metrics struct {
	DSName   string
	DSType   string
	Endpoint Endpoint
}

func NewMetrics 

func NewMetrics(dsName, dsType string, endpoint Endpoint) Metrics

func (*Metrics) CollectDuration 

func (m *Metrics) CollectDuration(source Source, status Status, duration float64)

func (*Metrics) WithEndpoint 

func (m *Metrics) WithEndpoint(endpoint Endpoint) Metrics

type Query deprecated 

type Query = sqlutil.Query

Deprecated: use sqlutil.Query directly instead

func GetQuery 

func GetQuery(query backend.DataQuery, headers http.Header, setHeaders bool) (*Query, error)

GetQuery wraps sqlutil's GetQuery to add headers if needed

type QueryArgSetter 

type QueryArgSetter interface {
	SetQueryArgs(ctx context.Context, headers http.Header) []interface{}
}

QueryArgSetter is an additional interface that could be implemented by driver. This adds the ability to the driver to optionally set query args that are then sent down to the database.

type QueryDataMutator 

type QueryDataMutator interface {
	MutateQueryData(ctx context.Context, req *backend.QueryDataRequest) (context.Context, *backend.QueryDataRequest)
}

QueryDataMutator is an additional interface that could be implemented by driver. This adds ability to the driver to optionally mutate the query before it's run with the QueryDataRequest.

type QueryErrorMutator 

type QueryErrorMutator interface {
	MutateQueryError(err error) backend.ErrorWithSource
}

type QueryMutator 

type QueryMutator interface {
	MutateQuery(ctx context.Context, req backend.DataQuery) (context.Context, backend.DataQuery)
}

QueryMutator is an additional interface that could be implemented by driver. This adds ability to the driver it can mutate query before run.

type Response 

type Response struct {
	// contains filtered or unexported fields
}

func NewResponse 

func NewResponse(res *backend.QueryDataResponse) *Response

func (*Response) Response 

func (r *Response) Response() *backend.QueryDataResponse

func (*Response) Set 

func (r *Response) Set(refID string, res backend.DataResponse)

type ResponseMutator 

type ResponseMutator interface {
	MutateResponse(ctx context.Context, res data.Frames) (data.Frames, error)
}

ResponseMutator is an additional interface that could be implemented by driver. This adds ability to the driver, so it can mutate a response from the driver before its returned to the client.

type SQLMock 

type SQLMock struct {
	ShouldFailToConnect bool
	// contains filtered or unexported fields
}

SQLMock connects to a local folder with csv files

func (*SQLMock) Connect 

func (h *SQLMock) Connect(_ context.Context, _ backend.DataSourceInstanceSettings, msg json.RawMessage) (*sql.DB, error)

Connect opens a sql.DB connection using datasource settings

func (*SQLMock) Converters 

func (h *SQLMock) Converters() []sqlutil.Converter

Converters defines list of string convertors

func (*SQLMock) Settings 

func (h *SQLMock) Settings(_ context.Context, _ backend.DataSourceInstanceSettings) DriverSettings

type Source 

type Source string

type Status 

type Status string

Source Files

View all Source files

Directories

Path Synopsis
csv
Package models provides Hydrolix plugin's configuration settings
 Package models provides Hydrolix plugin's configuration settings

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.