gocqlx

package module
v2.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 30, 2022 License: Apache-2.0 Imports: 8 Imported by: 145

README

🚀 GocqlX GoDoc Go Report Card Build Status

GocqlX makes working with Scylla easy and less error-prone. It’s inspired by Sqlx, a tool for working with SQL databases, but it goes beyond what Sqlx provides.

Features

  • Binding query parameters from struct fields, map, or both
  • Scanning query results into structs based on field names
  • Convenient functions for common tasks such as loading a single row into a struct or all rows into a slice (list) of structs
  • Making any struct a UDT without implementing marshalling functions
  • GocqlX is fast. Its performance is comparable to raw driver. You can find some benchmarks here.

Subpackages provide additional functionality:

Installation

    go get -u github.com/scylladb/gocqlx/v2

Getting started

Wrap gocql Session:

// Create gocql cluster.
cluster := gocql.NewCluster(hosts...)
// Wrap session on creation, gocqlx session embeds gocql.Session pointer. 
session, err := gocqlx.WrapSession(cluster.CreateSession())
if err != nil {
	t.Fatal(err)
}

Specify table model:

// metadata specifies table name and columns it must be in sync with schema.
var personMetadata = table.Metadata{
	Name:    "person",
	Columns: []string{"first_name", "last_name", "email"},
	PartKey: []string{"first_name"},
	SortKey: []string{"last_name"},
}

// personTable allows for simple CRUD operations based on personMetadata.
var personTable = table.New(personMetadata)

// Person represents a row in person table.
// Field names are converted to camel case by default, no need to add special tags.
// A field will not be persisted by adding the `db:"-"` tag or making it unexported.
type Person struct {
	FirstName string
	LastName  string
	Email     []string
	HairColor string `db:"-"`  // exported and skipped
	eyeColor  string           // unexported also skipped
}

Bind data from a struct and insert a row:

p := Person{
	"Michał",
	"Matczuk",
	[]string{"michal@scylladb.com"},
	"red",    // not persisted
	"hazel"   // not persisted
}
q := session.Query(personTable.Insert()).BindStruct(p)
if err := q.ExecRelease(); err != nil {
	t.Fatal(err)
}

Load a single row to a struct:

p := Person{
	"Michał",
	"Matczuk",
	nil, // no email
}
q := session.Query(personTable.Get()).BindStruct(p)
if err := q.GetRelease(&p); err != nil {
	t.Fatal(err)
}
t.Log(p)
// stdout: {Michał Matczuk [michal@scylladb.com]}

Load all rows in to a slice:

var people []Person
q := session.Query(personTable.Select()).BindMap(qb.M{"first_name": "Michał"})
if err := q.SelectRelease(&people); err != nil {
	t.Fatal(err)
}
t.Log(people)
// stdout: [{Michał Matczuk [michal@scylladb.com]}]

Generating table metadata with schemagen

Installation

go get -u "github.com/scylladb/gocqlx/v2/cmd/schemagen"

Usage:

$GOBIN/schemagen [flags]

Flags:
  -cluster string
    	a comma-separated list of host:port tuples (default "127.0.0.1")
  -keyspace string
    	keyspace to inspect (required)
  -output string
    	the name of the folder to output to (default "models")
  -pkgname string
    	the name you wish to assign to your generated package (default "models")

Example:

Running the following command for examples keyspace:

$GOBIN/schemagen -cluster="127.0.0.1:9042" -keyspace="examples" -output="models" -pkgname="models"

Generates models/models.go as follows:

// Code generated by "gocqlx/cmd/schemagen"; DO NOT EDIT.

package models

import "github.com/scylladb/gocqlx/v2/table"

// Table models.
var (
	Playlists = table.New(table.Metadata{
		Name: "playlists",
		Columns: []string{
			"album",
			"artist",
			"id",
			"song_id",
			"title",
		},
		PartKey: []string{
			"id",
		},
		SortKey: []string{
			"title",
			"album",
			"artist",
		},
	})

	Songs = table.New(table.Metadata{
		Name: "songs",
		Columns: []string{
			"album",
			"artist",
			"data",
			"id",
			"tags",
			"title",
		},
		PartKey: []string{
			"id",
		},
		SortKey: []string{},
	})
)

Examples

You can find lots of examples in example_test.go.

Go and run them locally:

make run-scylla
make run-examples

Performance

GocqlX performance is comparable to the raw gocql driver. Below benchmark results running on my laptop.

BenchmarkBaseGocqlInsert            2392            427491 ns/op            7804 B/op         39 allocs/op
BenchmarkGocqlxInsert               2479            435995 ns/op            7803 B/op         39 allocs/op
BenchmarkBaseGocqlGet               2853            452384 ns/op            7309 B/op         35 allocs/op
BenchmarkGocqlxGet                  2706            442645 ns/op            7646 B/op         38 allocs/op
BenchmarkBaseGocqlSelect             747           1664365 ns/op           49415 B/op        927 allocs/op
BenchmarkGocqlxSelect                667           1877859 ns/op           42521 B/op        932 allocs/op

See the benchmark in benchmark_test.go.

License

Copyright (C) 2017 ScyllaDB

This project is distributed under the Apache 2.0 license. See the LICENSE file for details. It contains software from:

Apache®, Apache Cassandra® are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.

GitHub star is always appreciated!

Documentation

Overview

Package gocqlx makes working with Scylla easy and error prone without sacrificing performance. It’s inspired by Sqlx, a tool for working with SQL databases, but it goes beyond what Sqlx provides.

For more details consult README.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var DefaultMapper = reflectx.NewMapperFunc("db", reflectx.CamelToSnakeASCII)

DefaultMapper uses `db` tag and automatically converts struct field names to snake case. It can be set to whatever you want, but it is encouraged to be set before gocqlx is used as name-to-field mappings are cached after first use on a type.

A custom mapper can always be set per Sessionm, Query and Iter.

View Source 
var DefaultUnsafe bool

DefaultUnsafe enables the behavior of forcing the iterator to ignore missing fields for all queries. See Unsafe below for more information.

View Source 
var UnsetEmptyTransformer = func(name string, val interface{}) interface{} {
	v := reflect.ValueOf(val)
	if v.IsZero() {
		return gocql.UnsetValue
	}
	return val
}

UnsetEmptyTransformer unsets all empty parameters. It helps to avoid tombstones when using the same insert/update statement for filled and partially filled named parameters.

Functions

func CompileNamedQuery 

func CompileNamedQuery(qs []byte) (stmt string, names []string, err error)

CompileNamedQuery translates query with named parameters in a form ':<identifier>' to query with '?' placeholders and a list of parameter names. If you need to use ':' in a query, i.e. with maps or UDTs use '::' instead.

func CompileNamedQueryString 

func CompileNamedQueryString(qs string) (stmt string, names []string, err error)

CompileNamedQueryString translates query with named parameters in a form ':<identifier>' to query with '?' placeholders and a list of parameter names. If you need to use ':' in a query, i.e. with maps or UDTs use '::' instead.

Types

type Iterx 

type Iterx struct {
	*gocql.Iter
	Mapper *reflectx.Mapper
	// contains filtered or unexported fields
}

Iterx is a wrapper around gocql.Iter which adds struct scanning capabilities.

func (*Iterx) Close 

func (iter *Iterx) Close() error

Close closes the iterator and returns any errors that happened during the query or the iteration.

func (*Iterx) Get 

func (iter *Iterx) Get(dest interface{}) error

Get scans first row into a destination and closes the iterator.

If the destination type is a struct pointer, then StructScan will be used. If the destination is some other type, then the row must only have one column which can scan into that type. This includes types that implement gocql.Unmarshaler and gocql.UDTUnmarshaler.

If you'd like to treat a type that implements gocql.Unmarshaler or gocql.UDTUnmarshaler as an ordinary struct you should call StructOnly().Get(dest) instead.

If no rows were selected, ErrNotFound is returned.

func (*Iterx) Scan 

func (iter *Iterx) Scan(dest ...interface{}) bool

Scan consumes the next row of the iterator and copies the columns of the current row into the values pointed at by dest. Use nil as a dest value to skip the corresponding column. Scan might send additional queries to the database to retrieve the next set of rows if paging was enabled.

Scan returns true if the row was successfully unmarshaled or false if the end of the result set was reached or if an error occurred. Close should be called afterwards to retrieve any potential errors.

func (*Iterx) Select 

func (iter *Iterx) Select(dest interface{}) error

Select scans all rows into a destination, which must be a pointer to slice of any type, and closes the iterator.

If the destination slice type is a struct, then StructScan will be used on each row. If the destination is some other type, then each row must only have one column which can scan into that type. This includes types that implement gocql.Unmarshaler and gocql.UDTUnmarshaler.

If you'd like to treat a type that implements gocql.Unmarshaler or gocql.UDTUnmarshaler as an ordinary struct you should call StructOnly().Select(dest) instead.

If no rows were selected, ErrNotFound is NOT returned.

func (*Iterx) StructOnly 

func (iter *Iterx) StructOnly() *Iterx

StructOnly forces the iterator to treat a single-argument struct as non-scannable. This is is useful if you need to scan a row into a struct that also implements gocql.UDTUnmarshaler or in rare cases gocql.Unmarshaler.

func (*Iterx) StructScan 

func (iter *Iterx) StructScan(dest interface{}) bool

StructScan is like gocql.Iter.Scan, but scans a single row into a single struct. Use this and iterate manually when the memory load of Select() might be prohibitive. StructScan caches the reflect work of matching up column positions to fields to avoid that overhead per scan, which means it is not safe to run StructScan on the same Iterx instance with different struct types.

func (*Iterx) Unsafe 

func (iter *Iterx) Unsafe() *Iterx

Unsafe forces the iterator to ignore missing fields. By default when scanning a struct if result row has a column that cannot be mapped to any destination field an error is reported. With unsafe such columns are ignored.

type Queryx 

type Queryx struct {
	*gocql.Query
	Names  []string
	Mapper *reflectx.Mapper
	// contains filtered or unexported fields
}

Queryx is a wrapper around gocql.Query which adds struct binding capabilities.

func Query deprecated 

func Query(q *gocql.Query, names []string) *Queryx

Query creates a new Queryx from gocql.Query using a default mapper.

Deprecated: Use gocqlx.Session.Query API instead.

func (*Queryx) Bind 

func (q *Queryx) Bind(v ...interface{}) *Queryx

Bind sets query arguments of query. This can also be used to rebind new query arguments to an existing query instance.

func (*Queryx) BindMap 

func (q *Queryx) BindMap(arg map[string]interface{}) *Queryx

BindMap binds query named parameters using map.

func (*Queryx) BindStruct 

func (q *Queryx) BindStruct(arg interface{}) *Queryx

BindStruct binds query named parameters to values from arg using mapper. If value cannot be found error is reported.

func (*Queryx) BindStructMap 

func (q *Queryx) BindStructMap(arg0 interface{}, arg1 map[string]interface{}) *Queryx

BindStructMap binds query named parameters to values from arg0 and arg1 using a mapper. If value cannot be found in arg0 it's looked up in arg1 before reporting an error.

func (*Queryx) Consistency 

func (q *Queryx) Consistency(c gocql.Consistency) *Queryx

Consistency sets the consistency level for this query. If no consistency level have been set, the default consistency level of the cluster is used.

func (*Queryx) CustomPayload 

func (q *Queryx) CustomPayload(customPayload map[string][]byte) *Queryx

CustomPayload sets the custom payload level for this query.

func (*Queryx) DefaultTimestamp 

func (q *Queryx) DefaultTimestamp(enable bool) *Queryx

DefaultTimestamp will enable the with default timestamp flag on the query. If enable, this will replace the server side assigned timestamp as default timestamp. Note that a timestamp in the query itself will still override this timestamp. This is entirely optional.

Only available on protocol >= 3

func (*Queryx) Err 

func (q *Queryx) Err() error

Err returns any binding errors.

func (*Queryx) Exec 

func (q *Queryx) Exec() error

Exec executes the query without returning any rows.

func (*Queryx) ExecCAS 

func (q *Queryx) ExecCAS() (applied bool, err error)

ExecCAS executes the Lightweight Transaction query, returns whether query was applied. See: https://docs.scylladb.com/using-scylla/lwt/ for more details.

When using Cassandra it may be necessary to use NoSkipMetadata in order to obtain an accurate "applied" value. See the documentation of NoSkipMetaData method on this page for more details.

func (*Queryx) ExecCASRelease 

func (q *Queryx) ExecCASRelease() (bool, error)

ExecCASRelease calls ExecCAS and releases the query, a released query cannot be reused.

func (*Queryx) ExecRelease 

func (q *Queryx) ExecRelease() error

ExecRelease calls Exec and releases the query, a released query cannot be reused.

func (*Queryx) Get 

func (q *Queryx) Get(dest interface{}) error

Get scans first row into a destination and closes the iterator.

If the destination type is a struct pointer, then Iter.StructScan will be used. If the destination is some other type, then the row must only have one column which can scan into that type. This includes types that implement gocql.Unmarshaler and gocql.UDTUnmarshaler.

If you'd like to treat a type that implements gocql.Unmarshaler or gocql.UDTUnmarshaler as an ordinary struct you should call Iter().StructOnly().Get(dest) instead.

If no rows were selected, ErrNotFound is returned.

func (*Queryx) GetCAS 

func (q *Queryx) GetCAS(dest interface{}) (applied bool, err error)

GetCAS executes a lightweight transaction. If the transaction fails because the existing values did not match, the previous values will be stored in dest object. See: https://docs.scylladb.com/using-scylla/lwt/ for more details.

func (*Queryx) GetCASRelease 

func (q *Queryx) GetCASRelease(dest interface{}) (bool, error)

GetCASRelease calls GetCAS and releases the query, a released query cannot be reused.

func (*Queryx) GetRelease 

func (q *Queryx) GetRelease(dest interface{}) error

GetRelease calls Get and releases the query, a released query cannot be reused.

func (*Queryx) Idempotent 

func (q *Queryx) Idempotent(value bool) *Queryx

Idempotent marks the query as being idempotent or not depending on the value.

func (*Queryx) Iter 

func (q *Queryx) Iter() *Iterx

Iter returns Iterx instance for the query. It should be used when data is too big to be loaded with Select in order to do row by row iteration. See Iterx StructScan function.

func (*Queryx) NoSkipMetadata 

func (q *Queryx) NoSkipMetadata() *Queryx

NoSkipMetadata will override the internal result metadata cache so that the driver does not send skip_metadata for queries, this means that the result will always contain the metadata to parse the rows and will not reuse the metadata from the prepared staement. This should only be used to work around cassandra bugs, such as when using CAS operations which do not end in Cas.

See https://issues.apache.org/jira/browse/CASSANDRA-11099 https://github.com/gocql/gocql/issues/612

func (*Queryx) Observer 

func (q *Queryx) Observer(observer gocql.QueryObserver) *Queryx

Observer enables query-level observer on this query. The provided observer will be called every time this query is executed.

func (*Queryx) PageSize 

func (q *Queryx) PageSize(n int) *Queryx

PageSize will tell the iterator to fetch the result in pages of size n. This is useful for iterating over large result sets, but setting the page size too low might decrease the performance. This feature is only available in Cassandra 2 and onwards.

func (*Queryx) PageState 

func (q *Queryx) PageState(state []byte) *Queryx

PageState sets the paging state for the query to resume paging from a specific point in time. Setting this will disable to query paging for this query, and must be used for all subsequent pages.

func (*Queryx) Prefetch 

func (q *Queryx) Prefetch(p float64) *Queryx

Prefetch sets the default threshold for pre-fetching new pages. If there are only p*pageSize rows remaining, the next page will be requested automatically.

func (*Queryx) RetryPolicy 

func (q *Queryx) RetryPolicy(r gocql.RetryPolicy) *Queryx

RetryPolicy sets the policy to use when retrying the query.

func (*Queryx) RoutingKey 

func (q *Queryx) RoutingKey(routingKey []byte) *Queryx

RoutingKey sets the routing key to use when a token aware connection pool is used to optimize the routing of this query.

func (*Queryx) Select 

func (q *Queryx) Select(dest interface{}) error

Select scans all rows into a destination, which must be a pointer to slice of any type, and closes the iterator.

If the destination slice type is a struct, then Iter.StructScan will be used on each row. If the destination is some other type, then each row must only have one column which can scan into that type. This includes types that implement gocql.Unmarshaler and gocql.UDTUnmarshaler.

If you'd like to treat a type that implements gocql.Unmarshaler or gocql.UDTUnmarshaler as an ordinary struct you should call Iter().StructOnly().Select(dest) instead.

If no rows were selected, ErrNotFound is NOT returned.

func (*Queryx) SelectRelease 

func (q *Queryx) SelectRelease(dest interface{}) error

SelectRelease calls Select and releases the query, a released query cannot be reused.

func (*Queryx) SerialConsistency 

func (q *Queryx) SerialConsistency(cons gocql.SerialConsistency) *Queryx

SerialConsistency sets the consistency level for the serial phase of conditional updates. That consistency can only be either SERIAL or LOCAL_SERIAL and if not present, it defaults to SERIAL. This option will be ignored for anything else that a conditional update/insert.

func (*Queryx) SetSpeculativeExecutionPolicy 

func (q *Queryx) SetSpeculativeExecutionPolicy(sp gocql.SpeculativeExecutionPolicy) *Queryx

SetSpeculativeExecutionPolicy sets the execution policy.

func (*Queryx) Trace 

func (q *Queryx) Trace(trace gocql.Tracer) *Queryx

Trace enables tracing of this query. Look at the documentation of the Tracer interface to learn more about tracing.

func (*Queryx) WithBindTransformer added in v2.6.0 

func (q *Queryx) WithBindTransformer(tr Transformer) *Queryx

WithBindTransformer sets the query bind transformer. The transformer is called right before binding a value to a named parameter.

func (*Queryx) WithContext 

func (q *Queryx) WithContext(ctx context.Context) *Queryx

WithContext returns a shallow copy of q with its context set to ctx.

The provided context controls the entire lifetime of executing a query, queries will be canceled and return once the context is canceled.

func (*Queryx) WithTimestamp 

func (q *Queryx) WithTimestamp(timestamp int64) *Queryx

WithTimestamp will enable the with default timestamp flag on the query like DefaultTimestamp does. But also allows to define value for timestamp. It works the same way as USING TIMESTAMP in the query itself, but should not break prepared query optimization

Only available on protocol >= 3

type Session 

type Session struct {
	*gocql.Session
	Mapper *reflectx.Mapper
}

Session wraps gocql.Session and provides a modified Query function that returns Queryx instance. The original Session instance can be accessed as Session. The default mapper uses `db` tag and automatically converts struct field names to snake case. If needed package reflectx provides constructors for other types of mappers.

Example 
cluster := gocql.NewCluster("host")
session, err := gocqlx.WrapSession(cluster.CreateSession())
if err != nil {
	// handle error
}

builder := qb.Select("foo")
session.Query(builder.ToCql())

Output:

func NewSession added in v2.0.3 

func NewSession(session *gocql.Session) Session

NewSession wraps existing gocql.session.

func WrapSession 

func WrapSession(session *gocql.Session, err error) (Session, error)

WrapSession should be called on CreateSession() gocql function to convert the created session to gocqlx.Session.

Example: 

session, err := gocqlx.WrapSession(cluster.CreateSession())

func (Session) ContextQuery 

func (s Session) ContextQuery(ctx context.Context, stmt string, names []string) *Queryx

ContextQuery is a helper function that allows to pass context when creating a query, see the "Query" function .

func (Session) ExecStmt 

func (s Session) ExecStmt(stmt string) error

ExecStmt creates query and executes the given statement.

func (Session) Query 

func (s Session) Query(stmt string, names []string) *Queryx

Query creates a new Queryx using the session mapper. The stmt and names parameters are typically result of a query builder (package qb) ToCql() function or come from table model (package table). The names parameter is a list of query parameters' names and it's used for binding.

type Transformer added in v2.6.0 

type Transformer func(name string, val interface{}) interface{}

Transformer transforms the value of the named parameter to another value. 

var DefaultBindTransformer Transformer

DefaultBindTransformer just do nothing.

A custom transformer can always be set per Query.

type UDT 

type UDT interface {
	// contains filtered or unexported methods
}

UDT is a marker interface that needs to be embedded in a struct if you want to marshal or unmarshal it as a User Defined Type.

Example 
// Just add gocqlx.UDT to a type, no need to implement marshalling functions
type FullName struct {
	gocqlx.UDT
	FirstName string
	LastName  string
}

Output:
Example (Wraper) 
type FullName struct {
	FirstName string
	LastName  string
}

// Create new UDT wrapper type
type FullNameUDT struct {
	gocqlx.UDT
	*FullName
}

Output:

Source Files

View all Source files

Directories

Path Synopsis
cmd
schemagen
Package dbutil provides various utilities built on top of core gocqlx modules.
 Package dbutil provides various utilities built on top of core gocqlx modules.
Package gocqlxtest provides test helpers for integration tests.
 Package gocqlxtest provides test helpers for integration tests.
Package migrate reads migrations from a flat directory containing CQL files.
 Package migrate reads migrations from a flat directory containing CQL files.
Package qb provides CQL query builders.
 Package qb provides CQL query builders.
Package table adds support for super simple CRUD operations based on table model.
 Package table adds support for super simple CRUD operations based on table model.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.