bolt

README

Neo4J Bolt Driver

Tested against Go 1.6 and up

Implements the Neo4J Bolt Protocol specification: As of the time of writing this, the current version is v3.1.0-M02

go get github.com/SermoDigital/bolt

Features

• Neo4j Bolt low-level binary protocol support
• Connection Pooling
• TLS support
• Compatible with sql.driver

TODO

• Message Pipelining for high concurrency

Usage

Please see the statement tests or the conn tests for A LOT of examples of usage

API

There is much more detailed information in the godoc

This implementation attempts to follow the best practices as per the Bolt specification, but also implements compatibility with Go's sql.driver interface.

As such, these interfaces closely match the sql.driver interfaces, but they also provide Neo4j Bolt specific functionality in addition to the sql.driver interface.

It is recommended that you use the Neo4j Bolt-specific interfaces if possible. The implementation is more efficient and can more closely support the Neo4j Bolt feature set.

The connection URI format is: bolt://[user[:password]]@[host][:port][?param!=value1&...] Schema must be bolt. User and password is only necessary if you are authenticating. Parameters are as follows:

• dial_timeout: Timeout for dialing a new connection in seconds.
• timeout: Read and write timeout in seconds.
• tls: Should the connection use TLS? 1 or 0.
• tls_ca_cert_file: Path to CA certificate file.
• tls_cert_file: Path to certificate file.
• tls_key_file: Path to key file.
• tls_no_verify: Should the connection not verify TLS? 1 or 0.

Additionally, environment variables can be used, although URI parameters will take precedence over envirnment variables. In the same order as above:

• BOLT_DRIVER_HOST
• BOLT_DRIVER_PORT
• BOLT_DRIVER_USER
• BOLT_DRIVER_PASS
• BOLT_DRIVER_TLS
• BOLT_DRIVER_TLS_CA_CERT_FILE
• BOLT_DRIVER_TLS_CERT_FILE
• BOLT_DRIVER_TLS_KEY_FILE
• BOLT_DRIVER_NO_VERIFY

Connection pooling is provided out of the box with the OpenPool function. You can give it the maximum number of connections to have at a time.

Dev Quickstart

# Put in git hooks
ln -s ../../scripts/pre-commit .git/hooks/pre-commit
ln -s ../../scripts/pre-push .git/hooks/pre-push

# No special build steps necessary
go build

# Testing with log info and a local bolt DB, getting coverage output
NEO4J_BOLT=bolt://localhost:7687 go test -coverprofile=./tmp/cover.out -coverpkg=./... -v -race && go tool cover -html=./tmp/cover.out

# Testing with trace output for debugging
NEO4J_BOLT=bolt://localhost:7687 go test -v -race

# Testing with running recorder to record tests for CI
NEO4J_BOLT=bolt://localhost:7687 RECORD_OUTPUT=1 go test -v -race

The tests are written in an integration testing style. Most of them are in the statement tests, but should be made more granular in the future.

In order to get CI, there's a recorder mechanism so you don't need to run Neo4j alongside the tests in the CI server. You run the tests locally against a Neo4j instance with the RECORD_OUTPUT=1 environment variable, it generates the recordings in the ./recordings folder. This is necessary if the tests have changed, or if the internals have significantly changed. Installing the git hooks will run the tests automatically on push. If there are updated tests, you will need to re-run the recorder to add them and push them as well.

You need access to a running Neo4J database to develop for this project, so you can run the tests to generate the recordings.

TODO

• Cypher Parser to implement NumInput and pre-flight checking
• More Tests
• Benchmark Tests

Documentation

Overview

Package bolt implements a driver for Neo4J's Bolt Protocol.

This package provides one standard driver and one "recording" driver. The standard driver is simply

bolt

It should be used in most circumstances.

The Recorder type implements driver.Driver and can be used to read from or create a recorded session. See Recorder's documentation for more information.

There are three main ways to query the database, checked in the following order:

1.) The first argument to any Query, Exec., etc is of the type Map and
    no other arguments are passed. This is the easiest and cleanest option.

2.) Every argument is of the type sql.NamedArg. Each argument will be
    interpreted as a a key-value pair. If the key == "", an error will be
    returned.

3.) An even number of arguments in key-value order, meaning the
    even-indexed values must be of the type string.

The connection URI format is:

bolt://[user[:password]]@[host][:port][?param1=value1&...]

Parameters are as follows:

• dial_timeout: Timeout for dialing a new connection in seconds.
• timeout: Read and write timeout in seconds.
• tls: Should the connection use TLS? 1 or 0.
• tls_ca_cert_file: Path to CA certificate file.
• tls_cert_file: Path to certificate file.
• tls_key_file: Path to key file.
• tls_no_verify: Should the connection _not_ verify TLS? 1 or 0.

Eenvironment variables can be used, although URI parameters will take precedence over envirnment variables. In the same order as above:

• BOLT_DRIVER_HOST
• BOLT_DRIVER_PORT
• BOLT_DRIVER_USER
• BOLT_DRIVER_PASS
• BOLT_DRIVER_TLS
• BOLT_DRIVER_TLS_CA_CERT_FILE
• BOLT_DRIVER_TLS_CERT_FILE
• BOLT_DRIVER_TLS_KEY_FILE
• BOLT_DRIVER_NO_VERIFY

Index

• Constants
• Variables
• func DialOpen(d Dialer, name string) (driver.Conn, error)
• func Open(name string) (driver.Conn, error)
• func WithSummary(ctx context.Context) (context.Context, func() *Summary)
• type Array
  • func (a *Array) Scan(val interface{}) error
• type Counters
  • func (c Counters) RowsAffected() (int64, error)
• type Dialer
  • func TLSDialer(caFile, certFile, keyFile string, noVerify bool) (Dialer, error)
• type Event
• type Map
  • func (m *Map) Scan(val interface{}) error
• type Notification
• type Plan
• type Position
• type Profile
• type Recorder
  • func (r *Recorder) Close() error
  • func (r *Recorder) Open(name string) (driver.Conn, error)
  • func (r *Recorder) Read(p []byte) (n int, err error)
  • func (r *Recorder) Write(b []byte) (n int, err error)
• type ServerInfo
• type Summary
• type Type
  • func (t *Type) SetString(s string)
  • func (t Type) String() string
• type UnrecognizedResponseErr
  • func (u UnrecognizedResponseErr) Error() string

Constants

const (
	// Version is the current version of this driver
	Version = "3.3"

	// ClientID is the id of this client
	ClientID = "SermoDigitalBolt/" + Version

	DefaultPort = "7687"      // default port for regular socket connections
	DefaultHost = "localhost" // default host for regular socket connections
	Scheme      = "bolt://"   // Bolt protocol's URI scheme
)

const (
	// HostEnv is the environment variable read to gather the host information.
	HostEnv = "BOLT_DRIVER_HOST"

	// PortEnv is the environment variable read to gather the port information.
	PortEnv = "BOLT_DRIVER_PORT"

	// UserEnv is the environment variable read to gather the username.
	UserEnv = "BOLT_DRIVER_USER"

	// PassEnv is the environment variable read to gather the password.
	PassEnv = "BOLT_DRIVER_PASS"

	// TLSEnv is the environment variable read to determine whether the Open
	// and OpenNeo methods should attempt to connect with TLS.
	TLSEnv = "BOLT_DRIVER_TLS"

	// TLSNoVerifyEnv is the environment variable read to determine whether
	// the TLS certificate's verification should be skipped.
	TLSNoVerifyEnv = "BOLT_DRIVER_NO_VERIFY"

	// TLSCACertFileEnv is the environment variable read that should contain
	// the CA certificate's path.
	TLSCACertFileEnv = "BOLT_DRIVER_TLS_CA_CERT_FILE"

	// TLSCertFileEnv is the environment variable read that should contain the
	// public key path.
	TLSCertFileEnv = "BOLT_TLS_CERT_FILE"

	// TLSKeyFileEnv is the environment variable read that should contain the
	// private key path.
	TLSKeyFileEnv = "BOLT_TLS_KEY_FILE"
)

const (
	DefaultDriver   = "bolt"
	RecordingDriver = "bolt-recorder"
)

Variables

var ErrInFailedTransaction = errors.New("bolt: operation inside failed transaction")

ErrInFailedTransaction is returned when an operation is attempted inside a failed transaction.

var ErrNotMap = errors.New("bolt: if one argument is passed it must of type Map")

ErrNotMap is returned when one argument is passed to a Query, Exec, etc. method and its type isn't Map.

var ErrRowsClosed = errors.New("bolt: rows have been closed")

ErrRowsClosed is returned when the Rows have already been closed.

var ErrStatementClosed = errors.New("bolt: statement is closed")

ErrStatementClosed is returned when an operation is attempted on a closed statement.

Functions

func DialOpen

func DialOpen(d Dialer, name string) (driver.Conn, error)

DialOpen opens a driver.Conn with the given Dialer and network configuration.

func Open

func Open(name string) (driver.Conn, error)

Open calls DialOpen with the default dialer.

func WithSummary

func WithSummary(ctx context.Context) (context.Context, func() *Summary)

WithSummary returns a context.Context that facilitates passing the summary of a Cypher query. The returned function is only valid after Rows.Close has been called or after Exec/ExecContext has returned. The function is idempotent.

Types

type Array

type Array []interface{}

Array implements sql.Scanner and should be used to retrieve, e.g., a list from sql.Rows.

func (*Array) Scan

func (a *Array) Scan(val interface{}) error

type Counters

type Counters struct {
	NodesCreated         int64
	NodesDeleted         int64
	RelationshipsCreated int64
	RelationshipsDeleted int64
	PropertiesSet        int64
	LabelsAdded          int64
	LabelsRemoved        int64
	IndicesAdded         int64
	IndicesRemoved       int64
	ConstraintsAdded     int64
	ConstraintsRemoved   int64
}

Counters counts the number of different operations the query performed.

func (Counters) RowsAffected

func (c Counters) RowsAffected() (int64, error)

RowsAffected returns the number of nodes and relationships created and deleted during the query. It partially implements driver.Result.

type Dialer

type Dialer interface {
	// Dial connects to the address on the named network.
	Dial(network, address string) (net.Conn, error)

	// DialTimeout acts like Dial but takes a timeout. The timeout should
	// included name resolution, if required.
	DialTimeout(network, address string, timeout time.Duration) (net.Conn, error)
}

Dialer is a generic interface for types that can dial network addresses.

func TLSDialer

func TLSDialer(caFile, certFile, keyFile string, noVerify bool) (Dialer, error)

TLSDialer returns a Dialer that is compatible with Neo4j. It can be passed to DialOpen and DialOpenNeo. It reads configuration information from environment variables, although the function parameters take precedence. noVerify will only be read from an environment variable if noVerify is false.

type Event

type Event struct {
	Timestamp int64 `json:"-"`
	Event     []byte
	IsWrite   bool
	Completed bool
	Error     error
}

Event represents a single recording (read or write) event in the recorder

type Map

type Map map[string]interface{}

Map is a utility type. See the package docs for its use in Query, Exec, etc. calls. It also implements sql.Scanner and should be used to retrieve, e.g., properties from sql.Rows.

func (*Map) Scan

func (m *Map) Scan(val interface{}) error

type Notification

type Notification struct {
	// Code is the notification code.
	Code string
	// Title is a short summary of the notification.
	Title string
	// Description is a longer description of the notification.
	Description string
	// Position is the position in a query this notificaiton points to.
	Position Position
	// Severity is the severity level of the notification.
	Severity string
}

Notification represents a notification that might occur during the exectution of a query.

type Plan

type Plan struct {
	// Operation is the type of operation the plan is performing.
	Operation string
	// Args contains the arguments the planner uses to during its execution.
	Args map[string]interface{}
	// Identifiers are identifiers used by the plan and can be generated by
	// either the user or planner.
	Identifiers []string
	// Profile is the executed plan.
	Profile *Profile
	// Children returns the next level of the planning tree.
	Children []Plan
}

Plan describes the plan the database planner used when executing the query.

type Position

type Position struct {
	// Offset is the character offset this position points to, starting at 0.
	Offset int64
	// Line is the line number this position points to, starting at 1.
	Line int64
	// Column is the column number this position points to, starting at 1.
	Column int64
}

Position is the position in a query a notification points to.

type Profile

type Profile struct {
	// Hits is the number of time the plan touched the underlying data stores.
	Hits int64
	// Records is the number of records the plan produced.
	Records int64
}

Profile describes an executed plan.

type Recorder

type Recorder struct {
	Name string

	net.Conn
	// contains filtered or unexported fields
}

Recorder allows for recording and playback of a session. The Name field can be set and when closed the Recorder will write out the session to a gzipped JSON file with the specified name. For example

const name = "TestRecordedSession"
sql.Register(name, &Recorder{Name: name})

db, err := sql.Open(name)
if err != nil { ... }

// Lots of code

// The recording will be saved as "TestRecordedSession.json.gzip"
if err := db.Close(); err != nil { ... }

Do note: a Recorder where Name == "" has already been registered using the name

bolt-recorder

and will create a random, timestamped file name.

func (*Recorder) Close

func (r *Recorder) Close() error

Close the net.Conn, outputting the recording.

func (*Recorder) Open

func (r *Recorder) Open(name string) (driver.Conn, error)

Open opens a simulated Neo4j connection using pre-recorded data if name is an empty string. Otherwise, it opens up an actual connection using that to create a new recording.

func (*Recorder) Read

func (r *Recorder) Read(p []byte) (n int, err error)

Read reads from the net.Conn, recording the interaction.

func (*Recorder) Write

func (r *Recorder) Write(b []byte) (n int, err error)

Write to the net.Conn, recording the interaction.

type ServerInfo

type ServerInfo struct {
	// Address is the remote address of the server where the query was executed.
	Address string
	// Version is a string indicating which version of the server executed the
	// query.
	Version string
}

ServerInfo describes basic information on the server that ran the query.

type Summary

type Summary struct {
	Query          string
	Counters       Counters
	Type           Type
	Plan           Plan
	Notifications  []Notification
	AvailableAfter time.Duration
	ConsumedAfter  time.Duration
	ServerInfo     ServerInfo
}

Summary lists details of the query, like profiling information and what type of statement was run.

type Type

type Type uint8

Type describes the type of query.

const (
	Read        Type = iota // read only
	ReadWrite               // read and write
	Write                   // write only
	SchemaWrite             // schema write only
)

func (*Type) SetString

func (t *Type) SetString(s string)

func (Type) String

func (t Type) String() string

type UnrecognizedResponseErr

type UnrecognizedResponseErr struct {
	// contains filtered or unexported fields
}

UnrecognizedResponseErr is an error used when the server sends a reply this library cannot recognize. It might indicate a version mismatch or a library bug.

func (UnrecognizedResponseErr) Error

func (u UnrecognizedResponseErr) Error() string