pq

package
v1.9.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 24, 2015 License: MIT, Apache-2.0 Imports: 29 Imported by: 0

README

pq - A pure Go postgres driver for Go's database/sql package

Build Status

Install

go get github.com/lib/pq

Docs

For detailed documentation and basic usage examples, please see the package documentation at http://godoc.org/github.com/lib/pq.

Tests

go test is used for testing. A running PostgreSQL server is required, with the ability to log in. The default database to connect to test with is "pqgotest," but it can be overridden using environment variables.

Example:

PGHOST=/var/run/postgresql go test github.com/lib/pq

Optionally, a benchmark suite can be run as part of the tests:

PGHOST=/var/run/postgresql go test -bench .

Features

  • SSL
  • Handles bad connections for database/sql
  • Scan time.Time correctly (i.e. timestamp[tz], time[tz], date)
  • Scan binary blobs correctly (i.e. bytea)
  • Package for hstore support
  • COPY FROM support
  • pq.ParseURL for converting urls to connection strings for sql.Open.
  • Many libpq compatible environment variables
  • Unix socket support
  • Notifications: LISTEN/NOTIFY

Future / Things you can help with

  • Better COPY FROM / COPY TO (see discussion in #181)

Thank you (alphabetical)

Some of these contributors are from the original library bmizerany/pq.go whose code still exists in here.

  • Andy Balholm (andybalholm)
  • Ben Berkert (benburkert)
  • Benjamin Heatwole (bheatwole)
  • Bill Mill (llimllib)
  • Bjørn Madsen (aeons)
  • Blake Gentry (bgentry)
  • Brad Fitzpatrick (bradfitz)
  • Charlie Melbye (cmelbye)
  • Chris Bandy (cbandy)
  • Chris Walsh (cwds)
  • Dan Sosedoff (sosedoff)
  • Daniel Farina (fdr)
  • Eric Chlebek (echlebek)
  • Eric Garrido (minusnine)
  • Eric Urban (hydrogen18)
  • Everyone at The Go Team
  • Evan Shaw (edsrzf)
  • Ewan Chou (coocood)
  • Federico Romero (federomero)
  • Fumin (fumin)
  • Gary Burd (garyburd)
  • Heroku (heroku)
  • James Pozdena (jpoz)
  • Jason McVetta (jmcvetta)
  • Jeremy Jay (pbnjay)
  • Joakim Sernbrant (serbaut)
  • John Gallagher (jgallagher)
  • Jonathan Rudenberg (titanous)
  • Joël Stemmer (jstemmer)
  • Kamil Kisiel (kisielk)
  • Kelly Dunn (kellydunn)
  • Keith Rarick (kr)
  • Kir Shatrov (kirs)
  • Lann Martin (lann)
  • Maciek Sakrejda (deafbybeheading)
  • Marc Brinkmann (mbr)
  • Marko Tiikkaja (johto)
  • Matt Newberry (MattNewberry)
  • Matt Robenolt (mattrobenolt)
  • Martin Olsen (martinolsen)
  • Mike Lewis (mikelikespie)
  • Nicolas Patry (Narsil)
  • Oliver Tonnhofer (olt)
  • Patrick Hayes (phayes)
  • Paul Hammond (paulhammond)
  • Ryan Smith (ryandotsmith)
  • Samuel Stauffer (samuel)
  • Timothée Peignier (cyberdelia)
  • Travis Cline (tmc)
  • TruongSinh Tran-Nguyen (truongsinh)
  • notedit (notedit)

Documentation

Overview

Package pq is a pure Go Postgres driver for the database/sql package.

In most cases clients will use the database/sql package instead of using this package directly. For example: 

import (
	"database/sql"

	_ "github.com/lib/pq"
)

func main() {
	db, err := sql.Open("postgres", "user=pqgotest dbname=pqgotest sslmode=verify-full")
	if err != nil {
		log.Fatal(err)
	}

	age := 21
	rows, err := db.Query("SELECT name FROM users WHERE age = $1", age)
	…
}

You can also connect to a database using a URL. For example: 

db, err := sql.Open("postgres", "postgres://pqgotest:password@localhost/pqgotest?sslmode=verify-full")

Connection String Parameters

Similarly to libpq, when establishing a connection using pq you are expected to supply a connection string containing zero or more parameters. A subset of the connection parameters supported by libpq are also supported by pq. Additionally, pq also lets you specify run-time parameters (such as search_path or work_mem) directly in the connection string. This is different from libpq, which does not allow run-time parameters in the connection string, instead requiring you to supply them in the options parameter.

For compatibility with libpq, the following special connection parameters are supported:

  • dbname - The name of the database to connect to
  • user - The user to sign in as
  • password - The user's password
  • host - The host to connect to. Values that start with / are for unix domain sockets. (default is localhost)
  • port - The port to bind to. (default is 5432)
  • sslmode - Whether or not to use SSL (default is require, this is not the default for libpq)
  • fallback_application_name - An application_name to fall back to if one isn't provided.
  • connect_timeout - Maximum wait for connection, in seconds. Zero or not specified means wait indefinitely.
  • sslcert - Cert file location. The file must contain PEM encoded data.
  • sslkey - Key file location. The file must contain PEM encoded data.
  • sslrootcert - The location of the root certificate file. The file must contain PEM encoded data.

Valid values for sslmode are:

  • disable - No SSL
  • require - Always SSL (skip verification)
  • verify-ca - Always SSL (verify that the certificate presented by the server was signed by a trusted CA)
  • verify-full - Always SSL (verify that the certification presented by the server was signed by a trusted CA and the server host name matches the one in the certificate)

See http://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING for more information about connection string parameters.

Use single quotes for values that contain whitespace: 

"user=pqgotest password='with spaces'"

A backslash will escape the next character in values: 

"user=space\ man password='it\'s valid'

Note that the connection parameter client_encoding (which sets the text encoding for the connection) may be set but must be "UTF8", matching with the same rules as Postgres. It is an error to provide any other value.

In addition to the parameters listed above, any run-time parameter that can be set at backend start time can be set in the connection string. For more information, see http://www.postgresql.org/docs/current/static/runtime-config.html.

Most environment variables as specified at http://www.postgresql.org/docs/current/static/libpq-envars.html supported by libpq are also supported by pq. If any of the environment variables not supported by pq are set, pq will panic during connection establishment. Environment variables have a lower precedence than explicitly provided connection parameters.

Queries

database/sql does not dictate any specific format for parameter markers in query strings, and pq uses the Postgres-native ordinal markers, as shown above. The same marker can be reused for the same parameter: 

rows, err := db.Query(`SELECT name FROM users WHERE favorite_fruit = $1
	OR age BETWEEN $2 AND $2 + 3`, "orange", 64)

pq does not support the LastInsertId() method of the Result type in database/sql. To return the identifier of an INSERT (or UPDATE or DELETE), use the Postgres RETURNING clause with a standard Query or QueryRow call: 

var userid int
err := db.QueryRow(`INSERT INTO users(name, favorite_fruit, age)
	VALUES('beatrice', 'starfruit', 93) RETURNING id`).Scan(&userid)

For more details on RETURNING, see the Postgres documentation: 

http://www.postgresql.org/docs/current/static/sql-insert.html
http://www.postgresql.org/docs/current/static/sql-update.html
http://www.postgresql.org/docs/current/static/sql-delete.html

For additional instructions on querying see the documentation for the database/sql package.

Errors

pq may return errors of type *pq.Error which can be interrogated for error details: 

if err, ok := err.(*pq.Error); ok {
    fmt.Println("pq error:", err.Code.Name())
}

See the pq.Error type for details.

Bulk imports

You can perform bulk imports by preparing a statement returned by pq.CopyIn (or pq.CopyInSchema) in an explicit transaction (sql.Tx). The returned statement handle can then be repeatedly "executed" to copy data into the target table. After all data has been processed you should call Exec() once with no arguments to flush all buffered data. Any call to Exec() might return an error which should be handled appropriately, but because of the internal buffering an error returned by Exec() might not be related to the data passed in the call that failed.

CopyIn uses COPY FROM internally. It is not possible to COPY outside of an explicit transaction in pq.

Usage example: 

txn, err := db.Begin()
if err != nil {
	log.Fatal(err)
}

stmt, err := txn.Prepare(pq.CopyIn("users", "name", "age"))
if err != nil {
	log.Fatal(err)
}

for _, user := range users {
	_, err = stmt.Exec(user.Name, int64(user.Age))
	if err != nil {
		log.Fatal(err)
	}
}

_, err = stmt.Exec()
if err != nil {
	log.Fatal(err)
}

err = stmt.Close()
if err != nil {
	log.Fatal(err)
}

err = txn.Commit()
if err != nil {
	log.Fatal(err)
}

Notifications

PostgreSQL supports a simple publish/subscribe model over database connections. See http://www.postgresql.org/docs/current/static/sql-notify.html for more information about the general mechanism.

To start listening for notifications, you first have to open a new connection to the database by calling NewListener. This connection can not be used for anything other than LISTEN / NOTIFY. Calling Listen will open a "notification channel"; once a notification channel is open, a notification generated on that channel will effect a send on the Listener.Notify channel. A notification channel will remain open until Unlisten is called, though connection loss might result in some notifications being lost. To solve this problem, Listener sends a nil pointer over the Notify channel any time the connection is re-established following a connection loss. The application can get information about the state of the underlying connection by setting an event callback in the call to NewListener.

A single Listener can safely be used from concurrent goroutines, which means that there is often no need to create more than one Listener in your application. However, a Listener is always connected to a single database, so you will need to create a new Listener instance for every database you want to receive notifications in.

The channel name in both Listen and Unlisten is case sensitive, and can contain any characters legal in an identifier (see http://www.postgresql.org/docs/current/static/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS for more information). Note that the channel name will be truncated to 63 bytes by the PostgreSQL server.

You can find a complete, working example of Listener usage at http://godoc.org/github.com/lib/pq/listen_example.

Index

Constants

View Source 
const (
	Efatal   = "FATAL"
	Epanic   = "PANIC"
	Ewarning = "WARNING"
	Enotice  = "NOTICE"
	Edebug   = "DEBUG"
	Einfo    = "INFO"
	Elog     = "LOG"
)

Error severities

Variables

View Source 
var (
	ErrNotSupported              = errors.New("pq: Unsupported command")
	ErrInFailedTransaction       = errors.New("pq: Could not complete operation in a failed transaction")
	ErrSSLNotSupported           = errors.New("pq: SSL is not enabled on the server")
	ErrSSLKeyHasWorldPermissions = errors.New("pq: Private key file has group or world access. Permissions should be u=rw (0600) or less.")
	ErrCouldNotDetectUsername    = errors.New("pq: Could not detect default username. Please provide one explicitly.")
)

Common error types

View Source 
var ErrChannelAlreadyOpen = errors.New("pq: channel is already open")
View Source 
var ErrChannelNotOpen = errors.New("pq: channel is not open")

Functions

func CopyIn 

func CopyIn(table string, columns ...string) string

CopyIn creates a COPY FROM statement which can be prepared with Tx.Prepare(). The target table should be visible in search_path.

func CopyInSchema 

func CopyInSchema(schema, table string, columns ...string) string

CopyInSchema creates a COPY FROM statement which can be prepared with Tx.Prepare().

func DialOpen 

func DialOpen(d Dialer, name string) (_ driver.Conn, err error)

func EnableInfinityTs 

func EnableInfinityTs(negative time.Time, positive time.Time)

* If EnableInfinityTs is not called, "-infinity" and "infinity" will return * []byte("-infinity") and []byte("infinity") respectively, and potentially * cause error "sql: Scan error on column index 0: unsupported driver -> Scan pair: []uint8 -> *time.Time", * when scanning into a time.Time value. * * Once EnableInfinityTs has been called, all connections created using this * driver will decode Postgres' "-infinity" and "infinity" for "timestamp", * "timestamp with time zone" and "date" types to the predefined minimum and * maximum times, respectively. When encoding time.Time values, any time which * equals or preceeds the predefined minimum time will be encoded to * "-infinity". Any values at or past the maximum time will similarly be * encoded to "infinity". * * * If EnableInfinityTs is called with negative >= positive, it will panic. * Calling EnableInfinityTs after a connection has been established results in * undefined behavior. If EnableInfinityTs is called more than once, it will * panic.

func Open 

func Open(name string) (_ driver.Conn, err error)

func ParseURL 

func ParseURL(url string) (string, error)

ParseURL no longer needs to be used by clients of this library since supplying a URL as a connection string to sql.Open() is now supported: 

sql.Open("postgres", "postgres://bob:secret@1.2.3.4:5432/mydb?sslmode=verify-full")

It remains exported here for backwards-compatibility.

ParseURL converts a url to a connection string for driver.Open. Example: 

"postgres://bob:secret@1.2.3.4:5432/mydb?sslmode=verify-full"

converts to: 

"user=bob password=secret host=1.2.3.4 port=5432 dbname=mydb sslmode=verify-full"

A minimal example: 

"postgres://"

This will be blank, causing driver.Open to use all of the defaults

func QuoteIdentifier 

func QuoteIdentifier(name string) string

QuoteIdentifier quotes an "identifier" (e.g. a table or a column name) to be used as part of an SQL statement. For example: 

tblname := "my_table"
data := "my_data"
err = db.Exec(fmt.Sprintf("INSERT INTO %s VALUES ($1)", pq.QuoteIdentifier(tblname)), data)

Any double quotes in name will be escaped. The quoted identifier will be case sensitive when used in a query. If the input string contains a zero byte, the result will be truncated immediately before it.

Types

type Dialer 

type Dialer interface {
	Dial(network, address string) (net.Conn, error)
	DialTimeout(network, address string, timeout time.Duration) (net.Conn, error)
}

type Error 

type Error struct {
	Severity         string
	Code             ErrorCode
	Message          string
	Detail           string
	Hint             string
	Position         string
	InternalPosition string
	InternalQuery    string
	Where            string
	Schema           string
	Table            string
	Column           string
	DataTypeName     string
	Constraint       string
	File             string
	Line             string
	Routine          string
}

Error represents an error communicating with the server.

See http://www.postgresql.org/docs/current/static/protocol-error-fields.html for details of the fields

func (Error) Error 

func (err Error) Error() string

func (*Error) Fatal 

func (err *Error) Fatal() bool

Fatal returns true if the Error Severity is fatal.

func (*Error) Get 

func (err *Error) Get(k byte) (v string)

Get implements the legacy PGError interface. New code should use the fields of the Error struct directly.

type ErrorClass 

type ErrorClass string

ErrorClass is only the class part of an error code.

func (ErrorClass) Name 

func (ec ErrorClass) Name() string

Name returns the condition name of an error class. It is equivalent to the condition name of the "standard" error code (i.e. the one having the last three characters "000").

type ErrorCode 

type ErrorCode string

ErrorCode is a five-character error code.

func (ErrorCode) Class 

func (ec ErrorCode) Class() ErrorClass

Class returns the error class, e.g. "28".

See http://www.postgresql.org/docs/9.3/static/errcodes-appendix.html for details.

func (ErrorCode) Name 

func (ec ErrorCode) Name() string

Name returns a more human friendly rendering of the error code, namely the "condition name".

See http://www.postgresql.org/docs/9.3/static/errcodes-appendix.html for details.

type EventCallbackType 

type EventCallbackType func(event ListenerEventType, err error)

type Listener 

type Listener struct {
	// Channel for receiving notifications from the database.  In some cases a
	// nil value will be sent.  See section "Notifications" above.
	Notify chan *Notification
	// contains filtered or unexported fields
}

Listener provides an interface for listening to notifications from a PostgreSQL database. For general usage information, see section "Notifications".

Listener can safely be used from concurrently running goroutines.

func NewListener 

func NewListener(name string,
	minReconnectInterval time.Duration,
	maxReconnectInterval time.Duration,
	eventCallback EventCallbackType) *Listener

NewListener creates a new database connection dedicated to LISTEN / NOTIFY.

name should be set to a connection string to be used to establish the database connection (see section "Connection String Parameters" above).

minReconnectInterval controls the duration to wait before trying to re-establish the database connection after connection loss. After each consecutive failure this interval is doubled, until maxReconnectInterval is reached. Successfully completing the connection establishment procedure resets the interval back to minReconnectInterval.

The last parameter eventCallback can be set to a function which will be called by the Listener when the state of the underlying database connection changes. This callback will be called by the goroutine which dispatches the notifications over the Notify channel, so you should try to avoid doing potentially time-consuming operations from the callback.

func (*Listener) Close 

func (l *Listener) Close() error

Close disconnects the Listener from the database and shuts it down. Subsequent calls to its methods will return an error. Close returns an error if the connection has already been closed.

func (*Listener) Listen 

func (l *Listener) Listen(channel string) error

Listen starts listening for notifications on a channel. Calls to this function will block until an acknowledgement has been received from the server. Note that Listener automatically re-establishes the connection after connection loss, so this function may block indefinitely if the connection can not be re-established.

Listen will only fail in three conditions:

  1. The channel is already open. The returned error will be ErrChannelAlreadyOpen.
  2. The query was executed on the remote server, but PostgreSQL returned an error message in response to the query. The returned error will be a pq.Error containing the information the server supplied.
  3. Close is called on the Listener before the request could be completed.

The channel name is case-sensitive.

func (*Listener) NotificationChannel 

func (l *Listener) NotificationChannel() <-chan *Notification

Returns the notification channel for this listener. This is the same channel as Notify, and will not be recreated during the life time of the Listener.

func (*Listener) Ping 

func (l *Listener) Ping() error

Ping the remote server to make sure it's alive. Non-nil return value means that there is no active connection.

func (*Listener) Unlisten 

func (l *Listener) Unlisten(channel string) error

Unlisten removes a channel from the Listener's channel list. Returns ErrChannelNotOpen if the Listener is not listening on the specified channel. Returns immediately with no error if there is no connection. Note that you might still get notifications for this channel even after Unlisten has returned.

The channel name is case-sensitive.

func (*Listener) UnlistenAll 

func (l *Listener) UnlistenAll() error

UnlistenAll removes all channels from the Listener's channel list. Returns immediately with no error if there is no connection. Note that you might still get notifications for any of the deleted channels even after UnlistenAll has returned.

type ListenerConn 

type ListenerConn struct {
	// contains filtered or unexported fields
}

ListenerConn is a low-level interface for waiting for notifications. You should use Listener instead.

func NewListenerConn 

func NewListenerConn(name string, notificationChan chan<- *Notification) (*ListenerConn, error)

Creates a new ListenerConn. Use NewListener instead.

func (*ListenerConn) Close 

func (l *ListenerConn) Close() error

func (*ListenerConn) Err 

func (l *ListenerConn) Err() error

Err() returns the reason the connection was closed. It is not safe to call this function until l.Notify has been closed.

func (*ListenerConn) ExecSimpleQuery 

func (l *ListenerConn) ExecSimpleQuery(q string) (executed bool, err error)

Execute a "simple query" (i.e. one with no bindable parameters) on the connection. The possible return values are:

  1. "executed" is true; the query was executed to completion on the database server. If the query failed, err will be set to the error returned by the database, otherwise err will be nil.
  2. If "executed" is false, the query could not be executed on the remote server. err will be non-nil.

After a call to ExecSimpleQuery has returned an executed=false value, the connection has either been closed or will be closed shortly thereafter, and all subsequently executed queries will return an error.

func (*ListenerConn) Listen 

func (l *ListenerConn) Listen(channel string) (bool, error)

Send a LISTEN query to the server. See ExecSimpleQuery.

func (*ListenerConn) Ping 

func (l *ListenerConn) Ping() error

Ping the remote server to make sure it's alive. Non-nil error means the connection has failed and should be abandoned.

func (*ListenerConn) Unlisten 

func (l *ListenerConn) Unlisten(channel string) (bool, error)

Send an UNLISTEN query to the server. See ExecSimpleQuery.

func (*ListenerConn) UnlistenAll 

func (l *ListenerConn) UnlistenAll() (bool, error)

Send `UNLISTEN *` to the server. See ExecSimpleQuery.

type ListenerEventType 

type ListenerEventType int
const (
	// Emitted only when the database connection has been initially
	// initialized.  err will always be nil.
	ListenerEventConnected ListenerEventType = iota

	// Emitted after a database connection has been lost, either because of an
	// error or because Close has been called.  err will be set to the reason
	// the database connection was lost.
	ListenerEventDisconnected

	// Emitted after a database connection has been re-established after
	// connection loss.  err will always be nil.  After this event has been
	// emitted, a nil pq.Notification is sent on the Listener.Notify channel.
	ListenerEventReconnected

	// Emitted after a connection to the database was attempted, but failed.
	// err will be set to an error describing why the connection attempt did
	// not succeed.
	ListenerEventConnectionAttemptFailed
)

type Notification 

type Notification struct {
	// Process ID (PID) of the notifying postgres backend.
	BePid int
	// Name of the channel the notification was sent on.
	Channel string
	// Payload, or the empty string if unspecified.
	Extra string
}

Notification represents a single notification from the database.

type NullTime 

type NullTime struct {
	Time  time.Time
	Valid bool // Valid is true if Time is not NULL
}

NullTime represents a time.Time that may be null. NullTime implements the sql.Scanner interface so it can be used as a scan destination, similar to sql.NullString.

func (*NullTime) Scan 

func (nt *NullTime) Scan(value interface{}) error

Scan implements the Scanner interface.

func (NullTime) Value 

func (nt NullTime) Value() (driver.Value, error)

Value implements the driver Valuer interface.

type PGError 

type PGError interface {
	Error() string
	Fatal() bool
	Get(k byte) (v string)
}

PGError is an interface used by previous versions of pq. It is provided only to support legacy code. New code should use the Error type.

Source Files

View all Source files

Directories

Path Synopsis
Below you will find a self-contained Go program which uses the LISTEN / NOTIFY mechanism to avoid polling the database while waiting for more work to arrive.
 Below you will find a self-contained Go program which uses the LISTEN / NOTIFY mechanism to avoid polling the database while waiting for more work to arrive.
Package oid contains OID constants as defined by the Postgres server.
 Package oid contains OID constants as defined by the Postgres server.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.