zapdriver

package module
v1.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 8, 2019 License: ISC Imports: 11 Imported by: 126

README

⚡ Zapdriver

Blazing fast, Zap-based Stackdriver logging.

Usage

This package provides three building blocks to support the full array of structured logging capabilities of Stackdriver:

The above components can be used separately, but to start, you can create a new Zap logger with all of the above included:

logger, err := zapdriver.NewProduction() // with sampling
logger, err := zapdriver.NewDevelopment() // with `development` set to `true`

The above functions give back a pointer to a zap.Logger object, so you can use Zap like you've always done, except that it now logs in the proper Stackdriver format.

You can also create a configuration struct, and build your logger from there:

config := zapdriver.NewProductionConfig()
config := zapdriver.NewDevelopmentConfig()

Or, get the Zapdriver encoder, and build your own configuration struct from that:

encoder := zapdriver.NewProductionEncoderConfig()
encoder := zapdriver.NewDevelopmentEncoderConfig()

Read on to learn more about the available Stackdriver-specific log fields, and how to use the above-mentioned components.

Special purpose logging fields

You can use the following fields to add extra information to your log entries. These fields are parsed by Stackdriver to make it easier to query your logs or to use the log details in the Stackdriver monitoring interface.

HTTP

You can log HTTP request/response cycles using the following field:

HTTP(req *HTTPPayload) zap.Field

You can either manually build the request payload:

req := &HTTPPayload{
  RequestMethod: "GET",
  RequestURL: "/",
  Status: 200,
}

Or, you can auto generate the struct, based on the available request and response objects:

NewHTTP(req *http.Request, res *http.Response) *HTTPPayload

You are free to pass in nil for either the request or response object, if one of them is unavailable to you at the point of logging. Any field depending on one or the other will be omitted if nil is passed in.

Note that there are some fields that are not populated by either the request or response object, and need to be set manually:

  • ServerIP string
  • Latency string
  • CacheLookup bool
  • CacheHit bool
  • CacheValidatedWithOriginServer bool
  • CacheFillBytes string

If you have no need for those fields, the quickest way to get started is like so:

logger.Info("Request Received.", zapdriver.HTTP(zapdriver.NewHTTP(req, res)))
Label

You can add a "label" to your payload as follows:

Label(key, value string) zap.Field

Note that underwater, this sets the key to labels.<key>. You need to be using the zapdriver.Core core for this to be converted to the proper format for Stackdriver to recognize the labels.

See "Custom Stackdriver Zap core" for more details.

If you have a reason not to use the provided Core, you can still wrap labels in the right labels namespace by using the available function:

Labels(fields ...zap.Field) zap.Field

Like so:

logger.Info(
  "Did something.",
  zapdriver.Labels(
    zapdriver.Label("hello", "world"),
    zapdriver.Label("hi", "universe"),
  ),
)

Again, wrapping the Label calls in Labels is not required if you use the supplied Zap Core.

SourceLocation

You can add a source code location to your log lines to be picked up by Stackdriver.

Note that you can set this manually, or use zapdriver.Core to automatically add this. If you set it manually, and use zapdriver.Core, the manual call stack will be preserved over the automated one.

SourceLocation(pc uintptr, file string, line int, ok bool) zap.Field

Note that the function signature equals that of the return values of runtime.Caller(). This allows you to catch the stack frame at one location, while logging it at a different location, like so:

pc, file, line, ok := runtime.Caller(0)

// do other stuff...

logger.Error("Something happened!", zapdriver.SourceLocation(pc, file, line, ok))

If you use zapdriver.Core, the above use-case is the only use-case where you would want to manually set the source location. In all other situations, you can simply omit this field, and it will be added automatically, using the stack frame at the location where the log line is triggered.

If you don't use zapdriver.Core, and still want to add the source location at the frame of the triggered log line, you'd do it like this:

logger.Error("Something happened!", zapdriver.SourceLocation(runtime.Caller(0)))
Operation

The Operation log field allows you to group log lines into a single "operation" performed by the application:

Operation(id, producer string, first, last bool) zap.Field

For a pair of logs that belong to the same operation, you should use the same id between them. The producer is an arbitrary identifier that should be globally unique amongst all the logs of all your applications (meaning it should probably be the unique name of the current application). You should set first to true for the first log in the operation, and last to true for the final log of the operation.

logger.Info("Started.", zapdriver.Operation("3g4d3g", "my-app", true, false))
logger.Debug("Progressing.", zapdriver.Operation("3g4d3g", "my-app", false, false))
logger.Info("Done.", zapdriver.Operation("3g4d3g", "my-app", false, true))

Instead of defining the "start" and "end" booleans, you can also use these three convenience functions:

OperationStart(id, producer string) zap.Field
OperationCont(id, producer string) zap.Field
OperationEnd(id, producer string) zap.Field
TraceContext

You can add trace context information to your log lines to be picked up by Stackdriver.

TraceContext(trace string, spanId string, sampled bool, projectName string) []zap.Field

Like so:

logger.Error("Something happened!", zapdriver.TraceContext("105445aa7843bc8bf206b120001000", "0", true, "my-project-name")...)
Pre-configured Stackdriver-optimized encoder

The Stackdriver encoder maps all Zap log levels to the appropriate Stackdriver-supported levels:

DEBUG (100) Debug or trace information.

INFO (200) Routine information, such as ongoing status or performance.

WARNING (400) Warning events might cause problems.

ERROR (500) Error events are likely to cause problems.

CRITICAL (600) Critical events cause more severe problems or outages.

ALERT (700) A person must take an action immediately.

EMERGENCY (800) One or more systems are unusable.

It also sets some of the default keys to use the right names, such as timestamp, severity, and message.

You can use this encoder if you want to build your Zap logger configuration manually:

zapdriver.NewProductionEncoderConfig()

For parity-sake, there's also zapdriver.NewDevelopmentEncoderConfig(), but it returns the exact same encoder right now.

Custom Stackdriver Zap core

A custom Zap core is included in this package to support some special use-cases.

First of all, if you use zapdriver.NewProduction() (or NewDevelopment) , you already have this core enabled, so everything just works ™.

There are two use-cases which require this core:

  1. If you use zapdriver.Label("hello", "world"), it will initially end up in your log with the key labels.hello and value world. Now if you have two labels, you could also have labels.hi with value universe. This works as- is, but for this to be correctly parsed by Stackdriver as true "labels", you need to use the Zapdriver core, so that both of these fields get rewritten, to use the namespace labels, and use the keys hello and hi within that namespace. This is done automatically.

  2. If you don't want to use zapdriver.SourceLocation() on every log call, you can use this core for the source location to be automatically added to each log entry.

When building a logger, you can inject the Zapdriver core as follows:

config := &zap.Config{}
logger, err := config.Build(zapdriver.WrapCore())
Using Error Reporting

To report errors using StackDriver's Error Reporting tool, a log line needs to follow a separate log format described in the Error Reporting documentation.

The simplest way to do this is by using NewProductionWithCore:

logger, err := zapdriver.NewProductionWithCore(zapdriver.WrapCore(
  zapdriver.ReportAllErrors(true),
  zapdriver.ServiceName("my service"),
))

For parity-sake, there's also zapdriver.NewDevelopmentWithCore()

If you are building a custom logger, you can use WrapCore() to configure the driver core:

config := &zap.Config{}
logger, err := config.Build(zapdriver.WrapCore(
  zapdriver.ReportAllErrors(true),
  zapdriver.ServiceName("my service"),
))

Configuring this way, every error log entry will be reported to Stackdriver's Error Reporting tool.

Reporting errors manually

If you do not want every error to be reported, you can attach ErrorReport() to log call manually:

logger.Error("An error to be reported!", zapdriver.ErrorReport(runtime.Caller(0)))
// Or get Caller details
pc, file, line, ok := runtime.Caller(0)
// do other stuff... and log elsewhere
logger.Error("Another error to be reported!", zapdriver.ErrorReport(pc, file, line, ok))

Please keep in mind that ErrorReport needs a ServiceContext attached to the log entry. If you did not configure this using WrapCore, error reports will get attached using service name as unknown. To prevent this from happeneing, either configure your core or attach service context before (or when) using the logger:

logger.Error(
  "An error to be reported!",
  zapdriver.ErrorReport(runtime.Caller(0)),
  zapdriver.ServiceContext("my service"),
)

// Or permanently attach it to your logger
logger = logger.With(zapdriver.ServiceContext("my service"))
// and then use it
logger.Error("An error to be reported!", zapdriver.ErrorReport(runtime.Caller(0)))

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EncodeLevel 

func EncodeLevel(l zapcore.Level, enc zapcore.PrimitiveArrayEncoder)

EncodeLevel maps the internal Zap log level to the appropriate Stackdriver level.

func ErrorReport added in v1.2.0 

func ErrorReport(pc uintptr, file string, line int, ok bool) zap.Field

ErrorReport adds the correct Stackdriver "context" field for getting the log line reported as error.

see: https://cloud.google.com/error-reporting/docs/formatting-error-messages

func HTTP 

func HTTP(req *HTTPPayload) zap.Field

HTTP adds the correct Stackdriver "HTTP" field.

see: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#HttpRequest

func Label 

func Label(key, value string) zap.Field

Label adds an optional label to the payload.

Labels are a set of user-defined (key, value) data that provides additional information about the log entry.

Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.

func Labels 

func Labels(fields ...zap.Field) zap.Field

Labels takes Zap fields, filters the ones that have their key start with the string `labels.` and their value type set to StringType. It then wraps those key/value pairs in a top-level `labels` namespace.

func NewDevelopment 

func NewDevelopment(options ...zap.Option) (*zap.Logger, error)

NewDevelopment builds a development Logger that writes DebugLevel and above logs to standard error in a human-friendly format.

It's a shortcut for NewDevelopmentConfig().Build(...Option).

func NewDevelopmentConfig 

func NewDevelopmentConfig() zap.Config

NewDevelopmentConfig is a reasonable development logging configuration. Logging is enabled at DebugLevel and above.

It enables development mode (which makes DPanicLevel logs panic), uses a console encoder, writes to standard error, and disables sampling. Stacktraces are automatically included on logs of WarnLevel and above.

func NewDevelopmentEncoderConfig 

func NewDevelopmentEncoderConfig() zapcore.EncoderConfig

NewDevelopmentEncoderConfig returns an opinionated EncoderConfig for development environments.

func NewDevelopmentWithCore added in v1.2.0 

func NewDevelopmentWithCore(core zap.Option, options ...zap.Option) (*zap.Logger, error)

NewDevelopmentWithCore is same as NewDevelopment but accepts a custom configured core

func NewProduction 

func NewProduction(options ...zap.Option) (*zap.Logger, error)

NewProduction builds a sensible production Logger that writes InfoLevel and above logs to standard error as JSON.

It's a shortcut for NewProductionConfig().Build(...Option).

func NewProductionConfig 

func NewProductionConfig() zap.Config

NewProductionConfig is a reasonable production logging configuration. Logging is enabled at InfoLevel and above.

It uses a JSON encoder, writes to standard error, and enables sampling. Stacktraces are automatically included on logs of ErrorLevel and above.

func NewProductionEncoderConfig 

func NewProductionEncoderConfig() zapcore.EncoderConfig

NewProductionEncoderConfig returns an opinionated EncoderConfig for production environments.

func NewProductionWithCore added in v1.2.0 

func NewProductionWithCore(core zap.Option, options ...zap.Option) (*zap.Logger, error)

NewProductionWithCore is same as NewProduction but accepts a custom configured core

func Operation 

func Operation(id, producer string, first, last bool) zap.Field

Operation adds the correct Stackdriver "operation" field.

Additional information about a potentially long-running operation with which a log entry is associated.

see: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogEntryOperation

func OperationCont 

func OperationCont(id, producer string) zap.Field

OperationCont is a convenience function for `Operation`. It should be called for any non-start/end operation log.

func OperationEnd 

func OperationEnd(id, producer string) zap.Field

OperationEnd is a convenience function for `Operation`. It should be called for the last operation log.

func OperationStart 

func OperationStart(id, producer string) zap.Field

OperationStart is a convenience function for `Operation`. It should be called for the first operation log.

func RFC3339NanoTimeEncoder 

func RFC3339NanoTimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder)

RFC3339NanoTimeEncoder serializes a time.Time to an RFC3339Nano-formatted string with nanoseconds precision.

func ReportAllErrors added in v1.2.0 

func ReportAllErrors(report bool) func(*core)

zapdriver core option to report all logs with level error or above to stackdriver using `ErrorReport()` when set to true

func ServiceContext added in v1.2.0 

func ServiceContext(name string) zap.Field

ServiceContext adds the correct service information adding the log line It is a required field if an error needs to be reported.

see: https://cloud.google.com/error-reporting/reference/rest/v1beta1/ServiceContext see: https://cloud.google.com/error-reporting/docs/formatting-error-messages

func ServiceName added in v1.2.0 

func ServiceName(name string) func(*core)

zapdriver core option to add `ServiceContext()` to all logs with `name` as service name

func SourceLocation 

func SourceLocation(pc uintptr, file string, line int, ok bool) zap.Field

SourceLocation adds the correct Stackdriver "SourceLocation" field.

see: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogEntrySourceLocation

func TraceContext added in v1.3.0 

func TraceContext(trace string, spanId string, sampled bool, projectName string) []zap.Field

TraceContext adds the correct Stackdriver "trace", "span", "trace_sampled fields

see: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry

func WrapCore 

func WrapCore(options ...func(*core)) zap.Option

WrapCore returns a `zap.Option` that wraps the default core with the zapdriver one.

Types

type HTTPPayload 

type HTTPPayload struct {
	// The request method. Examples: "GET", "HEAD", "PUT", "POST".
	RequestMethod string `json:"requestMethod"`

	// The scheme (http, https), the host name, the path and the query portion of
	// the URL that was requested.
	//
	// Example: "http://example.com/some/info?color=red".
	RequestURL string `json:"requestUrl"`

	// The size of the HTTP request message in bytes, including the request
	// headers and the request body.
	RequestSize string `json:"requestSize"`

	// The response code indicating the status of response.
	//
	// Examples: 200, 404.
	Status int `json:"status"`

	// The size of the HTTP response message sent back to the client, in bytes,
	// including the response headers and the response body.
	ResponseSize string `json:"responseSize"`

	// The user agent sent by the client.
	//
	// Example: "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Q312461; .NET CLR 1.0.3705)".
	UserAgent string `json:"userAgent"`

	// The IP address (IPv4 or IPv6) of the client that issued the HTTP request.
	//
	// Examples: "192.168.1.1", "FE80::0202:B3FF:FE1E:8329".
	RemoteIP string `json:"remoteIp"`

	// The IP address (IPv4 or IPv6) of the origin server that the request was
	// sent to.
	ServerIP string `json:"serverIp"`

	// The referrer URL of the request, as defined in HTTP/1.1 Header Field
	// Definitions.
	Referer string `json:"referer"`

	// The request processing latency on the server, from the time the request was
	// received until the response was sent.
	//
	// A duration in seconds with up to nine fractional digits, terminated by 's'.
	//
	// Example: "3.5s".
	Latency string `json:"latency"`

	// Whether or not a cache lookup was attempted.
	CacheLookup bool `json:"cacheLookup"`

	// Whether or not an entity was served from cache (with or without
	// validation).
	CacheHit bool `json:"cacheHit"`

	// Whether or not the response was validated with the origin server before
	// being served from cache. This field is only meaningful if cacheHit is True.
	CacheValidatedWithOriginServer bool `json:"cacheValidatedWithOriginServer"`

	// The number of HTTP response bytes inserted into cache. Set only when a
	// cache fill was attempted.
	CacheFillBytes string `json:"cacheFillBytes"`

	// Protocol used for the request.
	//
	// Examples: "HTTP/1.1", "HTTP/2", "websocket"
	Protocol string `json:"protocol"`
}

HTTPPayload is the complete payload that can be interpreted by Stackdriver as a HTTP request.

func NewHTTP 

func NewHTTP(req *http.Request, res *http.Response) *HTTPPayload

NewHTTP returns a new HTTPPayload struct, based on the passed in http.Request and http.Response objects.

func (HTTPPayload) MarshalLogObject 

func (req HTTPPayload) MarshalLogObject(enc zapcore.ObjectEncoder) error

MarshalLogObject implements zapcore.ObjectMarshaller interface.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.