golambda

package module
v1.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 30, 2021 License: BSD-2-Clause, MIT Imports: 23 Imported by: 3

README

golambda Travis-CI Report card Go Reference

A suite of Go utilities for AWS Lambda functions to ease adopting best practices.

Overview

Features
  • Event decapsulation: Parse event data received when invoking. Also golambda make easy to write unit test of Lambda function
  • Structured logging: golambda provides requisite minimum logging interface for Lambda function. It output log as structured JSON.
  • Error handling: Error structure with arbitrary variables and stack trace feature.
  • Get secret parameters: Secret values should be stored in AWS Secrets Manager and can be got easily.

NOTE: The suite is NOT focusing to Lambda function for API gateway, but partially can be leveraged for the function.

How to use
$ go get github.com/m-mizutani/golambda

Source event decapsulation

Lambda function can have event source(s) such as SQS, SNS, etc. The main data is encapsulated in their data structure. golambda provides not only decapsulation feature for Lambda execution but also encapsulation feature for testing. Following event sources are supported for now.

  • SQS body: DecapSQSBody
  • SNS message: DecapSNSMessage
  • SNS message over SQS: DecapSNSonSQSMessage
Lambda implementation
package main

import (
	"strings"

	"github.com/m-mizutani/golambda"
)

// MyEvent is exported for test
type MyEvent struct {
	Message string `json:"message"`
}

// Handler is sample function. The function concatenates all message in SQS body and returns it.
func Handler(event golambda.Event) (interface{}, error) {
	// Decapsulate body message(s) in SQS Event structure.
	// It can also handles multiple SQS records.
	events, err := event.DecapSQSBody()
	if err != nil {
		return nil, err
	}

	var response []string

	// Iterate body message(S)
	for _, ev := range events {
		var msg MyEvent
		// Unmarshal golambda.Event to MyEvent
		if err := ev.Bind(&msg); err != nil {
			return nil, err
		}

		// Do something
		response = append(response, msg.Message)
	}

	return strings.Join(response, ":"), nil
}

func main() {
	golambda.Start(Handler)
}
Unit test
package main_test

import (
	"testing"

	"github.com/m-mizutani/golambda"
	"github.com/stretchr/testify/require"

	main "github.com/m-mizutani/golambda/example/decapEvent"
)

func TestHandler(t *testing.T) {
	var event golambda.Event
	messages := []main.MyEvent{
		{
			Message: "blue",
		},
		{
			Message: "orange",
		},
	}
	require.NoError(t, event.EncapSQS(messages))

	resp, err := main.Handler(event)
	require.NoError(t, err)
	require.Equal(t, "blue:orange", resp)
}

Structured logging

Lambda function output log data to CloudWatch Logs by default. CloudWatch Logs and Insights that is rich CloudWatch Logs viewer supports JSON format logs. Therefore JSON formatted log is better for Lambda function.

golambda provides Logger for JSON format logging. It has With() and Set() to add a pair of key and value to a log message. Logger has lambda request ID by default if you use the logger with golambda.Start().

Output with temporary variable
v1 := "say hello"
golambda.Logger.With("var1", v1).Info("Hello, hello, hello")
/* Output:
{
	"level": "info",
	"lambda.requestID": "565389dc-c13f-4fc0-b113-xxxxxxxxxxxx",
	"time": "2020-12-13T02:44:30Z",
	"var1": "say hello",
	"message": "Hello, hello, hello"
}
*/
Set permanent variable to logger
golambda.Logger.Set("myRequestID", myRequestID)

// ~~~~~~~ snip ~~~~~~

golambda.Logger.Error("oops")
/* Output:
{
	"level": "error",
	"lambda.requestID": "565389dc-c13f-4fc0-b113-xxxxxxxxxxxx",
	"time": "2020-11-12T02:44:30Z",
	"myRequestID": "xxxxxxxxxxxxxxxxx",
	"message": "oops"
}
*/
Log level

golambda.Logger (golambda.LambdaLogger type) provides following log level. Log level can be configured by environment variable LOG_LEVEL.

  • TRACE
  • DEBUG
  • INFO
  • ERROR

Lambda function should return error to top level function when occurring unrecoverable error, should not exit suddenly. Therefore PANIC and FATAL is not provided according to the thought.

Error handling

NOTE: golambda.Error is obsoleted and use github.com/m-mizutani/goerr instead.

golambda.Error can have pairs of key and value to keep context of error. For example, golambda.Error can bring original string data when failed to unmarshal JSON. The string data can be extracted in caller function.

Also, golambda.Start supports general error handling:

  1. Output error log with
    • Pairs of key and value in golambda.Error as error.values
    • Stack trace of error as error.stacktrace
  2. Send error record to sentry.io if SENTRY_DSN is set as environment variable
    • Stack trace of golambda.Error is also available in sentry.io by compatibility with github.com/pkg/errors
    • Output event ID of sentry to log as error.sentryEventID
    • You can set SENTRY_ENV and SENTRY_RELEASE also optionally.
package main

import (
	"github.com/m-mizutani/golambda"
)

// Handler is exported for test
func Handler(event golambda.Event) (interface{}, error) {
	trigger := "something wrong"
	return nil, golambda.goerr.New("oops").With("trigger", trigger)
}

func main() {
	golambda.Start(Handler)
}

Then, golambda output following log to CloudWatch.

{
    "level": "error",
    "lambda.requestID": "565389dc-c13f-4fc0-b113-f903909dbd45",
    "trigger": "something wrong",
    "stacktrace": [
        {
            "func": "main.Handler",
            "file": "xxx/your/project/src/main.go",
            "line": 27
        },
        {
            "func": "github.com/m-mizutani/golambda.Start.func1",
            "file": "xxx/github.com/m-mizutani/golambda/lambda.go",
            "line": 107
        }
    ],
    "time": "2020-12-13T02:42:48Z",
    "message": "oops"
}

Get secret parameters

In general, parameters of Lambda function are stored sa environment variable, such as LOG_LEVEL. However secret parameters such as credential, API key/token, etc should be stored in AWS Secrets Manager or Parameter Store to control access permission more explicitly in many cases.

golambda.GetSecretValues fetches values of AWS Secrets Manager and binds to a structure variable.

type mySecret struct {
    Token string `json:"token"`
}
var secret mySecret
if err := golambda.GetSecretValues(os.Getenv("SECRET_ARN"), &secret); err != nil {
    log.Fatal("Failed: ", err)
}

// Access to other service with secret.Token

License

See LICENSE.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EmitError added in v1.1.3 

func EmitError(err error)

func GetSecretValues 

func GetSecretValues(secretArn string, values interface{}) error

GetSecretValues bind secret data of AWS Secrets Manager to values. values should be set as pointer of struct with json meta tag. 

type mySecret struct {
    Token string `json:"token"`
}
var secret mySecret
if err := golambda.GetSecretValues(secretARN, &secret); err != nil {
    log.Fatal("Failed: ", err)
}

func GetSecretValuesWithFactory added in v1.1.0 

func GetSecretValuesWithFactory(secretArn string, values interface{}, factory SecretsManagerFactory) error

GetSecretValuesWithFactory can call SecretsManager.GetSecretValue with your SecretsManagerClient by factory. It uses newDefaultSecretsManager if factory is nil

func NewSecretsManagerMock added in v1.1.2 

func NewSecretsManagerMock() (*SecretsManagerMock, SecretsManagerFactory)

NewSecretsManagerMock returns both of mock and factory method of the mock for testing. Developper can set secrets value as JSON to SecretsManagerMock.Secrets with key (secretes ARN). Also the mock stores Region that is extracted from secretArn and Input of secretsmanager.GetSecretValue when invoking GetSecretValuesWithFactory.

func Start 

func Start(callback Callback)

Start sets up Arguments and logging tools, then invoke Callback with Arguments. When exiting, it also does error handling if Callback returns error

Types

type Callback 

type Callback func(event Event) (interface{}, error)

Callback is callback function type of golambda.Start().

Trigger event data (SQS, SNS, etc) is included in Event.

Callback has 2 returned value. 1st value (interface{}) will be passed to Lambda. The 1st value is allowed nil if you do not want to return any value to Lambda. 2nd value (error) also will be passed to Lambda, however golambda.Start() does error handling: 1) Extract stack trace of error if err is golambda.Error 2) Send error record to sentry.io if SENTRY_DSN is set as environment variable 3) Output error log

type Error 

type Error struct {
	// contains filtered or unexported fields
}

Error is error interface for deepalert to handle related variables

func NewError 

func NewError(msg string) *Error

NewError creates a new error with message

func WrapError 

func WrapError(cause error, msg ...interface{}) *Error

WrapError creates a new Error and add message

func (*Error) Error 

func (x *Error) Error() string

Error returns error message for error interface

func (*Error) Format 

func (x *Error) Format(s fmt.State, verb rune)

Format returns: - %v, %s, %q: formated message - %+v: formated message with stack trace

func (*Error) StackTrace 

func (x *Error) StackTrace() StackTrace

StackTrace returns stack trace that is compatible with pkg/errors

func (*Error) Stacks 

func (x *Error) Stacks() []*Stack

Stacks returns stack trace array generated by pkg/errors

func (*Error) Unwrap 

func (x *Error) Unwrap() error

Unwrap returns *fundamental of github.com/pkg/errors

func (*Error) Values 

func (x *Error) Values() map[string]interface{}

Values returns map of key and value that is set by With. All wrapped golambda.Error key and values will be merged. Key and values of wrapped error is overwritten by upper golambda.Error.

func (*Error) With 

func (x *Error) With(key string, value interface{}) *Error

With adds key and value related to the error event

type Event 

type Event struct {
	Ctx    context.Context
	Origin interface{}
}

Event provides lambda original event converting utilities

func (*Event) Bind 

func (x *Event) Bind(v interface{}) error

Bind does json.Marshal original event and json.Unmarshal to v

func (*Event) DecapSNSMessage 

func (x *Event) DecapSNSMessage() ([]EventRecord, error)

DecapSNSMessage decapsulate wrapped body data in SNSEvent

func (*Event) DecapSNSonSQSMessage 

func (x *Event) DecapSNSonSQSMessage() ([]EventRecord, error)

DecapSNSonSQSMessage decapsulate wrapped body data to in SNSEntity over SQSEvent

func (*Event) DecapSQSBody 

func (x *Event) DecapSQSBody() ([]EventRecord, error)

DecapSQSBody decapsulate wrapped body data in SQSEvent

func (*Event) EncapSNS 

func (x *Event) EncapSNS(v interface{}) error

EncapSNS sets v as SNS entity. This function overwrite Origin for testing. EncapSNS allows both of one record and multiple record as slice or array e.g.) ev.EncapSNS("red") -> one SNS entity in SNSEvent ev.EncapSNS([]string{"blue", "orange"}) -> two SNS entity in SNSEvent

func (*Event) EncapSNSonSQSMessage 

func (x *Event) EncapSNSonSQSMessage(v interface{}) error

EncapSNSonSQSMessage sets v as SNS entity over SQS. This function overwrite Origin and should be used for testing. EncapSNSonSQSMessage allows both of one record and multiple record as slice or array

e.g.) 

ev.EncapSNSonSQSMessage("red") // -> one SQS message on one SQS event
ev.EncapSNSonSQSMessage([]string{"blue", "orange"}) // -> two SQS message on one SQS event

func (*Event) EncapSQS 

func (x *Event) EncapSQS(v interface{}) error

EncapSQS sets v as SQSEvent body. This function overwrite Origin for testing. EncapSQS allows both of one record and multiple record as slice or array e.g.) 

ev.EncapSQS("red") -> one SQSMessage in SQSEvent
ev.EncapSQS([]string{"blue", "orange"}) -> two SQSMessage in SQSEvent

type EventRecord 

type EventRecord []byte

EventRecord is decapsulate event data (e.g. Body of SQS event)

func (EventRecord) Bind 

func (x EventRecord) Bind(ev interface{}) error

Bind unmarshal event record to object

func (EventRecord) String 

func (x EventRecord) String() string

String returns raw string data

type LambdaLogger 

type LambdaLogger struct {
	// contains filtered or unexported fields
}

LambdaLogger provides basic logging features for Lambda function. golambda.Logger is configured by default as global variable of golambda. 

var Logger *LambdaLogger

Logger is common logging interface

func NewLambdaLogger 

func NewLambdaLogger(logLevel string) *LambdaLogger

NewLambdaLogger returns a new LambdaLogger. NOTE: golambda.Logger is recommended for general usage.

func (*LambdaLogger) Debug 

func (x *LambdaLogger) Debug(msg string)

Debug output log as Debug level message

func (*LambdaLogger) Entry 

func (x *LambdaLogger) Entry() *LogEntry

Entry returns a new LogEntry

func (*LambdaLogger) Error 

func (x *LambdaLogger) Error(msg string)

Error output log as Error level message

func (*LambdaLogger) Info 

func (x *LambdaLogger) Info(msg string)

Info output log as Info level message

func (*LambdaLogger) Set 

func (x *LambdaLogger) Set(key string, value interface{})

Set saves key and value to logger. The key and value are output permanently

func (*LambdaLogger) Trace 

func (x *LambdaLogger) Trace(msg string)

Trace output log as Trace level message

func (*LambdaLogger) Warn added in v1.1.2 

func (x *LambdaLogger) Warn(msg string)

Warn output log as Warning level message

func (*LambdaLogger) With 

func (x *LambdaLogger) With(key string, value interface{}) *LogEntry

With adds key and value to log message. Value will be represented by zerolog.Interface

type LogEntry 

type LogEntry struct {
	// contains filtered or unexported fields
}

LogEntry is one record of logging. Trace, Debug, Info and Error methods emit message and values

func (*LogEntry) Debug 

func (x *LogEntry) Debug(msg string)

Debug emits log message as debug level.

func (*LogEntry) Error 

func (x *LogEntry) Error(msg string)

Error emits log message as error level.

func (*LogEntry) Info 

func (x *LogEntry) Info(msg string)

Info emits log message as info level.

func (*LogEntry) Trace 

func (x *LogEntry) Trace(msg string)

Trace emits log message as trace level.

func (*LogEntry) Warn added in v1.1.2 

func (x *LogEntry) Warn(msg string)

Warn emits log message as Warn level.

func (*LogEntry) With 

func (x *LogEntry) With(key string, value interface{}) *LogEntry

With saves key and value into own and return own pointer.

type SecretsManagerClient added in v1.1.0 

type SecretsManagerClient interface {
	GetSecretValue(*secretsmanager.GetSecretValueInput) (*secretsmanager.GetSecretValueOutput, error)
}

SecretsManagerClient is wrapper of secretsmanager.SecretsManager

type SecretsManagerFactory added in v1.1.0 

type SecretsManagerFactory func(region string) (SecretsManagerClient, error)

SecretsManagerFactory is factory function type to replace SecretsManagerClient

type SecretsManagerMock added in v1.1.2 

type SecretsManagerMock struct {
	Secrets map[string]string
	Region  string
	Input   []*secretsmanager.GetSecretValueInput
}

SecretsManagerMock is mock of SecretsManagerClient for testing.

func (*SecretsManagerMock) GetSecretValue added in v1.1.2 

func (x *SecretsManagerMock) GetSecretValue(input *secretsmanager.GetSecretValueInput) (*secretsmanager.GetSecretValueOutput, error)

GetSecretValue is mock method for SecretsManagerMock. It checks if the secretId (ARN) exists in SecretsManagerMock.Secrets as key. It returns a string value if extsting or ResourceNotFoundException error if not existing.

type Stack 

type Stack struct {
	Func string `json:"func"`
	File string `json:"file"`
	Line int    `json:"line"`
}

Stack represents function, file and line No of stack trace

type StackTrace 

type StackTrace []frame

StackTrace is array of frame. It's exported for compatibility with github.com/pkg/errors

func (StackTrace) Format 

func (st StackTrace) Format(s fmt.State, verb rune)

Format formats the stack of Frames according to the fmt.Formatter interface. 

%s	lists source files for each Frame in the stack
%v	lists the source file and line number for each Frame in the stack

Format accepts flags that alter the printing of some verbs, as follows: 

%+v   Prints filename, function, and line number for each Frame in the stack.

Source Files

View all Source files

Directories

Path Synopsis
example
decapEvent
deployable/src

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.