aws-embedded-metrics-golang

module
v1.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 29, 2026 License: MIT

README

aws-embedded-metrics-golang

Go Reference ci security

A Go library for the AWS CloudWatch Embedded Metric Format (EMF).

EMF lets you publish CloudWatch metrics by writing JSON log lines. CloudWatch Logs reads them and creates the metrics for you, so there are no CloudWatch API calls, credentials or clients in your code, and nothing to mock in tests.

  • Simple, chainable API for metrics, dimensions, properties and namespaces
  • Lists of values and high-resolution (1 second) metrics
  • Output always follows the EMF spec, and is tested against AWS's JSON Schema and fuzzed
  • Bad input is skipped and reported, instead of silently breaking the whole log line
  • Safe to use from several goroutines at once
  • Lambda dimensions and properties added for you
  • The library itself only uses the standard library

Forked from prozz/aws-embedded-metrics-golang.

Installation

Requires Go 1.27 or newer.

go get github.com/dangrier/aws-embedded-metrics-golang

Usage

import "github.com/dangrier/aws-embedded-metrics-golang/emf"

emf.New().
	Namespace("shop").
	Dimension("Service", "checkout").
	MetricAs("OrderTotal", 120, emf.Count).
	Log()

This writes one line to stdout:

{"OrderTotal":120,"Service":"checkout","_aws":{"Timestamp":1791500000000,"CloudWatchMetrics":[{"Namespace":"shop","Dimensions":[["Service"]],"Metrics":[{"Name":"OrderTotal","Unit":"Count"}]}]}}

In Lambda, stdout already goes to CloudWatch Logs. Elsewhere, use the CloudWatch agent or another log shipper.

There are runnable examples in the package docs.

Metrics
logger := emf.New()

logger.Metric("Requests", 1)                            // int
logger.MetricFloat("Ratio", 0.75)                       // float64
logger.MetricAs("Errors", 2, emf.Count)                 // int with a unit
logger.MetricFloatAs("Latency", 12.5, emf.Milliseconds) // float64 with a unit
logger.Metrics(map[string]int{"Hits": 9, "Misses": 1})  // several at once
logger.MetricsFloatAs(map[string]float64{"P50": 10, "P99": 48}, emf.Milliseconds)

Put and PutValues take options for the unit and resolution:

logger.Put("Latency", 12.5, emf.Unit(emf.Milliseconds))
logger.Put("QueueDepth", 42, emf.Unit(emf.Count), emf.HighResolution())
logger.PutValues("Latency", []float64{12.5, 8.1, 30.2}, emf.Unit(emf.Milliseconds))

Every CloudWatch unit has a constant, like emf.Seconds, emf.Bytes, emf.Percent and emf.CountSecond.

Lists of values

A metric can have a list of values, such as the latency of each request in a batch. CloudWatch keeps every value, so statistics like percentiles stay accurate. Use PutValues, or log the same name more than once and the values are collected:

logger.Put("Latency", 12.5).Put("Latency", 8.1) // logs "Latency":[12.5,8.1]

The same name in two contexts is published in both namespaces, sharing all of its values, even when the contexts are written on separate lines. So a repeated name must use the same unit and resolution everywhere, or the new value is skipped and reported. The spec allows at most 100 values per metric, so longer lists are split across several lines.

High resolution

emf.HighResolution() stores a metric per second instead of per minute, so you can graph and alarm on it at 1 second intervals. High-resolution metrics cost more.

Dimensions
// Each call adds its own dimension set, so each metric is published once per set.
logger.Dimension("Service", "checkout").Dimension("Region", "ap-southeast-2")

// A set with the same keys as one already added, in any order, isn't added again, but its
// values are updated. This leaves one ["Service"] set, with Service now "cart".
logger.Dimension("Service", "cart")

// One set with both dimensions, so each metric is published once for the combination.
logger.DimensionSet(
	emf.NewDimension("Service", "checkout"),
	emf.NewDimension("Region", "ap-southeast-2"),
)

A key repeated within one set is written once, with the last value. Repeats are only removed within a context, since each context is a separate directive with its own namespace.

Properties

Properties are logged with the metrics, but don't become metrics or dimensions. They're handy for searching in CloudWatch Logs Insights.

logger.Property("requestId", requestID)

A property shares the log line with dimensions and metrics, so each key can only hold one of them. A property with the key of a metric or dimension is skipped and reported. A dimension or metric added later with a property's key is kept, and replaces the property, which is reported too. A dimension replaces it on every line, and a metric on the lines the metric is written on. The Lambda properties the library adds (see Lambda defaults) can be replaced without a report.

Namespaces and contexts

Metrics go in the aws-embedded-metrics namespace unless you set one. A context adds metrics with their own namespace and dimensions to the same log line.

logger := emf.New().Namespace("shop").Metric("Orders", 3)
logger.NewContext().
	Namespace("payments").
	Dimension("Provider", "card").
	MetricAs("Declined", 1, emf.Count)
logger.Log()

Each context keeps its own dimension values. Contexts that disagree on a dimension value (the same key with different values) are written on separate lines, so each context's metrics are published under its own value. Contexts that agree still share a line. Properties go on every line.

logger := emf.New()
logger.NewContext().Namespace("payments").Dimension("Provider", "card").Metric("Declined", 1)
logger.NewContext().Namespace("payments").Dimension("Provider", "paypal").Metric("Refunds", 2)
logger.Log() // two lines: one with "Provider":"card", one with "Provider":"paypal"
Logging with defer

Create a new logger for each unit of work, such as a request or a batch, and call Log() once at the end:

func processOrder(id string) error {
	metrics := emf.New().Namespace("api").Dimension("Operation", "ProcessOrder")
	defer metrics.Log()

	start := time.Now()
	// ... process the order ...
	metrics.MetricFloatAs("Latency", float64(time.Since(start).Milliseconds()), emf.Milliseconds)
	return nil
}

A logger isn't meant to be reused. Each Log() writes every value added since New(), so calling it again writes the earlier values again. A metric logged once before each of two Log() calls comes out as "Requests":1, then "Requests":[1,1]. Values keep building up for as long as the logger lives, and every line keeps the timestamp set when New() was called (or by WithTimestamp).

In Lambda, create the logger inside the handler, not at package level or in init. A logger shared across invocations double counts, keeps growing, and stamps every metric with the cold-start time.

Options

emf.New(
	emf.WithWriter(os.Stderr),                     // write somewhere other than stdout
	emf.WithTimestamp(time.Now().Add(-time.Hour)), // record metrics for another time
	emf.WithoutDimensions(),                       // leave out the Lambda dimensions and properties
	emf.WithLogGroup("my-logs"),                   // log group for the CloudWatch agent
	emf.WithErrorHandler(func(err error) {         // hear about skipped input and write errors
		log.Println(err)
	}),
)
Lambda defaults

When AWS_LAMBDA_FUNCTION_NAME is set, each logger adds:

  • a dimension set of ServiceName (the function name) and ServiceType (AWS::Lambda::Function)
  • executionEnvironment, memorySize, functionVersion and logStreamId properties

Use WithoutDimensions() to leave these out. If X-Ray sampled the request, the trace ID is also added as a traceId property.

Spec compliance and errors

Output always follows the EMF specification. Input that would break it is skipped, and the rest is still logged. This includes NaN or infinite values, invalid names, unknown units, and a name used for both a metric and a dimension.

Some input is reported, but still written:

  • Timestamp: CloudWatch only publishes metrics with a timestamp from 14 days in the past to 2 hours in the future. It keeps a line outside that window as a log event, but won't turn it into metrics.
  • Size: CloudWatch Logs rejects log lines over 1 MB, usually because of very large properties.
  • Replaced property: a dimension or metric with the same key as a property replaces it. See Properties.

Use WithErrorHandler to find out about any of these. The errors wrap emf.ErrInvalid:

emf.New(emf.WithErrorHandler(func(err error) {
	if errors.Is(err, emf.ErrInvalid) {
		log.Println("metrics:", err)
	}
}))
More than 100 metrics or values

A single call to Log() may write more than one line. The spec allows at most 100 metrics per log event, and 100 values per metric, and CloudWatch drops every metric in an event that has more. So if you log more than 100 metrics at once (across all contexts that share a line), or a metric with more than 100 values, they are split across several lines. Contexts that disagree on a dimension value are also written on separate lines, and each of those is split the same way. Every line keeps the same timestamp and properties, and each metric keeps its own namespace, dimensions and dimension values. If your code reads the output (in tests, for example), expect one or more lines per Log().

Concurrency

A Logger and its contexts are safe to use from several goroutines at once. Lines from concurrent Log() calls are never mixed together. The error handler may be called from several goroutines too, and it can safely use the logger.

Development

Common tasks use just:

Command What it does
just check Build, vet, lint and test, the same as CI
just test-race Tests with the race detector and coverage
just fuzz [time] Fuzz the output against the EMF spec (default 1 minute)
just bench Run the benchmarks
just bench-compare Compare benchmarks with main, and fail on a regression
just scan Scan for known vulnerabilities with Grype and govulncheck
just fmt Format the code

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. Please make sure to update tests.

License

MIT

Directories

Path Synopsis
Package emf writes CloudWatch metrics in the AWS Embedded Metric Format (EMF).
Package emf writes CloudWatch metrics in the AWS Embedded Metric Format (EMF).
internal
cmd/benchcompare command
Command benchcompare compares benchmarks between a git ref and the working tree, and fails on a significant performance regression.
Command benchcompare compares benchmarks between a git ref and the working tree, and fails on a significant performance regression.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL