gcp-clf

README

GCP Cloud Logging -> Combined Log Format

This repo contains utilities for exporting request logs from Cloud Logging to the Combined Log Format.

The Combined Log Format is just the Common Log Format with two additional fields to record the Referer and User-Agent of a request. This is the log format commonly used by NGINX and Apache web servers, and so is supported by log processing tools like GoAccess

Currently, this tool supports extracting logs specifically from Firebase Hosting). To use this tool with Firebase hosting logs, make sure to link Cloud Logging.

Usage with GoAccess

To view logs from your server over a time period with GoAccess, run:

go run ./cmd/exporter logs <project ID> \
  --start_time=<RFC3339 time> \
  --end_time=<RFC3339 time> > access.log

# For terminal UI
goaccess --log-format=combined access.log

# For web UI
goaccess --log-format=combined access.log -o report.html
# Then open `report.html` in your web browser of choice.

To view streaming logs, run:

# For terminal UI
go run ./cmd/exporter stream <project ID> | goaccess --log-format=COMBINED -

# For web UI
go run ./cmd/exporter stream <project ID> \
  | goaccess --log-format=COMBINED --real-time-html -o report.html -
# Then open `report.html` in your web browser of choice.

Contributing

Contribution guidelines can be found on our website.