go-library

README

go-library

Note for AI assistants: read CLAUDE.md first — it contains the condensed architecture, conventions, and commands for this repo. Prefer it over re-deriving context from the source tree to keep token usage low.

Reusable Go utilities for enterprise services. The centerpiece is a universal structured logger built on the standard library's log/slog with:

• One-call init — config-driven setup with sane defaults
• Request lifecycle tracing — a UUID request ID is generated the moment a request enters the app and persists (via context.Context) for its entire life, across REST, GraphQL, ETL, and Kafka boundaries
• Performance diagnostics — Powertools-style runtime snapshot (heap, GC, goroutines, cold start, uptime) attached to every completed request
• Platform aware — auto-detects Lambda, ECS, Kubernetes, container, or local and enriches logs accordingly (region, function name, pod, ...)
• Zero dependencies for the core; only graphql/gqlgen pulls in gqlgen, only oauth/redis pulls in go-redis, and only secrets/aws pulls in aws-sdk-go-v2

Packages

Package	Purpose
logger	Base logger: config, levels, request IDs, lifecycle, performance
requestid	Dependency-free UUIDv4 generator
rest	net/http middleware (works with chi, gorilla, plain mux)
graphql	Server-agnostic GraphQL operation logger with variable redaction
graphql/gqlgen	Drop-in gqlgen handler extension
etl	Batch/pipeline job logging with checkpoints and throughput
events	Client-agnostic Kafka consume/produce logging with header-based request ID propagation
oauth	OAuth 2.0 token acquisition (client_credentials, Salesforce password flow) with optional cache-aware fetching
oauth/redis	go-redis v9 adapter for oauth's token cache
secrets	Structured-logging loader over a Source (secrets + SSM parameters), with JSON parsing for key/value secrets
secrets/aws	aws-sdk-go-v2 adapter for secrets.Source (Secrets Manager + SSM Parameter Store)

Quickstart

log := logger.Init(logger.Config{
    Service:     "orders-graph",   // required
    Version:     "1.4.2",          // build tag or commit SHA
    Environment: "prod",
    Team:        "commerce",
    // Platform, Format, Level, Output all default sensibly:
    // platform auto-detected, JSON on deployed platforms / text locally, info level, stdout.
})

log.Info(ctx, "server listening", "port", 8080)
log.Error(ctx, "save failed", logger.Err(err))

Every line automatically carries service, version, environment, platform, hostname, pid, goVersion, region/function/pod when applicable, and — once a request ID is in the context — requestId.

REST

mux := http.NewServeMux()
mux.HandleFunc("/orders", ordersHandler)

handler := rest.Middleware(log, rest.WithSkipPaths("/health"))(mux)
http.ListenAndServe(":8080", handler)

The middleware accepts an inbound X-Request-ID (or generates one), echoes it in the response, logs completion with method/path/status/bytes/duration plus a performance snapshot, and recovers panics with a stack trace.

GraphQL (gqlgen)

import (
    graphqllog "github.com/JHeat89/go-library/graphql"
    gqlgenlog "github.com/JHeat89/go-library/graphql/gqlgen"
)

gl := graphqllog.New(log,
    graphqllog.WithRedactedVariables("password", "token"),
    graphqllog.WithMaxQueryLength(2000),
)

srv := handler.NewDefaultServer(generated.NewExecutableSchema(cfg))
srv.Use(gqlgenlog.New(gl))

Using a different GraphQL server? Call gl.OperationStart(ctx, op) yourself — the core has no gqlgen dependency.

ETL

ctx, job := etl.StartJob(ctx, log, "orders-nightly-export")
for batch := range batches {
    n, failed := process(batch)
    job.Processed(n)
    job.Failed(failed)
    job.Checkpoint(ctx, "batch flushed")
}
job.Complete(ctx) // or job.Fail(ctx, err)

Checkpoints log running totals, elapsed time, and records/sec so stalled runs are visible mid-flight. Each run gets its own request ID.

Kafka events (any client)

ev := events.New(log)

// Consuming — request ID is picked up from the x-request-id message header,
// preserving correlation with the producing service:
ctx, done := ev.ConsumeStart(ctx, events.Message{
    Topic:         m.Topic,
    Partition:     m.Partition,
    Offset:        m.Offset,
    Key:           string(m.Key),
    ConsumerGroup: "orders-consumer",
    Headers:       headerMap(m.Headers),
    Timestamp:     m.Time, // enables consumer-lag logging
})
err := handle(ctx, m)
done(err)

// Producing — stamp outgoing messages so downstream consumers correlate:
for k, v := range events.OutgoingHeaders(ctx) {
    msg.Headers = append(msg.Headers, kafka.Header{Key: k, Value: []byte(v)})
}
ev.Produced(ctx, events.Message{Topic: msg.Topic, Key: string(msg.Key)}, writeErr)

OAuth token acquisition

import (
    "github.com/JHeat89/go-library/oauth"
    oauthredis "github.com/JHeat89/go-library/oauth/redis"
)

client, err := oauth.New(log, oauth.Config{
    GrantType:    oauth.GrantClientCredentials, // or oauth.GrantSalesforcePassword
    TokenURL:     "https://login.salesforce.com/services/oauth2/token",
    ClientID:     os.Getenv("OAUTH_CLIENT_ID"),
    ClientSecret: os.Getenv("OAUTH_CLIENT_SECRET"),

    // Cache-aware fetching — optional, independently switchable:
    Cache:      oauthredis.New(redisClient), // any oauth.TokenCache works
    CacheRead:  true,
    CacheWrite: true,
})

tok, err := client.Token(ctx) // cache-aware: reads Cache first when CacheRead is set
req.Header.Set("Authorization", tok.TokenType+" "+tok.AccessToken)

// After a downstream 401, force a new grant rather than re-reading a cache
// entry that may be the one that just failed:
tok, err = client.Refresh(ctx)

Salesforce's username/password flow concatenates Password and SecurityToken in the form body and omits expires_in in its response, so Config.DefaultTTL (default 15m) governs the cached lifetime instead.

There is no in-memory token cache — every Token call consults Cache (when CacheRead is set) or fetches fresh, since an external process may refresh the cached token out-of-band. AccessToken, ClientSecret, Password, SecurityToken, and raw response bodies are never logged, independent of the base logger's RedactKeys configuration. Only oauth/redis imports go-redis; any store can implement oauth.TokenCache directly.

Secrets and parameters (AWS)

import (
    "github.com/JHeat89/go-library/secrets"
    awssecrets "github.com/JHeat89/go-library/secrets/aws"
    "github.com/aws/aws-sdk-go-v2/config"
)

cfg, err := config.LoadDefaultConfig(ctx) // consumer owns the aws.Config
src := awssecrets.New(cfg)                // Secrets Manager + SSM Parameter Store
loader := secrets.New(log, src)

// A Secrets Manager secret stored as a JSON object of key/value pairs:
dbCreds, err := loader.SecretJSON(ctx, "orders/db")
dsn := fmt.Sprintf("postgres://%s:%s@...", dbCreds["username"], dbCreds["password"])

// A single decrypted SSM parameter:
apiKey, err := loader.Parameter(ctx, "/orders/payments-api-key")

// Every parameter under a path, recursively:
flags, err := loader.ParametersByPath(ctx, "/orders/feature-flags")

// Wiring a loaded secret straight into another package's config:
oauthCreds, err := loader.SecretJSON(ctx, "orders/oauth-client")
client, err := oauth.New(log, oauth.Config{
    TokenURL:     "https://auth.example.com/oauth2/token",
    ClientID:     oauthCreds["clientId"],
    ClientSecret: oauthCreds["clientSecret"],
})

secrets.Loader does no caching — load once at startup and hold the values; add a TTL cache on top if you need one later. Only names, paths, source, and duration are ever logged — secret and parameter values never are. Only secrets/aws imports aws-sdk-go-v2; any store can implement secrets.Source directly.

Request lifecycle tracing

Any layer can add stages to the current request without plumbing extra arguments — the trace lives in the context:

ctx, lc := log.StartRequest(ctx, "GET /orders") // done for you by rest/graphql/etl/events
defer lc.End(ctx)

logger.LifecycleFrom(ctx).Stage(ctx, "db.query")
logger.LifecycleFrom(ctx).Stage(ctx, "downstream.inventory")

The completion line includes total durationMs, every stage with per-stage timing, and a performance group:

{
  "timestamp": "2026-07-24T20:15:01Z",
  "level": "info",
  "message": "request completed",
  "service": "orders-graph",
  "requestId": "9f8b7c6d-1a2b-4c3d-8e9f-0a1b2c3d4e5f",
  "request": "GET /orders",
  "durationMs": 42.7,
  "stages": [{"name": "db.query", "elapsedMs": 12.1, "sinceMs": 12.1}],
  "performance": {
    "heapAllocMB": 12.4, "heapSysMB": 24.0, "numGC": 3,
    "gcPauseTotalMs": 0.8, "goroutines": 14, "numCPU": 8,
    "uptimeSec": 1042.5, "coldStart": false
  }
}

Redaction

Configured once on the base logger, applied to every log line from every logger type (REST, GraphQL, ETL, events, raw slog):

log := logger.Init(logger.Config{
    Service: "orders-graph",
    // Key-based: case-insensitive, recursive into nested maps/slices/groups.
    RedactKeys: []string{"password", "token", "ssn"},
    // Pattern-based: scrubs structured PII (emails, SSNs, credit cards,
    // phone numbers) out of the middle of any string, message included.
    RedactPII: true,
    // Custom shapes the built-ins can't know:
    RedactPatterns: []logger.Pattern{
        {Regexp: regexp.MustCompile(`(?i)(customer )(\S+)`), Replacement: "${1}[REDACTED]"},
    },
})

log.Info(ctx, "Starting Execution for Customer JoeyCox notify joey@example.com")
// message: "Starting Execution for Customer [REDACTED] notify [REDACTED]"

Key-based redaction replaces the whole value; pattern-based replaces only the matched text so messages stay readable. Arbitrary personal names in free text are not rule-detectable — cover known message shapes with RedactPatterns or put names in dedicated fields and list those in RedactKeys.

GraphQL variable redaction (graphql.WithRedactedVariables) uses the same recursive engine and now catches keys nested inside input objects.

Runtime level changes

log.SetLevel("debug") // flip a live instance to debug while triaging

Requirements

Go 1.26+.

License

MIT — free to use, modify, and distribute. See LICENSE.