clues

README

CLUES

A golang library for tracking runtime variables via ctx, passing them upstream within errors, and retrieving context- and error-bound variables for logging.

Aggregate runtime state in ctx

Track runtime variables by adding them to the context.

func foo(ctx context.Context, someID string) error {
    ctx = clues.Add(ctx, "importantID", someID)
    return bar(ctx, someID)
}

Keep error messages readable and augment your telemetry by packing errors with structured data.

func bar(ctx context.Context, someID string) error {
    ctx = clues.Add(ctx, "importantID", someID)
    err := errors.New("a bad happened")
    if err != nil {
        return clues.Stack(err).WithClues(ctx)
    }
    return nil
}

Retrive structured data from your errors for logging and other telemetry.

func main() {
    err := foo(context.Background(), "importantID")
    if err != nil {
        logger.
            Error("calling foo").
            WithError(err).
            WithAll(clues.InErr(err))
    }
}

Track individual process flows

Each clues addition traces its additions with a tree of IDs, chaining those traces into the "clues_trace" value. This lets you quickly and easily filter logs to a specific process tree.

func iterateOver(ctx context.Context, users []string) {
    // automatically adds "clues_trace":"id_a"
    ctx = clues.Add(ctx, "status", good)
    for i, user := range users {
        // automatically appends another id to "clues_trace": "id_a,id_n"
        ictx := clues.Add(ctx, "currentUser", user, "iter", i)
        err := doSomething(ictx, user)
        if err != nil {
            ictx = clues.Add(ictx, "status", bad)
        }
    }
}

Interoperable with pkg/errors

Clues errors can be wrapped by pkg/errors without slicing out any stored data.

func getIt(someID string) error {
    return clues.New("oh no!").With("importantID", someID)
}

func getItWrapper(someID string) error {
    if err := getIt(someID); err != nil {
        return errors.Wrap(err, "getting the thing")
    }

    return nil
}

func main() {
    err := getItWrapper("id")
    if err != nil {
        fmt.Println("error getting", err, "with vals", clues.InErr(err))
    }
}

Stackable errors

Error stacking lets you embed error sentinels without slicing out the current error's data or relying on err.Error() strings.

var ErrorCommonFailure = "a common failure condition"

func do() error {
    if err := dependency.Do(); err != nil {
        return clues.Stack(ErrorCommonFailure, err)
    }
    
    return nil
}

func main() {
    err := do()
    if errors.Is(err, ErrCommonFailure) {
        // true!
    }
}

Labeling Errors

Rather than build an errors.As-compliant local error to annotate downstream errors, labels allow you to categorize errors with expected qualities.

Augment downstream errors with labels

func foo(ctx context.Context, someID string) error {
    err := externalPkg.DoThing(ctx, someID)
    if err != nil {
        return clues.Wrap(err).Label("retryable")
    }
    return nil
}

Check your labels upstream.

func main() {
    err := foo(context.Background(), "importantID")
    if err != nil {
        if clues.HasLabel(err, "retryable")) {
            err := foo(context.Background(), "importantID")
        }
    }
}

Design

Clues is not the first of its kind: ctx-err-combo packages already exist. Most other packages tend to couple the two notions, packing both into a single handler. This is, in my opinion, an anti-pattern. Errors are not context, and context are not errors. Unifying the two can couple layers together, and your maintenance woes from handling that coupling are not worth the tradeoff in syntactical sugar.

In turn, Clues maintains a clear separation between accumulating data into a context and passing data back in an error. Both handlers operate independent of the other, so you can choose to only use the ctx (accumulate data into the context, but maybe log it instead of returning data in the err) or the err (only pack immedaite details into the error).

References

• https://github.com/mvndaai/ctxerr
• https://github.com/suzuki-shunsuke/go-errctx

Similar Art

Fault is most similar in design to this package, and also attempts to maintain separation between errors and contexts. The differences are largely syntactical: Fault prefers a composable interface with decorator packages. I like to keep error production as terse as possible, thus preferring a more populated interface of methods over the decorator design.

References

• https://github.com/Southclaws/fault

Documentation

Index

• Constants
• Variables
• func Add(ctx context.Context, kvs ...any) context.Context
• func AddAgent(ctx context.Context, name string) context.Context
• func AddComment(ctx context.Context, msg string, vs ...any) context.Context
• func AddMap[K comparable, V any](ctx context.Context, m map[K]V) context.Context
• func AddSpan(ctx context.Context, name string, kvs ...any) context.Context
• func Close(ctx context.Context) error
• func CloseSpan(ctx context.Context) context.Context
• func In(ctx context.Context) *node.Node
• func InitializeOTEL(ctx context.Context, serviceName string, config OTELConfig) (context.Context, error)
• func InjectTrace[C node.TraceMapCarrierBase](ctx context.Context, mapCarrier C) C
• func ReceiveTrace[C node.TraceMapCarrierBase](ctx context.Context, mapCarrier C) context.Context
• func Relay(ctx context.Context, agent string, vs ...any)
• type OTELConfig

Constants

const (
	DefaultOTELGRPCEndpoint = "localhost:4317"
)

Variables

var ErrMissingOtelGRPCEndpoint = errors.New("missing otel grpc endpoint")

Functions

func Add

func Add(ctx context.Context, kvs ...any) context.Context

Add adds all key-value pairs to the clues.

func AddAgent

func AddAgent(
	ctx context.Context,
	name string,
) context.Context

AddAgent adds an agent with a given name to the context. What's an agent? It's a special case data adder that you can spawn to collect clues for you. Unlike standard clues additions, you have to tell the agent exactly what data you want it to Relay() for you.

Agents are recorded in the current clues node and all of its descendants. Data relayed by the agent will appear as part of the standard data map, namespaced by each agent.

Agents are specifically handy in a certain set of uncommon cases where retrieving clues is otherwise difficult to do, such as working with middleware that doesn't allow control over error creation. In these cases your only option is to relay that data back to some prior clues node.

func AddComment

func AddComment(
	ctx context.Context,
	msg string,
	vs ...any,
) context.Context

AddComment adds a long form comment to the clues.

Comments are special case additions to the context. They're here to, well, let you add comments! Why? Because sometimes it's not sufficient to have a log let you know that a line of code was reached. Even a bunch of clues to describe system state may not be enough. Sometimes what you need in order to debug the situation is a long-form explanation (you do already add that to your code, don't you?). Or, even better, a linear history of long-form explanations, each one building on the prior (which you can't easily do in code).

Should you transfer all your comments to clues? Absolutely not. But in cases where extra explantion is truly important to debugging production, when all you've got are some logs and (maybe if you're lucky) a span trace? Those are the ones you want.

Unlike other additions, which are added as top-level key:value pairs to the context, comments are all held as a single array of additions, persisted in order of appearance, and prefixed by the file and line in which they appeared. This means comments are always added to the context and never clobber each other, regardless of their location. IE: don't add them to a loop.

func AddMap

func AddMap[K comparable, V any](
	ctx context.Context,
	m map[K]V,
) context.Context

AddMap adds a shallow clone of the map to a namespaced set of clues.

func AddSpan

func AddSpan(
	ctx context.Context,
	name string,
	kvs ...any,
) context.Context

AddSpan stacks a clues node onto this context and uses the provided name for the trace id, instead of a randomly generated hash. AddSpan can be called without additional values if you only want to add a trace marker. The assumption is that an otel span is generated and attached to the node. Callers should always follow this addition with a closing `defer clues.CloseSpan(ctx)`.

func Close

func Close(ctx context.Context) error

Close will flush all buffered data waiting to be read. If Initialize was not called, this call is a no-op. Should be called in a defer after initializing.

func CloseSpan

func CloseSpan(ctx context.Context) context.Context

CloseSpan closes the current span in the clues node. Should only be called following a `clues.AddSpan()` call.

func In

func In(ctx context.Context) *node.Node

In retrieves the clues structured data from the context.

func InitializeOTEL

func InitializeOTEL(
	ctx context.Context,
	serviceName string,
	config OTELConfig,
) (context.Context, error)

InitializeOTEL will spin up the OTEL clients that are held by clues, Clues will eagerly use these clients in the background to provide additional telemetry hook-ins.

Clues will operate as expected in the event of an error, or if OTEL is not initialized. This is a purely optional step.

func InjectTrace

func InjectTrace[C node.TraceMapCarrierBase](
	ctx context.Context,
	mapCarrier C,
) C

InjectTrace adds the current trace details to the provided headers. If otel is not initialized, no-ops.

The mapCarrier is mutated by this request. The passed reference is returned mostly as a quality-of-life step so that callers don't need to declare the map outside of this call.

func ReceiveTrace

func ReceiveTrace[C node.TraceMapCarrierBase](
	ctx context.Context,
	mapCarrier C,
) context.Context

ReceiveTrace extracts the current trace details from the headers and adds them to the context. If otel is not initialized, no-ops.

func Relay

func Relay(
	ctx context.Context,
	agent string,
	vs ...any,
)

Relay adds all key-value pairs to the provided agent. The agent will record those values to the node in which it was created. All relayed values are namespaced to the owning agent.

Types

type OTELConfig

type OTELConfig struct {
	// specify the endpoint location to use for grpc communication.
	// If empty, no telemetry exporter will be generated.
	// ex: localhost:4317
	// ex: 0.0.0.0:4317
	// ex: opentelemetry-collector.monitoring.svc.cluster.local:4317
	GRPCEndpoint string
}