README
¶
cloudevents
CloudEvents in binary mode, over a header map. A codec, never a protocol binding, so the same envelope crosses NATS today and HTTP tomorrow without this module knowing which it was.
go get gitlab.com/phpboyscout/go/cloudevents
The whole API
func Marshal(e Event) (map[string][]string, []byte, error)
func Unmarshal(h map[string][]string, body []byte) (Event, error)
map[string][]string is the underlying type of both nats.Header and http.Header, so neither is
imported here and a caller converts with a type conversion rather than a copy.
h, body, err := cloudevents.Marshal(e)
if err != nil {
return err
}
err = conn.PublishMsg(&nats.Msg{Subject: "orders.created", Header: nats.Header(h), Data: body})
The problem it solves
Two services agree on a message shape. Six months later there are five services, three of them written by somebody who has left, and the shape is whatever the oldest one happens to emit.
CloudEvents is that agreement written down by somebody else, with an ecosystem that already reads it.
This module implements the specification — eight attributes, one prefix, one special case — rather
than taking the SDK, whose core requires go.uber.org/zap, json-iterator, go-cmp,
bytebufferpool and six more. This estate is slog everywhere; a second structured logger in every
service that sends a message is a poor trade for an envelope.
We own conformance instead, and the test suite carries it: fixtures written by hand from the
specification, and one produced by the CloudEvents SDK itself, checked in at
testdata/sdk-binary-1.0.txt. A round trip through our own code would pass happily if every header
were renamed consistently. That fixture is what would not.
What it deliberately does not do
| Why | |
|---|---|
| Touch a transport | The SDK's bindings own the nats.Conn, which would mean a CloudEvent bypassed go/nats and lost its bounded subscriptions, shed reporting and lifecycle. TestNoTransportIsImported enforces the absence |
| Implement structured mode | It hides the attributes from anything routing on them. Additive later; nothing written today becomes wrong |
Resolve a dataschema |
That is go/schema's job, and keeping it out is what lets a service use CloudEvents with no registry at all |
| Parse leniently | specversion tells a reader how to interpret every other attribute, so parsing past an unrecognised one is guessing rather than tolerance |
Values are percent-encoded in both directions, as both bindings require, so a subject as
ordinary as order 4172 survives the trip. Control characters are refused rather than encoded, and
an extension may not be named after a specification attribute — without that, Extensions["id"]
would overwrite ce-id and the event would marshal with a hijacked identity and no error.
Refusing, and still saying which event
Unmarshal validates and refuses — and returns the partially-parsed event alongside the error.
Core NATS has no dead letter, so a handler that returns on error has destroyed the message. If all it can log is "a message failed to parse", the event is unidentifiable and nobody can go and look at the producer.
e, err := cloudevents.Unmarshal(msg.Header, msg.Data)
if err != nil {
refused.Add(1)
logger.Error("refused a CloudEvent", "id", e.ID, "source", e.Source, "err", err)
return
}
Identifiers
gitlab.com/phpboyscout/go/cloudevents/urn carries the estate's URN grammar and depends on nothing
but go/errors, so anything can parse one without taking the
codec.
urn:phpboyscout:schema:orders.created:3
urn:phpboyscout:scope:continuity:01JQ8Z3F7K2M4N6P8R0T2V4W6X
A dataschema outlives the infrastructure that served it — it sits in logs and object stores for
years, long after a host has been renamed or replaced — so it is a URN rather than a URL. Identity is
separated from location, and resolution becomes configuration.
scope is the estate's tenancy attribute and is ADVISORY, AND NEVER THE AUTHORITY. It is a
claim in a message, for routing and legibility. Enforcement is NATS accounts with subject-scoped
credentials; a consumer that filters on scope has isolated nothing.
Documentation
Full documentation at cloudevents.go.phpboyscout.uk, and the API on pkg.go.dev.
The decisions behind all of the above are in spec 0001.
Licence
MIT. See LICENSE.
Documentation
¶
Overview ¶
Package cloudevents marshals and unmarshals CloudEvents in binary mode over a header map, and does nothing else.
A codec, not a protocol binding ¶
This module never opens a connection, subscribes, or publishes. It converts between an Event and the pair of values every message transport already has: a map of headers, and a body.
func Marshal(e Event) (map[string][]string, []byte, error) func Unmarshal(h map[string][]string, body []byte) (Event, error)
map[string][]string is the shape of both nats.Header and http.Header, so neither transport is imported here and the same codec works over either. The CloudEvents SDK's own bindings take the opposite approach — they own the connection — which would mean a service sending a CloudEvent over NATS bypassed go/nats and lost its bounded subscriptions, shed reporting and lifecycle management. Spec 0001 D1.
Binary mode ¶
Attributes ride in headers with a ce- prefix, datacontenttype rides in the unprefixed content-type header, extensions are any other ce--prefixed key with the prefix stripped, and the payload is the raw body. That is the mapping the CloudEvents SDK uses, which is what makes this module's output readable by any other CloudEvents consumer.
Structured mode — the whole envelope as JSON in the body — is not built. It hides the attributes from anything routing on them.
Strictness ¶
Unmarshal validates and refuses, and hands back what it managed to read alongside the error. specversion tells a reader how to interpret every other attribute, so parsing leniently past an unrecognised one is guessing rather than tolerance. The partial event is returned because a subscriber that drops a message needs something traceable to log: core NATS has no dead letter, so a refused message is gone unless something records which one it was.
If this codec is ever pointed at events produced outside the estate, revisit that: there, strictness is a denial of service anybody can cause.
The URN grammar ¶
The estate's dataschema and scope identifiers are URNs, and their grammar lives in gitlab.com/phpboyscout/go/cloudevents/urn — a package with no dependencies, so anything can parse one without taking the codec.
Index ¶
Examples ¶
Constants ¶
const ( // Prefix is what every attribute header key starts with, except // content-type. Prefix = "ce-" // HeaderContentType carries the datacontenttype attribute. HeaderContentType = "content-type" )
The wire mapping. Attributes ride in ce--prefixed headers; datacontenttype rides in the unprefixed content-type header the transport already has a meaning for.
const ( HeaderID = Prefix + "id" HeaderSource = Prefix + "source" HeaderSpecVersion = Prefix + "specversion" HeaderType = Prefix + "type" HeaderDataSchema = Prefix + "dataschema" HeaderSubject = Prefix + "subject" HeaderTime = Prefix + "time" // HeaderDataContentType is the NATS binding's spelling of datacontenttype. // // The two bindings genuinely differ here: the HTTP binding maps // datacontenttype onto Content-Type and says a ce-datacontenttype header // MUST NOT also be present, while the NATS binding prefixes it like every // other attribute. This codec writes the HTTP form — that is what the SDK's // NATS binding emits in practice — and accepts either on read, because // refusing a conformant NATS peer's content type would lose it silently. HeaderDataContentType = Prefix + "datacontenttype" )
The ce--prefixed header keys, in the canonical lower case this codec writes.
const SpecVersion = "1.0"
SpecVersion is the only CloudEvents version this codec implements.
Variables ¶
var ( // ErrMissingAttribute is returned when one of the four required attributes // — id, source, specversion, type — is absent or empty. ErrMissingAttribute = errors.NewSentinel("cloudevents.missing_attribute", "cloudevents: a required attribute is missing") // ErrUnsupportedSpecVersion is returned for a specversion this codec does // not implement. // // This is the refusal the rest of the strictness follows from. specversion // tells a reader how to interpret every other attribute, so continuing past // an unrecognised one is guessing at the shape of what is held rather than // being lenient about it. ErrUnsupportedSpecVersion = errors.NewSentinel("cloudevents.unsupported_specversion", "cloudevents: unsupported specversion") // ErrInvalidTime is returned when the time attribute is present and is not // an RFC 3339 timestamp. ErrInvalidTime = errors.NewSentinel("cloudevents.invalid_time", "cloudevents: time is not an RFC 3339 timestamp") // ErrDataSchemaNotAbsolute is returned when dataschema is present and is // not an absolute URI. // // The CloudEvents specification types dataschema as URI rather than // URI-reference, so unlike source it must be absolute: there is no base to // resolve a relative one against, and a consumer that received one could // not tell which schema was meant. ErrDataSchemaNotAbsolute = errors.NewSentinel("cloudevents.dataschema_not_absolute", "cloudevents: dataschema must be an absolute URI") // ErrInvalidExtensionName is returned when an extension attribute name // breaks the specification's naming MUSTs. ErrInvalidExtensionName = errors.NewSentinel("cloudevents.invalid_extension_name", "cloudevents: extension names must be one or more lower-case ASCII letters or digits, and not \"data\"") // ErrInvalidEncoding is returned when a header value is not validly // percent-encoded. // // Both bindings require a single round of percent-decoding on receive, and // a truncated or non-hexadecimal escape means the sender and this reader // disagree about where the value ends. Decoding it leniently would produce // a different string than the sender wrote, silently. ErrInvalidEncoding = errors.NewSentinel("cloudevents.invalid_encoding", "cloudevents: header value is not validly percent-encoded") // ErrControlCharacter is returned when an attribute value contains a // character the specification's String type disallows. // // U+0000-U+001F and U+007F-U+009F. Percent-encoding would carry them, which // is precisely why they are refused instead: net/http rewrites a newline in // a header rather than refusing it, so without this check a receiver gets a // different attribute value than the sender set, with no error anywhere. ErrControlCharacter = errors.NewSentinel("cloudevents.control_character", "cloudevents: attribute value contains a control character") // ErrReservedExtensionName is returned when an extension is named after a // specification attribute. // // Without this, Extensions["id"] overwrites ce-id on the way out and the // event marshals with a hijacked identity and no error — a round trip that // is green and wrong. ErrReservedExtensionName = errors.NewSentinel("cloudevents.reserved_extension_name", "cloudevents: extension name collides with a specification attribute") // ErrAmbiguousHeader is returned when one attribute arrives twice. // // Header maps differ in whether they fold case — http.Header canonicalises // keys, nats.Header does not — so this codec lower-cases on read. Two keys // that fold together with different values are then a real ambiguity, and // picking either would make the event mean different things on different // transports. ErrAmbiguousHeader = errors.NewSentinel("cloudevents.ambiguous_header", "cloudevents: an attribute was supplied more than once with different values") )
Refusals. Every one of them is a case Unmarshal or Marshal declines to guess at, and each is a sentinel so a consumer can count the kinds of bad event it is being sent rather than counting "parse failed".
Functions ¶
func Marshal ¶
Marshal renders an event as a header map and a body.
The header map is a plain map[string][]string, which is the underlying type of both nats.Header and http.Header, so a caller converts with a type conversion and this module imports no transport.
Two things a caller must know ¶
Keys are written in lower case, which is what the NATS binding uses on the wire. [http.Header.Get] canonicalises the key it is given, so it will NOT find them: read the returned map by iteration or by exact key, and let net/http canonicalise on write. Values are percent-encoded per the bindings, so a value read back out of this map is not the string that went in — use Unmarshal rather than reading it by hand.
The returned body ALIASES e.Data rather than copying it. A producer that reuses an encode buffer will mutate a message it has already handed to a transport. The copy is skipped deliberately, because a codec on a message hot path should not double every payload, but the ownership is the caller's to respect.
SpecVersion is filled in when empty, because there is exactly one value this codec can write and making every caller repeat it invites the typo.
Example ¶
ExampleMarshal builds an event and renders it. SpecVersion is left unset because there is one value this codec can write.
package main
import (
"fmt"
"maps"
"slices"
"time"
"gitlab.com/phpboyscout/go/cloudevents"
)
func main() {
e := cloudevents.Event{
ID: "01JQ8Z3F7K2M4N6P8R0T2V4W6X",
Source: "/phpbotscout/ingest",
Type: "uk.phpboyscout.orders.created",
Subject: "order-4172",
Time: time.Date(2026, 8, 28, 11, 22, 33, 0, time.UTC),
DataContentType: "application/json",
Data: []byte(`{"order":"4172"}`),
}
h, body, err := cloudevents.Marshal(e)
if err != nil {
fmt.Println("marshal:", err)
return
}
for _, key := range slices.Sorted(maps.Keys(h)) {
fmt.Printf("%s: %s\n", key, h[key][0])
}
fmt.Printf("\n%s\n", body)
}
Output: ce-id: 01JQ8Z3F7K2M4N6P8R0T2V4W6X ce-source: /phpbotscout/ingest ce-specversion: 1.0 ce-subject: order-4172 ce-time: 2026-08-28T11:22:33Z ce-type: uk.phpboyscout.orders.created content-type: application/json {"order":"4172"}
func ValidExtensionName ¶
ValidExtensionName reports whether a name may be used as an extension attribute.
It enforces the specification's naming MUSTs — one or more characters, each a lower-case ASCII letter or a digit, and not the name "data" which some event formats reserve — and deliberately not its SHOULDs. Starting with a letter and staying under twenty characters are recommendations, and refusing a twenty-one character name would reject events the rest of the world accepts, which is the interoperability failure that implementing a standard rather than inventing one exists to avoid.
It also refuses the eight specification attribute names. The specification calls extensions "additional context attributes with distinct names", and without the check an extension named "id" would overwrite ce-id on the way out — producing an event with a hijacked identity, no error, and a round trip that agrees with itself.
The character constraint is this narrow because some protocols treat metadata as case-sensitive and others do not, and one event may cross several of them in a single delivery.
Types ¶
type Event ¶
type Event struct {
// ID, with Source, uniquely identifies the event. A producer must not reuse
// a pair for two different events.
ID string
// Source identifies the context the event happened in. A URI-reference, so
// unlike DataSchema it may be relative.
Source string
// SpecVersion is the version of the CloudEvents specification the event
// uses. [Marshal] fills it in when it is empty; [Unmarshal] refuses any
// value but [SpecVersion].
SpecVersion string
// Type describes the kind of occurrence, and is what consumers usually
// filter on.
Type string
// DataContentType is the media type of Data. It travels in the unprefixed
// content-type header, not in a ce- one.
DataContentType string
// DataSchema identifies the schema Data conforms to. In this estate it is a
// URN — see [gitlab.com/phpboyscout/go/cloudevents/urn] — and it must be an
// absolute URI whatever form it takes.
DataSchema string
// Subject names the thing the event is about, within Source.
Subject string
// Time is when the occurrence happened. The zero value means absent.
Time time.Time
// Extensions are attributes outside the specification's set, keyed by name
// without the ce- prefix.
//
// Every one of them must be safe to log: opaque identifiers only, never a
// display name, a channel name or anything a person chose. Attributes ride
// in headers, headers are logged by default, and nothing routes them
// through a redaction pass the way a payload field is routed. A payload has
// a schema and a review; an attribute has neither, so the bar for adding
// one is higher rather than lower.
Extensions map[string]string
// Data is the payload, carried as the message body and never interpreted
// here.
Data []byte
}
Event is a CloudEvent: eight attributes, any number of extensions, and a payload this module never looks inside.
It is a plain struct rather than an interface with accessors because it holds eight fields and a map, and the SDK's equivalent brings a second structured logger, a JSON library and four more indirect dependencies into the graph of every service that sends a message.
func Unmarshal ¶
Unmarshal reads an event from a header map and a body.
It validates and refuses rather than parsing what it recognises, and it returns the event it managed to assemble alongside the error — on every error path, including the ambiguous-header one. A subscriber that drops a message needs something traceable to log: core NATS has no dead letter, so a refused message is gone unless the id and source of the thing that was rejected survive the refusal.
Callers must therefore check the error, and may read the returned Event for diagnostics whether or not it is nil.
The returned Data ALIASES body; see Marshal for the ownership note.
Example ¶
ExampleUnmarshal reads an event whose headers are in the canonical case http.Header would have produced. Keys are folded on read, so both spellings work.
package main
import (
"fmt"
"gitlab.com/phpboyscout/go/cloudevents"
)
func main() {
h := map[string][]string{
"Ce-Id": {"01JQ8Z3F7K2M4N6P8R0T2V4W6X"},
"Ce-Source": {"/phpbotscout/ingest"},
"Ce-Specversion": {"1.0"},
"Ce-Type": {"uk.phpboyscout.orders.created"},
"Ce-Scope": {"urn:phpboyscout:scope:continuity:01JQ8Z3F7K2M4N6P8R0T2V4W6X"},
"Content-Type": {"application/json"},
}
e, err := cloudevents.Unmarshal(h, []byte(`{"order":"4172"}`))
if err != nil {
fmt.Println("unmarshal:", err)
return
}
fmt.Println(e.Type)
fmt.Println(e.Extensions["scope"])
}
Output: uk.phpboyscout.orders.created urn:phpboyscout:scope:continuity:01JQ8Z3F7K2M4N6P8R0T2V4W6X
Example (Refused) ¶
ExampleUnmarshal_refused shows the half that is easy to drop: a refused event still names itself, so a subscriber logs something traceable rather than "a message failed to parse".
package main
import (
"fmt"
"gitlab.com/phpboyscout/go/errors"
"gitlab.com/phpboyscout/go/cloudevents"
)
func main() {
h := map[string][]string{
"ce-id": {"01JQ8Z3F7K2M4N6P8R0T2V4W6X"},
"ce-source": {"/phpbotscout/ingest"},
"ce-specversion": {"0.3"},
"ce-type": {"uk.phpboyscout.orders.created"},
}
e, err := cloudevents.Unmarshal(h, nil)
fmt.Println("refused:", errors.Is(err, cloudevents.ErrUnsupportedSpecVersion))
fmt.Println("but we still know which event:", e.ID)
}
Output: refused: true but we still know which event: 01JQ8Z3F7K2M4N6P8R0T2V4W6X
Source Files
Directories