README
¶
English | 繁體中文
mmfg-http
Serializes a Go standard-library net/http.Request into a binary format transportable over an MMFG connection, letting a Hub (resource manager) hand a request to a Node (processor) over this connection, and a Node hand it on to another Node in turn, all zero-copy.
The node that finishes processing answers the original HTTP client directly: it takes over that client connection's file descriptor and responds on it, instead of routing the response back through the Hub. See Response Delivery below.
The MMFG-HTTP-PROTO protocol defined by this library never modifies the Base Section once written; every subsequent mutation is recorded append-only, so nothing existing is ever moved or rewritten.
Platform: build tag
unix— Linux / macOS only.
Wire layout
An MMFG-HTTP-PROTO buffer is made of three sections:
┌─────────────────────────────────────────┐
│ Prefix 4B BaseSectionLen │ ← lets readers skip the Base Section
├─────────────────────────────────────────┤
│ Base Section Type/Method/Proto/URL │ ← written once by Inject, never modified
│ + Header/Cookie entries │
├─────────────────────────────────────────┤
│ CDC Log zero or more mutations │ ← every node-side edit is appended here
└─────────────────────────────────────────┘
All multi-byte integers are little-endian.
Prefix
[BaseSectionLen] 4B (uint32 LE)
The byte length of the Base Section below. Either side can read these 4 bytes and compute 4 + BaseSectionLen to jump straight to the start of the CDC Log without parsing the Base Section at all.
Apply() uses this to skip re-parsing and only replay the CDC Log.
Base Section
[Type] 1B
[Method] 1B
[ProtoMajor] 1B
[ProtoMinor] 1B
[URLLen] 2B (uint16 LE)
[URL] *B
// zero or more header entries (State=2)
// zero or more cookie entries (State=3)
[State=0] 1B ← section terminator
Header and cookie entries share the same KV format:
[TotalLen] 2B (uint16 LE) — len(Key) + len(Value)
[KeyLen] 2B (uint16 LE)
[Key] *B
[Value] *B — length = TotalLen - KeyLen
CDC Log
Mutations to headers, cookies, the URL, or the method are always appended to the end of the CDC Log using the same KV format; the Base Section is never touched:
[OpCode] 1B
[TotalLen] 2B
[KeyLen] 2B (0 for SetURL/SetMethod, which carry no key)
[Key] *B
[Value] *B
EOF or [OpCode=0] marks the end of the CDC Log. SetHeader/SetCookie with an empty Value means delete. AddHeader appends a value to whatever SetHeader already established for that key, letting a single UpdateHeader call produce a multi-value header.
Response marker
A node produces a response by overwriting the start of the Base Section in place (offset 4):
[Type=2] 1B (TypeResponse)
This single-byte overwrite doesn't touch anything from offset 5 onward; the CDC Log is untouched.
Field reference
Type
| Value | Constant | Meaning |
|---|---|---|
| 1 | TypeRequest |
HTTP request |
| 2 | TypeResponse |
HTTP response; marked in place by SelfRespond |
Method
| Value | Method |
|---|---|
| 1 | GET |
| 2 | POST |
| 3 | PUT |
| 4 | DELETE |
| 5 | HEAD |
| 6 | OPTIONS |
| 7 | PATCH |
| 8 | TRACE |
| 9 | CONNECT |
State / OpCode
| Value | Constant | Purpose |
|---|---|---|
| 0 | StateDeprecated |
section terminator |
| 2 | StateHeader |
Base Section: one header entry |
| 3 | StateCookie |
Base Section: one cookie entry |
| 4 | StateSetHeader |
CDC: set a header; empty Value means delete |
| 5 | StateAddHeader |
CDC: append a value to a header set by a preceding StateSetHeader |
| 6 | StateSetCookie |
CDC: set a cookie; empty Value means delete |
| 8 | StateSetURL |
CDC: replace the URL (Key is always empty) |
| 9 | StateSetMethod |
CDC: replace the method (Key is always empty) |
This wire format only serializes method/proto/URL/header/cookie: it never carries the HTTP body. The
*http.Requestrebuilt byApplyor lazy parsing always has anilBody. A self-responding node'sreq.Bodyis a separate path: it reads directly off the handed-off client connection (see Response Delivery below).
API
import "github.com/nautrouds/mmfg-http/go/mmfghttp"
On the node side, mmfghttp.New(conn) wraps a node.Connection directly into a *NodeRequest; on the hub side, go through Hub instead of calling New yourself.
It also maintains mmfg-http's own connection to each node, which self-response delivery depends on (see Response Delivery below).
Hub side
h, err := mmfghttp.NewHub()
// Dial establishes mmfg's own connection to the node. controlSocketPath is
// optional: pass "" for a node that never self-responds, or a second,
// independent socket path for a node that may (see "Response Delivery").
h.Dial("nodeName", socketPath, controlSocketPath)
// Allocates an MMFG connection, injects req into it, and remembers req so
// later calls to Apply/AcceptSelfResponse don't need it passed again.
r, err := h.Request(ctx, req)
// Hand off to the node (blocks until the node finishes).
selfResponded, err := r.Next("nodeName")
// Skip the Base Section, replay the CDC entries the node appended, apply onto req.
r.Apply()
Apply never re-parses the whole Base Section; it only reads the Prefix to find where to skip to, then replays the CDC entries in order.
Next reports whether the node it just handed off to called SelfRespond; see Response Delivery below for what to do with that.
Node side
r := mmfghttp.New(nodeConn)
The response itself is produced with SelfRespond, not through CDC/Apply.
See the Response Delivery section below.
Accessors & mutations (shared)
HubRequest and NodeRequest expose the same read/write API:
method, _ := r.Method()
u, _ := r.URL()
val, _ := r.Header("Authorization") // first value only
vals, _ := r.HeaderValues("Accept-Encoding") // every value, in order
cookies, _ := r.Cookies()
// Mutations are always CDC appends; the Base Section is never touched
r.UpdateHeader("X-Forwarded-For", "1.2.3.4")
r.UpdateHeader("Accept-Encoding", "gzip", "br") // variadic: replaces every existing value
r.UpdateHeader("X-Internal") // no values given == DeleteHeader
r.DeleteHeader("X-Internal")
r.SetURL("http://backend/new-path")
r.SetMethod("PUT")
r.SetCookie("session", "refreshed")
r.DeleteCookie("old-cookie")
Hub and Node read and write the same CDC log. On the node side, the first accessor call triggers a lazy parse (Base Section plus any existing CDC entries); mutation-only usage never triggers that parse, and once it has been triggered, mutations keep the parsed cache in sync so read-then-write logic sees the latest value.
Connection interface
type connection interface {
io.Reader
io.Writer
io.WriterAt // used by SelfRespond for the in-place overwrite
io.ReaderAt // used by IsSelfResponse to peek without disturbing the sequential read cursor
DataLen() uint32
}
hub.Connection and node.Connection both satisfy this interface; it's the minimal abstraction the package needs over either side's connection.
Response Delivery
A node decides to answer the client itself by calling SelfRespond, which marks the buffer and returns.
From mmfg's point of view, that handler call is now finished.
If the hub accepts, HubRequest.AcceptSelfResponse(w) hijacks the original client's http.ResponseWriter, replays the node's CDC entries onto the request the hub already has, and sends the fully-resolved request together with the client's connection to the node over a dedicated control connection.
Every fd copy the hub itself is holding closes automatically once the send completes. Along with the fd, it forwards whatever bytes net/http had already buffered off the wire before the hijack, so the node's view of the connection picks up exactly where the client left off.
The node receives a self-contained event (full request + connection) and can hand it to any goroutine to answer like an ordinary backend request; there's no code to look up and no cache to consult.
If the hub doesn't accept (it calls Next again for another node, or calls Apply), nothing is sent to the node at all. There's no rejection message.
Control connection
Hub.Dial(nodeName, socketPath, controlSocketPath) takes a second, independent socket path distinct from mmfg's own. Passing "" means that node has no self-response capability.
Next returns an error if the node calls SelfRespond on a connection dialed that way. A node granted a controlSocketPath gets a plain, dedicated listener for it; mmfg's own listener is untouched and needs no demuxing.
Hub side
h, _ := mmfghttp.NewHub()
h.Dial("nodeName", socketPath, controlSocketPath)
r, _ := h.Request(ctx, req)
selfResponded, _ := r.Next("nodeName")
if selfResponded {
r.AcceptSelfResponse(w) // w is the original client's http.ResponseWriter
} else {
r.Apply() // no self-response, normal CDC path
}
AcceptSelfResponse requires w to support http.Hijacker; it fails for response writers that don't (and for hijacked connections that don't support File(), such as *tls.Conn).
Node side
mmfg-http doesn't wrap the mmfg Node itself; it wraps the mmfg conn.
mmfghttp.New(conn) wraps an mmfg request as mmfg-http, and the mmfg listener stays fully native:
n := node.NewNode(node.WithHandler(handler))
go n.Listen(socketPath) // vanilla mmfg, nothing mmfg-http-specific
For the control connection, there are two layers to choose from.
Low-level
ReadControlMessage returns a ready-to-use net.Conn.
Reading from it transparently starts with any bytes the hub's net/http had already buffered before handing the connection off, then continues with whatever the client sends afterward. Everything else (accept loop, HTTP wire format) is up to the caller:
for {
conn, _ := controlListener.Accept()
go func(conn *net.UnixConn) {
for {
req, c, err := mmfghttp.ReadControlMessage(conn)
if err != nil {
return
}
go handleAsBackend(req, c) // any goroutine, just like a normal request
}
}(conn.(*net.UnixConn))
}
High-level (Handler/Serve)
Handler/HandlerFunc/ResponseWriter mirror net/http.Handler, and Serve/ListenAndServe handle the accept loop and HTTP/1.1 framing:
mmfghttp.ListenAndServe(controlSocketPath, mmfghttp.HandlerFunc(
func(w mmfghttp.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body) // bounded by Content-Length, if the client sent one
w.Write([]byte("handled: " + string(body)))
},
))
Streaming & chunked fallback: ResponseWriter.Write streams straight to the connection.
The first Write (or an explicit WriteHeader) sends the status line and headers immediately. Set Content-Length on w.Header() before that if it's known; otherwise the response falls back to Transfer-Encoding: chunked (a malformed Content-Length value is dropped and falls back the same way). Connection: close is always sent, since a handed-off connection answers exactly one request. r.Body is a real reader.
Bounded by Content-Length when the client sent one, decoded on the fly when the client sent Transfer-Encoding: chunked instead (http.NoBody if neither is present).
Content-Length validation: a declared Content-Length is enforced against what's actually written: Write rejects a call that would push the total past it (returning 0 and an error, nothing partial gets written), and if the handler writes less than it declared, the internally-called finish returns an error that Serve/ListenAndServe log.
Neither direction is silently truncated or padded.
Server config: Serve(l *net.UnixListener, handler Handler) accepts a pre-bound listener; ListenAndServe(socketPath string, handler Handler) binds one first.
The same split as node.Node.Serve/node.Node.Listen. Both are convenience wrappers around a zero-value Server:
s := &mmfghttp.Server{
IdleTimeout: 10 * time.Second, // zero means a 30s default
MaxBodyBytes: 8 << 20, // zero means a 32 MiB default
Handler: mmfghttp.HandlerFunc(handler),
}
s.ListenAndServe(controlSocketPath)
IdleTimeout is refreshed on every Read/Write, not a cap on the whole exchange.
An actively-streaming request or response is never cut short, but a connection that stalls mid-exchange eventually is. MaxBodyBytes caps how much of a request body a handler may read, enforced identically whether the size came from a declared Content-Length or was only discovered as chunks arrived; exceeding it surfaces as a read error rather than a silent truncation. Trailer headers on a chunked request aren't captured.
Build requirements
- Go 1.25+
- Linux / macOS (build tag
unix) - Dependency:
github.com/nautrouds/mmfg/v2 v2.0.0
Directories