Documentation
¶
Overview ¶
Package ipc is the wire between a pdf-spec CLI and whatever holds the model.
It exists so the parser and the model are separate processes. Loading a vision model costs seconds and gigabytes and is worth paying once for a thousand pages, while a PDF parse costs milliseconds and should not be entangled with it. Two processes over a local socket gets that, and it gets three more things: the CLI stays a small static binary with no GPU dependency, the model host can be replaced without recompiling anything, and a model that crashes on a malformed page takes down a subprocess rather than the run.
Byte-compatible with inferd ¶
The framing, the JSON shapes, and the socket paths are inferd's generation protocol v2 (its ADR 0015/0016/0021), reimplemented here rather than imported. That is deliberate and the reasons are specific: this package needs the *server* side, which inferd's Go client does not provide; pdf-spec stays dependency-free at its own boundary, as objects and render do; and a shared module would couple this repo's release cadence to a daemon's. The cost is that a protocol change upstream is a change here too, which is what Version and the conformance test in ipc_test.go exist to catch.
What compatibility buys is concrete: point the CLI at a running inferd and it works, with no adapter in between, because the daemon cannot tell the difference. Until inferd carries granite-docling, ocr/docd serves the same wire itself.
The framing, and why it is not NDJSON ¶
Each frame is a uvarint payload length, one type byte, then exactly that many bytes:
[uvarint len][1 byte type][payload]
Type 0x01 is JSON, 0x02 is a raw binary blob. There is no delimiter, so a payload may contain any byte — which is the point. A page of raw RGB at 200 DPI is about 12 MB of arbitrary bytes; NDJSON would need base64, costing a third more bytes and a full copy in each direction on the hot path. inferd's embeddings and admin surfaces are NDJSON because their payloads are small and text; generation is framed binary because its payloads are neither.
The length is read before anything is allocated, and a length past MaxFrame is an error rather than a large allocation — F-1 in inferd's threat model, and the same reasoning as the tag-depth limit in ADR 0001. A peer that sends one is not resynced with: the stream position is no longer known, so the only safe move is to close.
Images arrive decoded ¶
An image attachment carries interleaved RGB, width*height*3 bytes, no alpha, in a blob frame. No PNG, no JPEG. inferd's ADR 0016 puts the decode on the consumer because the daemon links no image codec, and the same posture is right here for a different reason: the pages come straight from the rasterizer as pixels, so encoding them to PNG for transport and decoding on the far side would be work performed purely to undo itself.
Index ¶
- Constants
- func DecodeImage(a Attachment) (*image.RGBA, error)
- func DefaultAddr() string
- func Serve(ctx context.Context, conn *Conn, h Handler, log *slog.Logger) error
- type Attachment
- type Block
- type Closer
- type Conn
- type Engine
- type Handler
- type Listener
- type Local
- type Message
- type OutBlock
- type Request
- type Response
- type Usage
Constants ¶
const ( FrameJSON byte = 0x01 FrameBlob byte = 0x02 )
Frame type tags.
const ( RespFrame = "frame" RespDone = "done" RespError = "error" StopEndTurn = "end_turn" StopMaxTokens = "max_tokens" StopError = "error" )
Response types and stop reasons.
const ( ErrInvalidRequest = "invalid_request" ErrFrameTooLarge = "frame_too_large" ErrAttachmentUnsupported = "attachment_unsupported" ErrWireVersion = "wire_version_unsupported" ErrInternal = "internal" )
Error codes, matching inferd's ErrorCodeV2. Enumerated rather than free-text because a caller has to distinguish "retry later" from "this request is wrong" and parsing a message string for that is how brittle retry loops get written.
const MaxFrame = 64 << 20
MaxFrame bounds one frame's payload at 64 MiB, matching inferd.
Sized for the payload that actually reaches it: a raster page. A 300 DPI US Letter page is 2550x3300, which as RGB is 25 MB, so the cap is roughly 2.5x the largest realistic attachment and far below anything that threatens a process. It is checked against the declared length before a buffer is allocated, which is the property that makes it a bound rather than a check.
const MaxImageAxis = 1 << 16
MaxImageAxis bounds an attachment's width and height, on both sides of the wire.
65,536 is four times the longest edge of a US Letter page at 1200 DPI, so no page anything here rasterizes comes near it. The bound exists to be checked *before* width*height*3 is multiplied: without it a hostile pair of dimensions can overflow the product into a small number that then agrees with a short payload, and the receiver reads the page diagonally instead of failing.
const Version uint32 = 1
Version is the wire version carried in every request and checked by the server.
In-band rather than negotiated at connect. A version mismatch is then a named error on the first request instead of a hang or a misparse, and the check costs one integer comparison. 1 is inferd's current generation wire version; this constant moving means this package is no longer compatible with a daemon that speaks 1, so it moves only alongside upstream.
Variables ¶
This section is empty.
Functions ¶
func DecodeImage ¶
func DecodeImage(a Attachment) (*image.RGBA, error)
DecodeImage turns an image attachment back into an RGBA raster, opaque.
The server side of ImageAttachment. Validates that the byte count matches the declared dimensions, because everything downstream indexes by row: a blob one byte short of width*height*3 either panics or reads the page diagonally, and neither says what went wrong.
func DefaultAddr ¶
func DefaultAddr() string
DefaultAddr returns the platform default socket path, resolved exactly as inferd resolves it, so a CLI with no configuration finds a running daemon.
The Unix chain is XDG_RUNTIME_DIR, then ~/.inferd/run, then /tmp — a per-user runtime directory first because a socket in a world-writable /tmp is a pre-creation target for another user, and XDG_RUNTIME_DIR is mode 0700 and owned by the session. Windows has one answer, a named pipe, since it has no filesystem socket to place.
func Serve ¶
Serve handles requests on one connection until the peer closes it or the context is cancelled.
One request at a time, serially. That matches the protocol — one in-flight request per connection — and it matches the resource: a model host has one model, and interleaving two pages through it would only add queueing without adding throughput. Concurrency is several connections.
A protocol violation closes the connection after an error frame. A framing error cannot be resynced from, because the stream position is no longer known and the next bytes read would be the middle of someone's payload.
Types ¶
type Attachment ¶
type Attachment struct {
Kind string `json:"kind"`
ID string `json:"id"`
Width uint32 `json:"width,omitempty"`
Height uint32 `json:"height,omitempty"`
Bytes []byte `json:"-"`
}
Attachment is one binary payload's metadata. The bytes themselves are `json:"-"` and ride in a blob frame keyed by ID, so this struct is what a reader sees when it decodes the request JSON and the blob has not arrived yet.
func ImageAttachment ¶
func ImageAttachment(id string, img *image.RGBA) (Attachment, error)
ImageAttachment builds an image attachment from an RGBA raster.
Drops the alpha channel and the row padding: the wire format is exactly width*height*3 interleaved RGB octets. Both drops are correct rather than lossy — a rasterized page is opaque, and an *image.RGBA's Stride may exceed 4*width, so copying Pix flat would skew every row after the first. This is the same padding trap ADR 0005 documents on the pdfium side, at the other end of the same pixels.
Returns an error on the dimensions DecodeImage would reject, so the check is the same at both ends. A sender that could not be refused would put a frame on the wire that the receiver must refuse, which is a worse place to find out.
type Block ¶
type Block struct {
Type string `json:"type"`
Text string `json:"text,omitempty"`
AttachmentID string `json:"attachment_id,omitempty"`
}
Block is one piece of a message's content: text, or a reference to an attachment.
func ImageBlock ¶
ImageBlock references an image attachment by id.
The image is a reference rather than inline content because the bytes travel in a separate blob frame. That indirection is what keeps the request JSON small enough to log and to read in a test.
type Conn ¶
type Conn struct {
// contains filtered or unexported fields
}
Conn is a framed connection. Both directions of the protocol use it, so the codec exists once and a client and a server cannot disagree about it.
Writes are mutex-guarded because a frame is three separate writes — length, type, payload — and two goroutines interleaving those produces a stream that is not merely wrong but unrecoverable. Reads are not guarded: one connection carries one in-flight request, so there is one reader by construction.
func NewConn ¶
func NewConn(rw io.ReadWriteCloser) *Conn
NewConn wraps a transport. The buffer sizes are ordinary; the writer is flushed at the end of every frame, so a peer never waits on bytes sitting in a buffer.
func (*Conn) ReadFrame ¶
ReadFrame reads one frame.
io.EOF before the first length byte means the peer closed cleanly between frames, which is normal and is how a server learns a client is done. An EOF after that is io.ErrUnexpectedEOF from ReadFull — a truncated frame, which is not normal. The two are distinguishable by design, because "the client went away" and "the client was killed mid-page" call for different handling.
func (*Conn) ReadRequest ¶
ReadRequest reads a request and fills in its attachments' bytes from the blob frames that follow.
Rejects a wire-version mismatch here rather than deeper in, so the error names both versions while the numbers are still in scope. Also rejects a blob whose declared length disagrees with its frame: the two are written from the same slice, so a disagreement means the stream is not what it claims and the bytes must not be used as pixels.
func (*Conn) WriteFrame ¶
WriteFrame writes one length-prefixed, type-tagged frame and flushes.
func (*Conn) WriteRequest ¶
WriteRequest writes a request and the blob frames its attachments need.
Order is fixed and load-bearing: the request JSON first, then per attachment a descriptor frame and its blob, in the order the attachments appear in the request. A reader can therefore allocate once it has the JSON and knows how many blobs are coming, without buffering the whole request.
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine is an ocr.Engine that sends pages over this wire.
It holds one connection and serves one request at a time, matching both the protocol — one in-flight request per connection, no multiplexing — and ocr.Engine's contract. A caller rendering pages in parallel opens one Engine per worker, the way the render verb opens one Rasterizer per worker.
func Dial ¶
Dial connects to a model host at addr, or at DefaultAddr when addr is empty.
A successful connect is the readiness signal, not a heartbeat: the server binds its socket only once its model is loaded, so connect-refused means "not ready yet" and there is nothing further to ask. That is inferd's posture (its threat model F-13) and this package matches it, which is why there is no Ping.
func (*Engine) Recognize ¶
Recognize sends one page and collects the DocTags the model streams back.
Returns whatever was generated alongside any error, rather than discarding it. That is not politeness: the dominant failure on a dense page is a generation that runs into its token bound mid-table, and ocr/doctags parses a truncated document by design — so a partial page is worth strictly more than an empty one, and the caller decides.
type Handler ¶
type Handler interface {
Generate(ctx context.Context, img *image.RGBA, prompt string, maxTokens int, emit func(string)) error
// Name identifies the backend on the done frame. Diagnostic only, and worth
// having: "which model produced this" is the first question about a bad page.
Name() string
}
Handler generates a response for one request.
It receives the page already decoded to an RGBA raster and the prompt already extracted from the message blocks, because every handler would otherwise do that unpacking itself and a handler that got it subtly wrong would fail as a model quality problem rather than as a protocol bug.
emit is called with each chunk of generated text as it becomes available. Streaming is not optional: a page takes tens of seconds, and a client that receives nothing until the end cannot distinguish slow from hung. A handler returning an error after emitting text still delivers that text — the client keeps partial output by design.
type Listener ¶
type Listener struct {
// contains filtered or unexported fields
}
Listener accepts framed connections.
func Listen ¶
Listen binds addr, or DefaultAddr when addr is empty.
Bind late. A caller should load its model first and call this only once it can serve, because a bound socket is this protocol's readiness signal — there is no health frame, so a socket that exists before the model does turns every early client's wait into a failure.
func (*Listener) Accept ¶
Accept serves connections until ctx is cancelled or Close is called.
Each connection gets a goroutine and is served serially within it: one model, one page at a time, but several clients may queue without any of them being refused. A per-connection failure is logged and the connection dropped, never propagated — one client sending a malformed frame must not take down a host that took ten minutes to warm up.
type Local ¶
type Local struct {
// contains filtered or unexported fields
}
Local adapts a Handler to ocr.Engine without a socket.
The reason this exists is that the common case is one CLI invocation over one document, and there the process boundary buys nothing: the model is loaded and discarded by the same run either way, so a socket would add serialization, base64-free framing, and a platform-specific listener to move pixels between two halves of the same process. Local skips all of it.
The IPC path earns its keep in the other case — a warm host serving many invocations, or a host on a machine with the GPU — and that case is exactly what a running inferd is. So this is not a shortcut around the protocol; it is the same Handler reached without one, which is why the protocol lives behind an interface rather than being the only way in.
type Request ¶
type Request struct {
WireVersion uint32 `json:"wire_version"`
ID string `json:"id,omitempty"`
Messages []Message `json:"messages"`
Attachments []Attachment `json:"attachments,omitempty"`
MaxTokens *uint32 `json:"max_tokens,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
}
Request is one generation request. Field names and JSON tags match inferd's RequestV2 byte for byte; a field this package does not use is still spelled the same, because a rename would make the two incompatible for no gain.
type Response ¶
type Response struct {
ID string `json:"id"`
Type string `json:"type"`
Block *OutBlock `json:"block,omitempty"`
Usage *Usage `json:"usage,omitempty"`
StopReason string `json:"stop_reason,omitempty"`
Backend string `json:"backend,omitempty"`
Code string `json:"code,omitempty"`
Message string `json:"message,omitempty"`
}
Response is one frame off the response stream: a text delta, a terminal done, or a terminal error.
func (Response) IsTerminal ¶
IsTerminal reports whether this frame ends a request's stream.
Source Files