transport

Documentation

Overview

Package transport defines the contract between the rendr engine and a concrete network transport (tcp, quic, udp_opaque, gvisor, ...).

The transport contract has two responsibilities the engine cannot fulfil itself:

1. Hide path liveness from Read/Write. A PathConn signals death via OnDeath, never by surfacing an error from Read or Write. The engine relies on this to keep the application-visible net.Conn alive across a path swap (CLAUDE.md hard rule #1).

2. Classify the cause of any death as either CleanClose (an orderly remote BYE / io.EOF on a known-quiesced stream) or TransportError (any timeout, reset, handshake failure, idle kill, ...). The engine migrates on TransportError and tears down on CleanClose. Misclassifying TransportError as CleanClose is the hy2scale Phase 1 regression we will not repeat (CLAUDE.md hard rule #2).

Index

• type DatagramAccelerationMode
• type DatagramAccelerationObserver
• type DatagramAccelerationStatus
• type DeathCause
  • func Classify(err error, quiesced bool, byeSeen bool) DeathCause
• type FrameBatchWriter
• type FrameDispatchAttemptState
• type FrameDispatchAuthorization
• type FrameDispatchCompletion
• type FrameDispatchKind
• type FrameDispatchSpan
• type FrameDispatchTracer
• type FrameDispatchWriter
• type IngressQueueObserver
• type IngressQueueStats
• type OwnedFrameReader
• type PathConn
• type PathFactory
• type PathInfo
• type PathListener
• type PathQuality
• type PathQualityReader
• type PathSessionKind
• type PathSpec
  • func (s PathSpec) Clone() PathSpec

Types

type DatagramAccelerationMode

type DatagramAccelerationMode string

DatagramAccelerationMode identifies the socket send treatment selected by rendr. Selection does not prove that the protocol stack produced a batch; callers must use the counters to distinguish eligibility from actual use. It is observation only and cannot request or authorize an offload.

const (
	DatagramAccelerationUnknown  DatagramAccelerationMode = "unknown"
	DatagramAccelerationGSO      DatagramAccelerationMode = "gso_active"
	DatagramAccelerationOrdinary DatagramAccelerationMode = "ordinary_fallback"
)

type DatagramAccelerationObserver

type DatagramAccelerationObserver interface {
	DatagramAccelerationStatus() DatagramAccelerationStatus
}

DatagramAccelerationObserver is an optional PathConn extension reporting its underlying UDP socket. Accepted QUIC paths share their listener socket, so those paths expose socket-wide aggregate counters rather than per-path counters. It must be safe to call concurrently with transport I/O and Close.

type DatagramAccelerationStatus

type DatagramAccelerationStatus struct {
	Mode            DatagramAccelerationMode
	Cause           string
	ProbeGeneration uint64
	ProbedAt        time.Time
	// BatchCalls and BatchDatagrams prove actual multi-message UDP writes.
	// They are independent of GSO: one sendmmsg call may carry ordinary UDP
	// messages, GSO super-packets, or both.
	BatchCalls          uint64
	BatchDatagrams      uint64
	GSOAttempts         uint64
	GSOSuperPackets     uint64
	GSOSegments         uint64
	OrdinaryDatagrams   uint64
	FallbackTransitions uint64
}

DatagramAccelerationStatus is a syscall-free snapshot of one UDP socket's selected treatment and actual send evidence. GSOSuperPackets counts only successful kernel writes carrying UDP_SEGMENT; a selected mode alone is not proof that the fast path was exercised.

type DeathCause

type DeathCause uint8

DeathCause classifies why a PathConn went down. See package doc for the strict semantic contract.

const (
	// CauseUnknown should be used only at the boundary, before any
	// classification has happened. It is never the final cause.
	CauseUnknown DeathCause = 0

	// CauseCleanClose means the remote side issued an orderly BYE
	// or the stream reached an application-visible EOF. The engine
	// does NOT migrate; it propagates EOF to the application.
	CauseCleanClose DeathCause = 1

	// CauseTransportError covers everything else: timeouts, RST,
	// quic.IdleTimeoutError, HandshakeTimeoutError, ApplicationError,
	// TransportError, "the underlying socket suddenly returned 0
	// bytes for no reason". The engine MUST migrate, not propagate.
	CauseTransportError DeathCause = 2
)

func Classify

func Classify(err error, quiesced bool, byeSeen bool) DeathCause

Classify maps a Go error into a DeathCause. It is the canonical helper transport adapters should call before invoking OnDeath.

The bias here is intentional and matches CLAUDE.md hard rule #2: when in doubt, classify as TransportError. CleanClose only on unambiguous signals.

Currently recognised CleanClose signals:

• io.EOF on a stream that has been quiesced (caller signals quiesced=true to opt in).
• A protocol-level BYE was already received before the close (caller signals byeSeen=true).

Everything else, including io.ErrUnexpectedEOF, becomes TransportError. ErrUnexpectedEOF means data was cut off mid-message, which is precisely what should trigger a migration.

type FrameBatchWriter

type FrameBatchWriter interface {
	WriteFrameBatch(frames [][]byte) (completed int, err error)
}

FrameBatchWriter is an optional PathConn fast path for transports that can submit multiple already-framed packets through one physical write operation. Frames must be accepted in slice order. completed is the exact number of whole frames accepted from the prefix of frames.

A return with completed < len(frames) must include a non-nil error. If the transport partially writes the next frame, that frame is not completed and the error applies to it and every suffix frame. A transport must never report a negative completed count or one greater than len(frames).

PathConn.Write remains mandatory. The engine uses this extension only for bounded, immediately available packet DATA runs; stream, control, replay, and adapters without this interface retain ordinary Write behavior. Close must promptly unblock WriteFrameBatch under the same rule as PathConn.Write.

type FrameDispatchAttemptState

type FrameDispatchAttemptState uint8

FrameDispatchAttemptState records what the adapter can prove about one authorized physical occurrence. In particular, a batch writer's incomplete suffix is Unknown: the completed-prefix contract cannot prove whether the transport touched those frames.

const (
	FrameDispatchAttemptUnknown FrameDispatchAttemptState = iota
	FrameDispatchAttempted
	FrameDispatchNotAttempted
)

type FrameDispatchAuthorization

type FrameDispatchAuthorization struct {
	Sequence                  uint64
	PublishedNext             uint64
	AckNext                   uint64
	AdmissionLedgerGeneration uint64
	LedgerGeneration          uint64
	AdmissionID               uint64
	AttemptID                 uint64
	PhysicalOccurrence        uint64
	EndpointGeneration        uint64
	FrameOffset               int
	Kind                      FrameDispatchKind
	AuthorizedAt              time.Time
	FrameBytes                int
	FrameDigest               proto.FrameDigest
}

FrameDispatchAuthorization is a diagnostic snapshot taken at the engine's final logical submission boundary. AttemptID identifies the logical submission. PhysicalOccurrence, EndpointGeneration, and FrameOffset bind a tracer span to one exact endpoint incarnation and contiguous frame suffix. A zero PhysicalOccurrence is the logical authorization passed to a FrameDispatchWriter; that writer must assign monotonically increasing, one-based occurrences before forwarding diagnostics to an endpoint tracer. For an ordinary single-route dispatch, that boundary is immediately before the physical writer. A race authorizes every child in one coherent fanout cohort before any child can produce an ACK; each child still receives its own AttemptID and completion.

The snapshot proves that this exact immutable DATA frame still belonged to the replay ledger when the physical attempt was committed. AdmissionLedgerGeneration records the earlier recursive-dispatch admission; LedgerGeneration records the final writer-boundary snapshot.

This contract is observational. Implementations must not use it to alter dispatch, ACK, replay, or migration decisions.

type FrameDispatchCompletion

type FrameDispatchCompletion struct {
	StartedAt         time.Time
	CompletedAt       time.Time
	FrameBytes        int
	BytesWritten      int
	BytesWrittenKnown bool
	WriteCalls        int
	AttemptState      FrameDispatchAttemptState
	// WriteAttempted is true only when AttemptState is
	// FrameDispatchAttempted. False is not proof that no write occurred; callers
	// must inspect AttemptState to distinguish Unknown from NotAttempted.
	WriteAttempted     bool
	WholeFrameAccepted bool
	BatchIndex         int
	BatchSize          int
	Err                error
}

FrameDispatchCompletion describes the result returned by the physical PathConn write associated with one authorization. BytesWrittenKnown is false when a batch writer reports only a completed prefix and cannot identify the partial suffix. WholeFrameAccepted records the adapter's factual result and is independent of Err: io.Writer permits a full byte count with an error.

type FrameDispatchKind

type FrameDispatchKind uint8

FrameDispatchKind identifies the ledger role of one DATA write attempt. InitialCohort may be observed more than once for race or bond fan-out; it does not assert that a particular physical write was globally first.

const (
	FrameDispatchKindUnknown FrameDispatchKind = iota
	FrameDispatchKindInitialCohort
	FrameDispatchKindReplay
)

type FrameDispatchSpan

type FrameDispatchSpan interface {
	FinishFrameDispatch(FrameDispatchCompletion)
}

FrameDispatchSpan binds one physical occurrence to its aggregate contiguous write result. One occurrence may contain multiple short io.Writer calls; WriteCalls and BytesWritten expose that fact without storing every syscall. FinishFrameDispatch is called exactly once and must return promptly without calling back into the engine.

type FrameDispatchTracer

type FrameDispatchTracer interface {
	BeginFrameDispatch(FrameDispatchAuthorization) FrameDispatchSpan
}

FrameDispatchTracer is an optional diagnostic PathConn extension. The engine calls BeginFrameDispatch after its final replay-ledger check. It is immediately before Write or WriteFrameBatch for single-route dispatches; a race child may have queued after the coherent fanout check. A nil span disables completion observation for that occurrence.

Adapters that wrap caller-provided net.Conn values may forward this hook to an underlying tracer. Implementations must return promptly and must not call back into the engine.

type FrameDispatchWriter

type FrameDispatchWriter interface {
	WriteFrameDispatch([]byte, FrameDispatchAuthorization) (int, error)
}

FrameDispatchWriter is an optional atomic DATA-write extension for adapters whose physical endpoint can change behind one stable PathConn. The engine calls this method instead of separately invoking FrameDispatchTracer and PathConn.Write, so authorization can be bound to the exact endpoint incarnation and write critical section that performs the I/O.

An implementation that forwards diagnostics to an underlying FrameDispatchTracer owns the complete Begin/Finish lifecycle. It must emit one span per physical endpoint incarnation if in-place maintenance interrupts and resumes a logical write, assigning a distinct PhysicalOccurrence and the exact EndpointGeneration and FrameOffset. The input follows io.Writer's ownership rule: it must not be retained or modified. The engine additionally supplies a defensive copy so a violating external adapter cannot corrupt replay-owned bytes. Built-in adapters may use a sealed module-internal owned-frame extension. The method must not call back into the engine.

type IngressQueueObserver

type IngressQueueObserver interface {
	IngressQueueStats() IngressQueueStats
}

IngressQueueObserver is an optional PathConn observability extension used to distinguish transport ingress saturation from engine or application loss. It must be safe to call concurrently with Read and Close.

type IngressQueueStats

type IngressQueueStats struct {
	Depth     uint64
	HighWater uint64
	Capacity  uint64
}

IngressQueueStats is a transport-owned receive queue snapshot. A zero Capacity means the transport has no observable ingress queue.

type OwnedFrameReader

type OwnedFrameReader interface {
	ReadOwnedFrame() ([]byte, error)
}

OwnedFrameReader is an optional PathConn fast path for transports whose receive API already returns a uniquely owned frame allocation. The returned slice must remain immutable and valid after the next call; ownership passes to the engine. Implementations must not recycle or reuse its backing array.

PathConn.Read remains mandatory for callers that don't understand this extension. The engine prefers ReadOwnedFrame when available to avoid an otherwise redundant full-frame allocation and copy on high-rate paths.

type PathConn

type PathConn interface {
	io.ReadWriteCloser

	// Quality returns the latest measurement for direct caller diagnostics.
	// May return a zero PathQuality if the transport has not yet probed. The
	// engine uses PathQualityReader instead so observation is cancellable.
	Quality() PathQuality

	// OnDeath registers a callback the transport invokes exactly
	// once when the path is no longer usable. The cause MUST be
	// CleanClose or TransportError - CauseUnknown is forbidden as a
	// final value. fn may be called from any goroutine.
	OnDeath(fn func(cause DeathCause, err error))

	// LocalAddr / RemoteAddr forward the underlying transport's
	// addresses for diagnostics. They are advisory; the engine does
	// not key off them.
	LocalAddr() string
	RemoteAddr() string
}

PathConn is one live path. It MUST NOT surface migration-class errors via Read/Write; use OnDeath instead. Close MUST promptly unblock every concurrent Read and Write. Engine admission deadlines and bounded shutdown rely on this contract; adapters that cannot provide it are not conforming.

type PathFactory

type PathFactory interface {
	// DialPath establishes a single PathConn for spec. The returned
	// PathConn is already past any TLS/handshake stage; if the
	// handshake itself fails, DialPath returns the error and no
	// PathConn.
	DialPath(ctx context.Context, spec PathSpec) (PathConn, error)

	// Probe returns the best estimate of path quality without
	// promoting the path to the active set. Implementations may
	// short-circuit by dialing and immediately closing if the
	// transport has no cheap probe primitive.
	Probe(ctx context.Context, spec PathSpec) (PathQuality, error)
}

PathFactory opens one kind of already-framed network path. A factory is registered explicitly on a rendr Runtime under an opaque caller-chosen ID; the ID is not part of this interface and cannot select leaf mobility.

type PathInfo

type PathInfo struct {
	ID      uint32
	Spec    PathSpec
	Quality PathQuality
	Since   time.Time
	// LocalAddr and RemoteAddr are the current physical endpoints reported by
	// the PathConn. They may change while the logical path ID remains stable
	// during in-place leaf mobility (for example QUIC CID rebind).
	LocalAddr  string
	RemoteAddr string
	Reads      uint64
	Writes     uint64
	// DataWrites and ControlWrites split engine-framed egress so probes and
	// cumulative ACKs cannot masquerade as application throughput.
	DataWrites     uint64
	ControlWrites  uint64
	DataDispatches uint64
	// FirstDataDispatches counts successful DATA writes issued by the
	// frame's initial publication. DataDispatches also includes recovery
	// replay, so comparing the two exposes retransmission overhead without
	// letting replay masquerade as a scheduler route decision.
	FirstDataDispatches uint64
	// BatchWriteCalls, BatchWriteFrames, and BatchWriteMax report successful
	// engine-to-transport FrameBatchWriter submissions. They are zero for
	// ordinary writes and never include control or replay traffic.
	BatchWriteCalls  uint64
	BatchWriteFrames uint64
	BatchWriteMax    uint64
	Active           bool
	// RecvDups: inbound frames on this path whose SEQ was already
	// delivered or buffered (race-mode duplicates, accidental
	// retransmits). Sum across all paths equals ConnStats.RecvDups.
	RecvDups uint64
	// LastRecvAt is the wall-clock time at which a frame was last
	// successfully received on this path (any type: data, ctrl,
	// probe). Zero if no frame has arrived. Distinct from
	// Quality.At which only updates on probe replies; LastRecvAt
	// surfaces "path is genuinely idle" as opposed to "probe-fresh
	// but no traffic" so monitoring can flag NAT keepalive timeouts.
	LastRecvAt time.Time
	// LastSendAt is the wall-clock time at which a frame was last
	// successfully written to this path's socket. Pair with
	// LastRecvAt to distinguish "I'm sending but peer is silent"
	// from "peer is sending but I'm idle".
	LastSendAt time.Time
	// IngressQueue reports a transport-owned receive queue when the path
	// implements IngressQueueObserver. It is zero for transports without a
	// distinct observable ingress queue.
	IngressQueue IngressQueueStats
	// DatagramAcceleration reports the selected UDP treatment and cumulative
	// socket evidence when the path implements DatagramAccelerationObserver.
	// Accepted paths may share listener-wide counters. The zero value means
	// that the transport doesn't expose datagram acceleration telemetry.
	DatagramAcceleration DatagramAccelerationStatus
}

PathInfo is the engine's read-only snapshot of one attached path.

Reads / Writes are cumulative frame counters; transports that don't instrument framing leave them at 0. Active is true iff this path is the current send target under selector mode (always one active path); under race/bond, Active flags the first path returned by dispatch's iteration order and should not be used for routing decisions.

type PathListener

type PathListener interface {
	AcceptPath(context.Context) (PathConn, error)
	SessionKind() PathSessionKind
	Close() error
	Addr() net.Addr
}

PathListener accepts paths that already implement rendr transport framing, quality reporting, and death notification. AcceptPath must honor ctx and return a non-nil PathConn on success. Close must promptly unblock any concurrent AcceptPath call without invalidating paths already returned. A Runtime invokes AcceptPath serially per listener, may retry after a context deadline, and owns each successfully returned path. SessionKind must return a valid, immutable factual contract.

type PathQuality

type PathQuality struct {
	RTT    time.Duration
	Jitter time.Duration
	// LossPP is loss in parts-per-thousand (0-1000) so a uint16 wire
	// representation has enough resolution.
	LossPP uint16
	// At is the local clock at which the measurement was last updated.
	At time.Time
}

PathQuality is the most recent measurement of one path. Encoded on the wire as fixed-width integers (PATH_QUALITY control frame, see the proto package); these Go fields are ergonomic only.

type PathQualityReader

type PathQualityReader interface {
	QualityContext(ctx context.Context) (PathQuality, error)
}

PathQualityReader is the optional cancellable quality-observation extension used by the engine. Implementations must return promptly after ctx is done. The engine does not call PathConn.Quality because third-party legacy methods may block without a cancellation boundary.

type PathSessionKind

type PathSessionKind uint8

PathSessionKind states which rendr application contract a framed listener can safely carry. It describes framing semantics, not the network carrier.

const (
	// PathSessionAny accepts both stream and packet rendr sessions.
	PathSessionAny PathSessionKind = iota
	// PathSessionStream accepts only net.Conn-style rendr sessions.
	PathSessionStream
	// PathSessionPacket accepts only net.PacketConn-style rendr sessions.
	PathSessionPacket
)

type PathSpec

type PathSpec struct {
	// Transport names the adapter ("tcp", "quic", ...).
	Transport string
	// Address is the remote address for the transport (interpretation
	// is transport-specific).
	Address string
	// Local optionally pins the local source endpoint.
	Local string
	// Opts carries transport-specific options (TLS config, ALPN, ...).
	// The engine treats it as opaque.
	Opts map[string]string
	// Weight is an advisory hint to the mode layer (bond/race) about
	// share of frames. Selector ignores Weight.
	Weight uint16
}

PathSpec is the declarative description of a candidate path. A transport adapter consumes it to dial a live PathConn.

func (PathSpec) Clone

func (s PathSpec) Clone() PathSpec

Clone returns an owned snapshot of the path specification. Opts is opaque to the engine, but it is still mutable caller-owned state and must not cross an asynchronous engine boundary by reference.