Documentation
¶
Overview ¶
Package modelengine wires the in-kernel model (internal/model) into the kernel as a registered abi.EngineDriver under the id "inkernel".
Until now internal/model was a proven-correct forward-pass runtime with NO seam into the dispatch path: internal/agent and internal/engine never imported it, so `fak run`/`fak agent` could only dispatch a tool call to the mock or an HTTP upstream — never to the model fused into the kernel. This package closes that gap with the SAME mechanism every other backend uses: a driver that implements EngineDriver and registers itself from init(), so selecting it is one `--engine inkernel` flag (or one blank-import line), never a kernel edit.
What "completing a tool call on the in-kernel model" means here: the driver materializes the call's argument bytes, byte-tokenizes them into a prompt, and runs a REAL greedy Prefill+Step decode over a kernel-owned KV cache (model.Session.Generate) — the exact cache path the HF-oracle-verified model uses. The result payload carries the generated token ids + token accounting.
Weights: by default the driver runs a small DETERMINISTIC synthetic checkpoint (model.NewSynthetic) so the engine works on a CI box with no model export and a test is reproducible — the same honesty stance the KV-quarantine bridge takes (its wiring is proven on a synthetic model; the numerics are proven separately by the HF oracle in internal/model). Point FAK_MODEL_DIR at a real export to load genuine weights (model.Load); the dispatch path is identical either way.
The model is built LAZILY on the first Complete (guarded by sync.Once) so merely blank-importing this package — which every binary does via internal/registrations — costs nothing at startup; the synthetic checkpoint is only constructed if a call is actually routed to "inkernel".
Index ¶
Constants ¶
const EngineID = "inkernel"
EngineID is the registered id the kernel selects this backend by.
Variables ¶
var Default = New()
Default is the registered instance.
Functions ¶
func PreloadQ4K ¶
PreloadQ4K installs preloaded resident-Q4_K weights on the registered Default engine.
func SyntheticConfig ¶
SyntheticConfig is the small, valid, deterministic checkpoint shape the engine runs when no real export is configured. VocabSize 256 makes the byte->token map total (every input byte is a valid token id).
Types ¶
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine is the in-kernel-model EngineDriver. The model is constructed lazily.
func New ¶
func New() *Engine
New returns an Engine backed by the default synthetic config. The model itself is not built until the first Complete (or an explicit warmup in a test).
func (*Engine) Caps ¶
func (e *Engine) Caps() []abi.Capability
Caps advertises the in-kernel engine capability. A worker that doesn't know it simply never negotiates it; the engine is still selectable by id.
func (*Engine) Complete ¶
Complete runs the call's arguments through a real in-kernel-model decode and returns the generated tokens as the result. This is the EngineDriver seam: the kernel folds adjudication at Submit, then dispatches an ALLOWED call here at Reap.
func (*Engine) Preload ¶
Preload installs an already-constructed model as this engine's backing weights, claiming the once-guard so the lazy synthetic/FAK_MODEL_DIR path never runs. The host calls it at boot (fak serve --gguf) so the heavy weight load is part of the measured startup sequence rather than a lazy cost paid on the first request. The FIRST caller wins; a later Preload or lazy model() is a no-op.
func (*Engine) PreloadQ4K ¶
PreloadQ4K installs a resident-Q4_K-constructed model and flags the engine so Complete routes the dispatch decode through the Q4_K kernel (Session.Q4K=true), the path P1/P2 shipped for Qwen3.6-27B (NEON SDOT int8 decode GEMV). It mirrors the FAK_Q4K branch in cmd/fakchat and cmd/q4kdiag: the same loader, the same session flags. The once-guard means a plain Preload already claimed by an earlier caller makes this a no-op — the host picks ONE preload path at boot.
Source Files