Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
jitllm
command
Command jitllm is the jitllm CLI.
|
Command jitllm is the jitllm CLI. |
|
Package convert turns a source model into a jlm container.
|
Package convert turns a source model into a jlm container. |
|
gguf
Package gguf parses GGUF model files into a meta.File.
|
Package gguf parses GGUF model files into a meta.File. |
|
hf
Package hf acquires GGUF weights from HuggingFace.
|
Package hf acquires GGUF weights from HuggingFace. |
|
library
Package library is the list of models jitllm can fetch by name: for each, a Hugging Face repository, the file in it, and the vision tower beside it when there is one.
|
Package library is the list of models jitllm can fetch by name: for each, a Hugging Face repository, the file in it, and the vision tower beside it when there is one. |
|
safetensors
Package safetensors is a parser for the safetensors container, and nothing else.
|
Package safetensors is a parser for the safetensors container, and nothing else. |
|
dev
|
|
|
agenteval
command
Command agenteval measures whether an agent can use and change jitllm: it sets up each task in a fresh workspace, hands the task's prompt to the agent command it is given, then checks the result by machine and writes a JSON report.
|
Command agenteval measures whether an agent can use and change jitllm: it sets up each task in a fresh workspace, hands the task's prompt to the agent command it is given, then checks the result by machine and writes a JSON report. |
|
bench
Package bench is the measurement harness.
|
Package bench is the measurement harness. |
|
decisionbench
command
Command decisionbench reproduces the user's decision-model table against any server speaking TypeSafe's POST /v1/systemone -- jitllm's, llama-server's, `lev serve` -- so every engine in the table is measured by one harness (docs/design/decision-models.md, section 5).
|
Command decisionbench reproduces the user's decision-model table against any server speaking TypeSafe's POST /v1/systemone -- jitllm's, llama-server's, `lev serve` -- so every engine in the table is measured by one harness (docs/design/decision-models.md, section 5). |
|
httpbench
command
httpbench is the one HTTP load harness every engine is measured with: a closed-loop concurrency sweep of streamed OpenAI-compatible requests, two or more endpoints run as interleaved arms.
|
httpbench is the one HTTP load harness every engine is measured with: a closed-loop concurrency sweep of streamed OpenAI-compatible requests, two or more endpoints run as interleaved arms. |
|
httpbench/hb
Package hb is the HTTP load harness behind dev/httpbench: one closed-loop concurrency sweep, measured the same way against every OpenAI-compatible engine, with two or more engines run as interleaved arms.
|
Package hb is the HTTP load harness behind dev/httpbench: one closed-loop concurrency sweep, measured the same way against every OpenAI-compatible engine, with two or more engines run as interleaved arms. |
|
tools/fmtbench
command
Command fmtbench prices each quant format's CPU decode matvec against a read wall measured in the same process, every shape sized DRAM-resident.
|
Command fmtbench prices each quant format's CPU decode matvec against a read wall measured in the same process, every shape sized DRAM-resident. |
|
tools/mvbench
command
Command mvbench measures the GPU decode matvec against a device read-bandwidth ceiling measured in the same process.
|
Command mvbench measures the GPU decode matvec against a device read-bandwidth ceiling measured in the same process. |
|
tools/ptxdump
command
Command ptxdump lowers named kernels to PTX so ptxas and cuobjdump can be pointed at them: the SASS is the only thing that says what a kernel actually issues (the MMA matvec turned out to be mostly address arithmetic).
|
Command ptxdump lowers named kernels to PTX so ptxas and cuobjdump can be pointed at them: the SASS is the only thing that says what a kernel actually issues (the MMA matvec turned out to be mostly address arithmetic). |
|
tools/readbench
command
readbench prices the jlm reader: parse, then page-in by four strategies.
|
readbench prices the jlm reader: parse, then page-in by four strategies. |
|
tools/stridebench
command
stridebench prices the access pattern the packed kernel actually makes.
|
stridebench prices the access pattern the packed kernel actually makes. |
|
engine
|
|
|
grammar
Package grammar constrains generation to a context-free grammar: GBNF text (llama.cpp's grammar format) or a JSON Schema compiled to it, matched one code point at a time by a pushdown automaton and turned, at each step, into the set of vocabulary tokens that keep the output inside the grammar.
|
Package grammar constrains generation to a context-free grammar: GBNF text (llama.cpp's grammar format) or a JSON Schema compiled to it, matched one code point at a time by a pushdown automaton and turned, at each step, into the set of vocabulary tokens that keep the output inside the grammar. |
|
model
Package model loads a jlm container and runs its transformer graph on generated kernels, on the host and on whichever devices take its blocks.
|
Package model loads a jlm container and runs its transformer graph on generated kernels, on the host and on whichever devices take its blocks. |
|
sched
Package sched runs work across the machine's performance cores.
|
Package sched runs work across the machine's performance cores. |
|
format
|
|
|
jlm
Package jlm is jitllm's own model container: weights already in the device layout, one page per block, the tokenizer travelling with them.
|
Package jlm is jitllm's own model container: weights already in the device layout, one page per block, the tokenizer travelling with them. |
|
quant
Package quant is the quantization formats themselves: the type enum, each format's block geometry, and the reference decoders.
|
Package quant is the quantization formats themselves: the type enum, each format's block geometry, and the reference decoders. |
|
internal
|
|
|
capabilities
Package capabilities is the machine-readable answer to "does this build support my workload": the architectures, Hugging Face classes and projectors the converter takes, the input formats, the quantization types, the backends (implemented, and verified on which hardware, with the gate that is the evidence), the HTTP API's endpoints, request fields, limits and refused combinations, and the build's version and revision.
|
Package capabilities is the machine-readable answer to "does this build support my workload": the architectures, Hugging Face classes and projectors the converter takes, the input formats, the quantization types, the backends (implemented, and verified on which hardware, with the gate that is the evidence), the HTTP API's endpoints, request fields, limits and refused combinations, and the build's version and revision. |
|
cmd/goheap
Package goheap is the process-wide collector setup both binaries share: cmd/jitllm and server/cmd/jitllmd.
|
Package goheap is the process-wide collector setup both binaries share: cmd/jitllm and server/cmd/jitllmd. |
|
localserver
Package localserver is the one place every front end (the CLI, the window, the terminal app and jitllmd itself) learns where the local service is and where its models live: the default address, the per-system models folder, and the check "does a jitllmd answer here, and which version".
|
Package localserver is the one place every front end (the CLI, the window, the terminal app and jitllmd itself) learns where the local service is and where its models live: the default address, the per-system models folder, and the check "does a jitllmd answer here, and which version". |
|
localserver/service
Package service installs jitllmd as the person's own background service, the way each system starts a per-user program at login: a systemd user unit on Linux, a launchd user agent on macOS, and on Windows the login script and Run key the installer (packaging/windows) writes.
|
Package service installs jitllmd as the person's own background service, the way each system starts a per-user program at login: a systemd user unit on Linux, a launchd user agent on macOS, and on Windows the login script and Run key the installer (packaging/windows) writes. |
|
modelsdoc
Package modelsdoc renders docs/models.md, the supported-models list, from the converter's own tables: convert.Architectures (the GGUF names), convert.HFClasses (the Hugging Face classes), convert.Projectors (the vision projectors), quant's type lists and the download catalogue.
|
Package modelsdoc renders docs/models.md, the supported-models list, from the converter's own tables: convert.Architectures (the GGUF names), convert.HFClasses (the Hugging Face classes), convert.Projectors (the vision projectors), quant's type lists and the download catalogue. |
|
oracle
Package oracle is the engine's arithmetic written in plain Go: the reference every generated kernel is gated against, and nothing else.
|
Package oracle is the engine's arithmetic written in plain Go: the reference every generated kernel is gated against, and nothing else. |
|
schemacheck
Package schemacheck is the structured-output gates' JSON Schema validator, for the subset engine/grammar compiles: what an output is held to, written independently of the grammar that produced it.
|
Package schemacheck is the structured-output gates' JSON Schema validator, for the subset engine/grammar compiles: what an output is held to, written independently of the grammar that produced it. |
|
testlock
Package testlock serialises GPU tests across processes.
|
Package testlock serialises GPU tests across processes. |
|
testmodels
Package testmodels is where a test finds a model file.
|
Package testmodels is where a test finds a model file. |
|
jit
|
|
|
cpu
Package cpu emits x86-64 and arm64 machine code at load time.
|
Package cpu emits x86-64 and arm64 machine code at load time. |
|
gpu/amdgpu
Package amdgpu lowers jit/gpu/ir kernels to LLVM IR text for AMD GPUs.
|
Package amdgpu lowers jit/gpu/ir kernels to LLVM IR text for AMD GPUs. |
|
gpu/backend
Package backend puts one interface over the three GPU runtimes so a benchmark or a tuner can ask the same question of each.
|
Package backend puts one interface over the three GPU runtimes so a benchmark or a tuner can ask the same question of each. |
|
gpu/ffi
Package ffi binds C functions without cgo, on top of goffi.
|
Package ffi binds C functions without cgo, on top of goffi. |
|
gpu/hip
Package hip binds AMD's ROCm runtime (libamdhip64 on Linux, amdhip64_N.dll from the HIP SDK on Windows) and code object manager (libamd_comgr) through jit/gpu/ffi, with no cgo.
|
Package hip binds AMD's ROCm runtime (libamdhip64 on Linux, amdhip64_N.dll from the HIP SDK on Windows) and code object manager (libamd_comgr) through jit/gpu/ffi, with no cgo. |
|
gpu/ir
Package ir is a very small typed kernel IR for NN compute.
|
Package ir is a very small typed kernel IR for NN compute. |
|
gpu/kernels
Package kernels holds the NN kernels written against the IR.
|
Package kernels holds the NN kernels written against the IR. |
|
gpu/metal
Package metal is Apple-only.
|
Package metal is Apple-only. |
|
gpu/msl
Package msl lowers the IR to Metal Shading Language.
|
Package msl lowers the IR to Metal Shading Language. |
|
gpu/ptx
Package ptx is an assembler for PTX text: registers, labels and a buffer.
|
Package ptx is an assembler for PTX text: registers, labels and a buffer. |
|
gpu/spirv
Package spirv emits SPIR-V compute modules from the IR.
|
Package spirv emits SPIR-V compute modules from the IR. |
|
gpu/tier
Package tier is jitllm's GPU decode tier: an nn.Device backed by the generated device kernels (jit/gpu).
|
Package tier is jitllm's GPU decode tier: an nn.Device backed by the generated device kernels (jit/gpu). |
|
gpu/vulkan
Package vulkan runs SPIR-V compute kernels through the Vulkan loader.
|
Package vulkan runs SPIR-V compute kernels through the Vulkan loader. |
|
scripts
|
|
|
checks
command
Command checks says which checks a change needs, and runs them.
|
Command checks says which checks a change needs, and runs them. |
|
genmodels
command
Command genmodels writes docs/models.md, the supported-models list, from the converter's own tables (internal/modelsdoc):
|
Command genmodels writes docs/models.md, the supported-models list, from the converter's own tables (internal/modelsdoc): |
|
gentok
command
Command gentok generates tok/pretok/table.go: for every tokenizer.ggml.pre name a GGUF can carry, the ordered pipeline of typed stages that name denotes.
|
Command gentok generates tok/pretok/table.go: for every tokenizer.ggml.pre name a GGUF can carry, the ordered pipeline of typed stages that name denotes. |
|
genunicode
command
Command genunicode generates tok/unicodedata.go from llama.cpp's checked-in Unicode tables.
|
Command genunicode generates tok/unicodedata.go from llama.cpp's checked-in Unicode tables. |
|
modelfree
command
Command modelfree runs every test of the four modules on whatever host CI gives it, with no model download and no GPU, and writes what was SKIPPED into the job summary.
|
Command modelfree runs every test of the four modules on whatever host CI gives it, with no model download and no GPU, and writes what was SKIPPED into the job summary. |
|
server
module
|
|
|
Package tok implements the SentencePiece tokenizer that llama-architecture GGUF models carry (tokenizer.ggml.model == "llama").
|
Package tok implements the SentencePiece tokenizer that llama-architecture GGUF models carry (tokenizer.ggml.model == "llama"). |
|
jinja
Package jinja implements a Jinja2-compatible template engine.
|
Package jinja implements a Jinja2-compatible template engine. |
|
pretok
Package pretok is the pre-tokenizer model: the stages a tokenizer runs before merging, the parser that reads them out of a tokenizer.json, and the table generated from the artefacts themselves.
|
Package pretok is the pre-tokenizer model: the stages a tokenizer runs before merging, the parser that reads them out of a tokenizer.json, and the table generated from the artefacts themselves. |
Click to show internal directories.
Click to hide internal directories.







