Documentation
¶
Overview ¶
Command bench_gemm micro-benchmarks low-precision (fp8 / Q4_K) weight GEMMs against an f32 baseline in ztensor's engine, at the n>1 ("batched", not n==1 decode) shapes representative of the LTX-2 DiT denoise regime.
This is the measurement tool for E127/T127.1.0b PART 2: it sizes whether the denoise-regime dequant-to-f32 path is acceptable or whether new kernels are needed before the converter storage mapping and perf budget are committed. The fp8 sub-format is F8_E4M3 (confirmed by Phase-0 header read; see docs/devlog.md), which is what tensor.NewFP8E4M3Storage produces.
IMPORTANT: the real numbers MUST be produced on the GB10 via Spark (see docs/bench/manifests/ltx2-fp8-spike.yaml and the Hardware doctrine in CLAUDE.md), NOT via interactive SSH. On a host without CUDA the GPU engine is unavailable; -cpu runs the harness against the CPU engine for a local smoke (f32, plus any quant path the CPU engine supports via dequant).
Source Files