Directories
¶
| Path | Synopsis |
|---|---|
|
Package benchmark provides a standardized benchmark suite for measuring ML model inference performance: tok/s decode, tok/s prefill, memory usage, and time to first token.
|
Package benchmark provides a standardized benchmark suite for measuring ML model inference performance: tok/s decode, tok/s prefill, memory usage, and time to first token. |
|
Package compare provides a model comparison tool that runs the same prompts through multiple models and compares their performance metrics.
|
Package compare provides a model comparison tool that runs the same prompts through multiple models and compares their performance metrics. |
Click to show internal directories.
Click to hide internal directories.