streaming

README

Streaming Chat Example

Demonstrates real-time token streaming using Model.ChatStream. Each token is printed to the terminal as it arrives, rather than waiting for the full response.

Build

go build -o streaming ./examples/streaming/

Usage

# With a local GGUF model file
./streaming --model ./models/gemma-3-1b-Q4_K_M.gguf

# With a HuggingFace model ID (downloads on first use)
./streaming --model google/gemma-3-1b

Type a message and press Enter. Tokens stream to the terminal in real time. Type quit to exit.

Documentation

Overview

Command streaming demonstrates streaming chat generation using the zerfoo API.

Unlike the chat example which waits for the full response, this example prints each token to the terminal as it arrives using zerfoo.Model.ChatStream.

Usage:

go build -o streaming ./examples/streaming/
./streaming --model path/to/model.gguf