jetkvm-cli

README

jetkvm: give an agent eyes and hands below the OS

jetkvm is a small, static CLI for controlling a JetKVM without driving its web UI. It turns the KVM into a practical tool for LLMs, automation agents, and shell scripts:

• Eyes: capture the host's real HDMI output as an image.
• Hands: type text, press boot keys, and use the absolute mouse.
• Telemetry: read video, USB, network, firmware, DC, and ATX state as JSON.
• Escape hatch: call any JetKVM JSON-RPC method, including virtual media.

That combination matters when SSH is down, the OS has not booted, a machine is stuck in firmware, or the only truth is what appears on the physical console.

$ go install github.com/ehrlich-b/jetkvm-cli/cmd/jetkvm@latest

$ jetkvm -H 10.80.3.206 status
$ jetkvm -H 10.80.3.206 snapshot -o screen.png
$ jetkvm -H 10.80.3.206 key F7 --n 40 --interval 100ms
$ jetkvm -H 10.80.3.206 type 'root'
$ jetkvm -H 10.80.3.206 key ctrl+alt+Delete

Everything except image decoding is built into the binary. snapshot uses ffmpeg from PATH to turn the H.264 stream into a PNG, JPEG, or other image.

The agent loop

The reliable pattern is observe → act → wait → verify, not a long blind sequence of keystrokes.

# 1. Establish what the device and input signal are doing.
jetkvm -H bench status

# 2. Observe the console.
jetkvm -H bench snapshot -o /tmp/bench-before.png --wake

# 3. Take one bounded action based on the image.
jetkvm -H bench key ArrowDown Enter

# 4. Let the screen settle, then verify the result visually.
jetkvm -H bench snapshot -o /tmp/bench-after.png --settle 2s

Feed each captured image to a vision-capable model. Ask it to describe the screen, locate the relevant control, choose one action, and state what visual change will count as success. If that change is absent in the next snapshot, re-observe instead of repeating the action indefinitely.

A useful agent instruction is:

Use JetKVM as a perception/action loop. Check status, capture a screenshot, take at most one state-changing step, and capture another screenshot to verify it. Never infer that a boot, login, menu selection, mount, or power action worked only because the command exited successfully. Avoid power actions unless explicitly authorized.

Why this CLI works well as an LLM tool

• Commands are small and composable; there is no interactive terminal state to parse.
• status and rpc produce JSON on stdout.
• Errors and rejected arguments exit nonzero.
• Every command is fully parsed and validated before connecting, so a typo does not evict the current KVM session.
• Unsupported text fails before any partial typing occurs.
• -v puts signaling and JSON-RPC traffic on stderr for diagnosis without contaminating JSON stdout.
• A copied device URL works directly: -H http://10.80.3.206/.

Command contract

Global flags go before the command. Command-specific flags can appear before or after positional arguments.

Command	Effect	Agent-friendly output
status	Read video, USB, network, version, DC, and ATX state	One JSON object
snapshot [-o path] [--settle d] [--wake]	Save the last decoded frame	Path and dimensions
key <combo>... [-n count]	Press and release one or more HID keys	Count and target
type <text>	Type US-layout text; - reads stdin	Character count
mouse <x> <y> [--click button] [-n count]	Move/click in absolute HID space	Coordinates and click count
power short|long|reset|state	Use the wired ATX extension	JSON for state; confirmation otherwise
rpc <method> [k=v]...	Call any registered JSON-RPC method	JSON result
keys	List accepted HID key names and codes	Stable text table

Durations use Go syntax such as 20ms, 1s, or 2.5s. RPC values are parsed as JSON when possible, so these have different types:

jetkvm -H bench rpc example enabled=true count=3 labels='["a","b"]' name=bench

true, 3, and the array are typed JSON values; bench is a string.

Seeing the host

jetkvm -H bench snapshot -o screen.png
jetkvm -H bench snapshot -o screen.jpg --settle 2s
jetkvm -H bench snapshot -o screen.png --wake

The device only starts video when WebRTC reaches connected, and its encoder may take time to start. The CLI allows up to 60 seconds for the track and actively requests an H.264 keyframe.

--settle defaults to one second. After the first keyframe, decoding continues for that duration and the last frame wins. This is useful for menus, spinners, and redraws. --wake sends a harmless modifier-only Shift tap before waiting for video, which can wake a blanked console.

If ffmpeg is absent, the command preserves the capture as a sibling .h264 file and exits nonzero because the requested image was not produced. Decode it later with:

ffmpeg -i screen.h264 -frames:v 1 screen.png

Turning image coordinates into mouse coordinates

JetKVM's absolute mouse uses 0..32767 on each axis, independent of resolution. For a pixel (px, py) in a screenshot of (width, height):

x = round(px / (width  - 1) * 32767)
y = round(py / (height - 1) * 32767)

Then:

jetkvm -H bench mouse 16384 16384
jetkvm -H bench mouse 16384 16384 --click left
jetkvm -H bench mouse 16384 16384 --click left -n 2

An agent should re-snapshot after a click. Firmware screens may change layout, resolution, focus, or input mode between steps.

Typing and keys

jetkvm -H bench key F2
jetkvm -H bench key ctrl+alt+Delete
jetkvm -H bench key ArrowDown ArrowDown Enter
jetkvm -H bench key F7 --n 40 --interval 100ms --hold 20ms
jetkvm -H bench type 'root'
printf '%s\n' 'a longer command' | jetkvm -H bench type -

key sends physical HID usages. The final token is the key; preceding + tokens are modifiers. A bare modifier such as shift is supported. Press and release reports are acknowledged in order, and release is still attempted when an operation is interrupted.

type maps printable US keyboard characters to HID keys. It validates the entire input before connecting and refuses unsupported Unicode instead of silently dropping characters—important when an LLM is entering credentials or commands. Use jetkvm keys for physical key names.

Status: decide before acting

status is the cheapest first probe:

{
  "getATXState": {"hdd": false, "power": false},
  "getDCPowerState": {"current": 0, "isOn": false, "power": 0, "voltage": 0},
  "getLocalVersion": {"appVersion": "0.5.8", "systemVersion": "0.2.8"},
  "getNetworkState": {"ipv4_address": "10.80.3.206", "online": true},
  "getUSBState": "configured",
  "getVideoState": {"fps": 60, "height": 1080, "ready": true, "streaming": 0, "width": 1920}
}

Fields vary by firmware. In particular, streaming: 0 while idle does not mean the HDMI signal is absent. Prefer ready, width, and height when deciding whether a snapshot should be possible.

Naming devices and authentication

Put one name address pair per line in $XDG_CONFIG_HOME/jetkvm/hosts (override with $JETKVM_HOSTS):

bench   192.0.2.10          # test bench
lab     http://192.0.2.11/  # copied browser URL is also accepted

Then use jetkvm -H bench status, or set $JETKVM_HOST as the default.

For a password-protected device, prefer an environment variable so the secret does not appear in an agent transcript or process listing:

JETKVM_PASSWORD='...' jetkvm -H bench status

--password is also available. The device used for current live testing runs authMode: noPassword; password mode is unit-tested against its HTTP contract but has not been exercised on physical hardware.

The RPC escape hatch

The CLI exposes the complete firmware registry:

jetkvm -H bench rpc getVideoState
jetkvm -H bench rpc getEDID
jetkvm -H bench rpc mountWithHTTP url='http://server/os.iso' mode=cdrom

Useful methods include:

Method	Use
mountWithHTTP {url, mode}	Mount an ISO over HTTP as virtual media
mountWithStorage {filename, mode}	Mount an image stored on the device
setEDID {edid} / getEDID	Correct a bad or missing display mode
getSerialSettings / sendCustomCommand	Use the serial extension
setUsbDeviceState {device, enabled}	Attach/detach keyboard, mouse, or storage
sendWOLMagicPacket {macAddress}	Wake a host without the ATX extension
reboot {force}	Reboot the JetKVM, not the attached host

RPC names and parameters are firmware internals, not a stable public API. Read state before mutating it, keep actions narrow, and verify afterward.

Power deserves a separate safety boundary

power short, power long, and power reset call setATXPowerAction. They require the ATX extension to be physically wired to the motherboard headers. The device can report the extension as active while the cable is disconnected; in that case commands are accepted but do nothing.

The HID key map separately includes Power, so jetkvm key Power can send a keyboard power-key event. JetKVM 0.5.8 does not show that key on its on-screen virtual keyboard, and it is not an ATX action: the attached host's firmware or OS decides whether it sleeps, shuts down, or ignores the event. Treat it as a state-changing power action and require the same explicit authorization.

For an LLM tool policy, keep status, snapshot, and read-only RPC methods available by default. Gate power, write RPC methods, virtual-media changes, and arbitrary typing behind explicit authorization appropriate to the task.

Session behavior agents need to understand

• A new JetKVM WebRTC session evicts the existing browser or CLI session. This is device behavior.
• Firmware can take tens of seconds to notice a disconnected session. An immediate second snapshot can connect as the “second” active session without restarting the encoder. If a snapshot has a connected data channel but no RTP, let the device go idle for roughly a minute and retry once.
• Another client connecting while an agent is working will drop the agent's session. Re-observe; do not replay a stale action plan.
• A blanked host console may emit no frames. Retry once with --wake.

These are exactly the cases where a bounded observe/act/verify loop beats a large macro.

Protocol notes

JetKVM has no documented control API. Input and video travel over WebRTC, with JSON-RPC 2.0 on a data channel named rpc.

The client uses GET /webrtc/signaling/client as a WebSocket. It sends an offer and trickles ICE candidates in both directions. Candidate order is guarded on both sides: local candidates cannot overtake the offer, and remote candidates received before the answer are queued. This is what lets routed LAN/VPN paths work where the legacy one-shot POST /webrtc/session often stalls.

The offer advertises only H.264. Firmware resolveCodec() chooses H.265 if the offer merely contains the string H265, regardless of normal codec preference, while this client only depacketizes H.264. The H.264-only offer is a protocol requirement, not an optimization.

Keyboard reports contain HID usage codes and a modifier bitmask. Their keys field must be a JSON number array—Go's []byte is intentionally avoided because encoding/json would turn it into a base64 string.

The implementation was checked against the open-source jetkvm/kvm release 0.5.8, including web.go, webrtc.go, jsonrpc.go, usb.go, and ui/src/keyboardMappings.ts.

Limits

• H.265 and VP8 capture are not implemented.
• Audio and the JetKVM cloud/relay path are not implemented; this is for direct LAN or routed/VPN access.
• Keyboard text generation currently targets the US layout.
• Password authentication has automated coverage but no physical-device test.

Development and verification

go test ./...
go test -race ./...
go vet ./...
go build ./cmd/jetkvm

The suite covers CLI preflight behavior, HID mappings and JSON wire format, RPC correlation and cleanup, endpoint/auth handling, H.264 parameter sets and packet loss, and ffmpeg encoding/fallback. Device-backed review has verified status, JSON-RPC, signaling, H.264 negotiation, and 1920×1080 snapshot capture on JetKVM app 0.5.8 / system 0.2.8.

A final practical rule

If the attached OS is healthy enough to do the job itself, use the OS. For example, efibootmgr -n is more deterministic than racing a boot menu. JetKVM is most valuable precisely where normal management has stopped being truthful or available.

License

MIT