screenctx

Documentation

Overview

Command screenctx captures screen context by combining CGWindowListCopyWindowInfo with Vision OCR to produce an XML map of on-screen text attributed to the owning application.

Usage:

screenctx                    # capture entire screen (needs screen recording permission)
screenctx <screenshot>       # read from file
screenctx -                  # read from stdin

Output is XML:

<screen width="W" height="H">
  <app name="Terminal" pid="1234" bounds="x,y,w,h">
    <text confidence="0.98" bounds="x,y,w,h">Hello</text>
  </app>
</screen>