desktop

package
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 13, 2026 License: MIT Imports: 45 Imported by: 0

Documentation

Rendered for windows/amd64

Overview

Package desktop is the Windows automation engine underneath the MCP tools. It reimplements the capabilities of the Python Windows-MCP project (UI Automation tree walking, synthetic input, screenshots, window management, PowerShell execution, WMI-backed system data) on top of the deploymenttheory go-bindings-win32 and go-bindings-wmi SDKs.

COM threading model

UI Automation and much of the Win32 input/window surface are thread-affine: COM objects created in one apartment must be used from that apartment, and foreground/focus operations are tied to a specific thread. The engine therefore owns a single dedicated OS thread (locked with runtime.LockOSThread) initialized as an STA, and funnels every COM/UIA/input call through it via Do. This mirrors the single-threaded traversal the Python implementation relies on to avoid COM apartment deadlocks.

Credential Manager integration.

Credentials supplied to the server at init are installed into the Windows Credential Manager for the *calling user's* credential set, so anything running in that user context (browsers, RDP, mapped drives, Windows SSO, or a launched line-of-business app) can consume them the normal way.

The plaintext secret is confined to this file. It is read back from the Credential Manager only inside InjectCredential, converted straight to keystrokes, and the intermediate buffers are zeroed. No function here returns a secret to a caller, so a secret can never reach a tool result, the audit log, or the model's context.

wincred (CredRead/CredWrite/CredDelete) is a plain advapi32 API — not COM, and not thread-affine — so the store operations do not need the engine's STA thread. InjectCredential does run on it, because it synthesizes input.

Visual feedback overlays for screen capture and video recording.

The overlay manager draws click-through, top-most, no-activate layered windows so that a viewer (or a recording) can see what the automation is doing: a translucent green border around a window when it is activated or targeted, and a brief orange ring at each click point. Overlays never intercept input (WS_EX_TRANSPARENT) and never take focus (WS_EX_NOACTIVATE), so they cannot perturb the automation they visualize.

Rendering is push-based via UpdateLayeredWindow with a premultiplied-BGRA DIB, so no WM_PAINT loop is required; a light message pump keeps the windows well-behaved and processes teardown. All window/GDI objects live on one dedicated OS thread (windows are thread-affine).

Index

Constants

View Source 
const (
	// MaxTypeChars bounds one Type call.
	MaxTypeChars = 10000
	// MaxBatchItems bounds the point and edit counts of the batch tools.
	MaxBatchItems = 100
)

Limits on a single synthetic-input call.

The engine serializes every COM, UIA and input operation onto one thread, so a call that runs for minutes stops the whole desktop surface -- snapshots, the overlay, every other tool -- for that long. TypeText sends one SendInput pair per rune and ClickMany sleeps between points, so neither had an upper bound on how long it could hold the thread; a single Type of a few megabytes was an effective denial of service on the session, and it audited as one tool call.

The limits are set well above real usage rather than tightly: a UI test that pastes a long description should not be refused. They exist to bound a runaway or hostile call, not to shape ordinary ones.

View Source 
const (
	DefaultRecordStopKey = 0x78 // VK_F9
	DefaultRecordMarkKey = 0x77 // VK_F8
)

DefaultRecordStopKey is the virtual-key code that ends a recording (F9), and DefaultRecordMarkKey the one that marks an assertion (F8). Both are consumed, not recorded, so pressing either never appears in the journey.

View Source 
const (
	MatchExact    = "exact"
	MatchContains = "contains"
	MatchMatches  = "matches"

	OccurrenceUnique = "unique"
	OccurrenceFirst  = "first"
)

Selector matching and occurrence modes, matching the journey vocabulary.

Variables

View Source 
var (
	// ErrCredentialNotFound reports a target absent from the Credential Manager.
	ErrCredentialNotFound = errors.New("credential not found")
	// ErrCredentialEmpty reports a stored credential with a zero-length secret.
	ErrCredentialEmpty = errors.New("credential has an empty secret")
	// ErrSecretTooLarge reports a secret over the Credential Manager blob limit.
	ErrSecretTooLarge = errors.New("secret exceeds the Credential Manager blob limit")
)
View Source 
var (
	ErrNoMatch            = errors.New("no element matched")
	ErrAmbiguousSelector  = errors.New("the selector matches more than one element")
	ErrOccurrenceOutOfRan = errors.New("the occurrence index is past the last match")
	ErrBadOccurrence      = errors.New(`occurrence must be "unique", "first", or a 0-based index`)
	ErrBadNameMatch       = errors.New(`name_match must be "exact", "contains" or "matches"`)
	// ErrLabelNotFound reports a label that is not in the current snapshot, which
	// normally means the tree has been rebuilt since it was issued.
	ErrLabelNotFound = errors.New("label not found in the last Snapshot; take a fresh Snapshot first")
)

ErrNoMatch reports a selector that matched nothing; ErrAmbiguousSelector reports one that matched more than one element under unique occurrence.

View Source 
var ErrAVISizeLimit = errors.New("avi: 4 GiB format limit reached")

ErrAVISizeLimit reports that appending another frame would exceed what AVI 1.0 can address. Callers should stop recording and finalize; the file written so far stays valid.

View Source 
var ErrClosed = errors.New("desktop: engine is closed")

ErrClosed is returned when work is submitted to a closed engine.

View Source 
var ErrCoordinateOutOfRange = errors.New("coordinate is outside the display coordinate space")

ErrCoordinateOutOfRange reports a coordinate that cannot be represented in the display coordinate space.

View Source 
var ErrFFmpegFinalize = errors.New("ffmpeg did not finalize the recording within the timeout")

ErrFFmpegFinalize reports that ffmpeg had to be killed rather than allowed to finish writing the container. The recording on disk is truncated but the session shut down; callers surface it as a warning, not a failure.

View Source 
var ErrInjectTargetNotMasked = errors.New("refusing to inject the credential")

ErrInjectTargetNotMasked is returned when a credential injection cannot be confirmed to be landing in a control that masks its input. Callers match on it to distinguish a refused destination from an engine failure; the wrapped detail explains which of the checks could not be satisfied, and every message names the allow_unmasked_target remedy through injectRemedy.

View Source 
var ErrInputTooLarge = errors.New("input exceeds the per-call limit")

ErrInputTooLarge reports a synthetic-input call above its limit. It is a refusal the model can act on by splitting the work, not an engine failure.

Functions

func InjectRemedy added in v1.2.0 

func InjectRemedy() string

InjectRemedy names the documented escape hatch, for the tool layer to append to a refusal. It is not baked into each error so the sentinel's message stays short and the guidance appears exactly once.

Types

type CredentialInfo 

type CredentialInfo struct {
	Name     string `json:"name"`
	Target   string `json:"target"`
	Username string `json:"username,omitempty"`
	Type     string `json:"type"`
	Persist  string `json:"persist"`
	Present  bool   `json:"present"`
	// Injectable is false for credential classes Windows will not read back.
	Injectable bool `json:"injectable"`
	// AllowUnmaskedTarget lets this credential be injected into a control that
	// does not report itself as masked. It defaults to false, so injection
	// normally requires a confirmed password field — see requireMaskedFocus. It is
	// an operator decision, declared per credential in the credentials document,
	// because it trades the never-read guarantee for reach into destinations that
	// cannot report IsPassword (a console window, some Electron and Java apps).
	AllowUnmaskedTarget bool `json:"allow_unmasked_target"`
}

CredentialInfo is the non-secret view of an installed credential — everything the Credentials tool is allowed to report.

type CredentialPersist 

type CredentialPersist string

CredentialPersist controls how long Windows retains the credential. 

const (
	// PersistSession drops the credential at logoff. The default.
	PersistSession CredentialPersist = "session"
	// PersistLocalMachine keeps it on this machine across logons.
	PersistLocalMachine CredentialPersist = "local_machine"
	// PersistEnterprise additionally roams with the profile.
	PersistEnterprise CredentialPersist = "enterprise"
)

type CredentialSpec 

type CredentialSpec struct {
	// Name is the stable handle the agent uses to refer to this credential. It is
	// never the secret and is safe to log.
	Name string
	// Target is the Credential Manager target name (e.g. a host or URL).
	Target string
	// Username is stored alongside the secret; safe to log.
	Username string
	// Comment is an optional human-readable note stored on the credential.
	Comment string
	// Secret is the UTF-8 plaintext. Zeroed by the caller after WriteCredential.
	Secret  []byte
	Type    CredentialType
	Persist CredentialPersist
}

CredentialSpec is one credential to install.

Secret is held as a byte slice rather than a string so the caller can zero it after installation; Go strings are immutable and cannot be wiped.

type CredentialType 

type CredentialType string

CredentialType is the Credential Manager credential class. 

const (
	// CredentialGeneric is CRED_TYPE_GENERIC: an application-defined credential,
	// the right class for app, web, and API secrets.
	CredentialGeneric CredentialType = "generic"
	// CredentialDomainPassword is CRED_TYPE_DOMAIN_PASSWORD, consumed by Windows
	// itself for network authentication (SSO, mapped drives, RDP). Windows will not
	// hand the blob back to a caller for this class — see InjectCredential.
	CredentialDomainPassword CredentialType = "domain_password"
)

func (CredentialType) Readable 

func (t CredentialType) Readable() bool

Readable reports whether Windows will return this class's secret to a caller. Only CRED_TYPE_GENERIC blobs are readable by the owning user; domain-password blobs are write-only by design, so they can be installed for Windows to use but never injected as keystrokes.

type Desktop 

type Desktop struct {
	// contains filtered or unexported fields
}

Desktop is the Windows automation engine. It is safe for concurrent use by multiple goroutines: all Win32/COM work is serialized onto one STA thread. Construct it with New and release it with Close.

func New 

func New(logger *slog.Logger, opts Options) (*Desktop, error)

New starts the engine: it spins up the dedicated STA thread, makes the process per-monitor-DPI aware, initializes COM, and creates the UI Automation client. It returns once startup has completed (or failed).

func (*Desktop) ActivateWindow 

func (d *Desktop) ActivateWindow(titleSubstr string) (WindowInfo, error)

ActivateWindow brings the first window matching titleSubstr to the foreground and (if overlays are on) highlights it. Returns the activated window.

func (*Desktop) Click 

func (d *Desktop) Click(x, y int, button string, clicks int) error

Click moves the cursor to (x,y) and clicks. button is "left", "right", or "middle"; clicks of 0 hovers (move only), 1 single-clicks, 2 double-clicks.

func (*Desktop) ClickMany 

func (d *Desktop) ClickMany(points [][2]int, holdCtrl bool) error

ClickMany left-clicks a series of points in order. When holdCtrl is true, Ctrl is held down for the whole sequence (multi-select).

func (*Desktop) ClipboardGet 

func (d *Desktop) ClipboardGet() (string, error)

ClipboardGet returns the clipboard's Unicode text, or "" if the clipboard holds no text. It runs on the engine STA thread.

func (*Desktop) ClipboardSet 

func (d *Desktop) ClipboardSet(text string) error

ClipboardSet replaces the clipboard contents with the given Unicode text. It runs on the engine STA thread.

func (*Desktop) Close 

func (d *Desktop) Close() error

Close shuts down the engine thread. After Close, Do returns ErrClosed. It is idempotent (finalizes the recording before tearing down the overlay, so the security banner is captured on the final frames).

func (*Desktop) CollapseLabel 

func (d *Desktop) CollapseLabel(label int) error

func (*Desktop) ControlService 

func (d *Desktop) ControlService(name, action string) (string, error)

ControlService starts, stops, or restarts a service by exact name (case-insensitive). Returns a human-readable result.

func (*Desktop) CoordinatesForLabel 

func (d *Desktop) CoordinatesForLabel(label int) (x, y int, ok bool)

CoordinatesForLabel resolves a label from the most recent Snapshot to a click point. It is safe to call from any goroutine.

func (*Desktop) CredentialPresent 

func (d *Desktop) CredentialPresent(target string, t CredentialType) (bool, error)

CredentialPresent reports whether a target exists in the credential set. The secret is read but never returned, and its buffer is released immediately.

func (*Desktop) DeleteCredential 

func (d *Desktop) DeleteCredential(target string, t CredentialType) error

DeleteCredential removes a credential. A target that is already absent is not an error, so shutdown cleanup is idempotent.

func (*Desktop) DismissSecurityBanner 

func (d *Desktop) DismissSecurityBanner()

DismissSecurityBanner removes the security banner if shown.

func (*Desktop) Displays 

func (d *Desktop) Displays() ([]DisplayInfo, error)

Displays enumerates the monitors with their bounds, work area, and DPI/scale.

func (*Desktop) Do 

func (d *Desktop) Do(fn func() error) error

Do runs fn on the engine's dedicated STA thread and returns its error. It is the only sanctioned way to touch COM/UIA/input state. Calls are serialized: fn runs to completion before the next queued job starts.

func (*Desktop) DomainAndSKU 

func (d *Desktop) DomainAndSKU() (HostFacts, error)

DomainAndSKU queries device domain-join, OS SKU, hostname, and BIOS serial via WMI.

func (*Desktop) ElementState added in v1.3.0 

func (d *Desktop) ElementState(label int) (ElementState, error)

ElementState reads the current state of a labeled element. STA thread.

func (*Desktop) ExpandLabel 

func (d *Desktop) ExpandLabel(label int) error

ExpandLabel / CollapseLabel operate the ExpandCollapse pattern (combo boxes, tree items, expanders).

func (*Desktop) GetSystemInfo 

func (d *Desktop) GetSystemInfo() (SystemInfo, error)

GetSystemInfo collects OS, computer-system, CPU, and disk inventory via WMI.

func (*Desktop) InjectCredential 

func (d *Desktop) InjectCredential(target string, t CredentialType, clickAt *Point, allowUnmasked bool) (int, error)

InjectCredential types a stored secret as keystrokes without ever returning it.

If clickAt is non-nil the point is clicked first, so a caller can focus the password field in the same serialized operation — no window can steal focus between the click and the keystrokes.

It returns the number of characters typed, which is safe to report: the length alone tells a caller the injection happened without disclosing the value.

func (*Desktop) InvokeLabel 

func (d *Desktop) InvokeLabel(label int) error

InvokeLabel activates the labeled control via the Invoke pattern (buttons, links, menu items).

func (*Desktop) LastState 

func (d *Desktop) LastState() *DesktopState

LastState returns the most recent Snapshot, or nil if none has been taken.

func (*Desktop) LaunchApp 

func (d *Desktop) LaunchApp(ctx context.Context, name string) (string, error)

LaunchApp starts an application by name via the shell (Start-Process), which resolves apps on PATH and registered app execution aliases (e.g. "notepad", "msedge"). It waits briefly and returns the launched window if one appears.

func (*Desktop) LaunchExecutable 

func (d *Desktop) LaunchExecutable(path string, args []string, cwd string) (int, error)

LaunchExecutable starts an executable directly (detached), returning its PID.

func (*Desktop) ListServices 

func (d *Desktop) ListServices(filter string) ([]ServiceInfo, error)

ListServices returns Windows services, optionally filtered by a name/display substring (case-insensitive).

func (*Desktop) Logger 

func (d *Desktop) Logger() *slog.Logger

Logger returns the engine's logger.

func (*Desktop) MarkRecording 

func (d *Desktop) MarkRecording(label string) bool

MarkRecording adds an agent-supplied marker to the session recording timeline, if recording is active.

The label is namespaced and length-capped, and both matter. Guardrail events go into the same file as "SECURITY: ..." (see ShowSecurityBanner), so an unprefixed agent marker reading "SECURITY: kill switch disarmed by operator" was byte-identical to a real one in the forensic timeline. The prefix makes the author of every marker unambiguous; the cap stops an unbounded label filling the recording volume.

func (*Desktop) Matches added in v1.3.0 

func (d *Desktop) Matches(spec SelectorSpec) ([]LabeledElement, error)

Matches returns every interactive element in the most recent snapshot that the spec matches, in tree order. It is what element.count reads, and what makes the ambiguity check possible: the count is the ground truth a hand-written selector can only guess at.

func (*Desktop) MoveCursor 

func (d *Desktop) MoveCursor(x, y int) error

MoveCursor moves the cursor to (x,y) without clicking.

func (*Desktop) Notify 

func (d *Desktop) Notify(ctx context.Context, title, message string)

Notify shows a best-effort Windows toast notification (WinRT via Windows PowerShell 5.1). Used by the guardrails layer to surface run-context auto-limiting and policy actions. Failures are logged, not returned.

func (*Desktop) ProcessKill 

func (d *Desktop) ProcessKill(pid uint32, nameSubstr string) (int, error)

func (*Desktop) ProcessList 

func (d *Desktop) ProcessList(sortBy string, limit int) ([]ProcessInfo, error)

ProcessList returns running processes via WMI, sorted by sortBy ("memory", "name", or "pid") and limited to at most limit entries (0 = no limit).

func (*Desktop) QueryWMI 

func (d *Desktop) QueryWMI(namespace, wql string) ([]wmi.Row, error)

QueryWMI runs a WQL query against an arbitrary WMI namespace (e.g. root\Microsoft\Windows\DeviceGuard or root\CIMV2\Security\MicrosoftVolumeEncryption) and returns the raw rows. The go-bindings-wmi Service is thread-affine — Connect locks the OS thread and Close unlocks it — so the whole Connect → Query → Close sequence runs on one dedicated goroutine. Used for the just-in-time device-health posture reads outside the long-lived root\cimv2 worker.

func (*Desktop) ReadLabel 

func (d *Desktop) ReadLabel(label int) (name, value string, err error)

ReadLabel returns the labeled element's current name and value (via the Value pattern when available), for verification and assertions. STA thread.

func (*Desktop) RecordInput added in v1.1.0 

func (d *Desktop) RecordInput(ctx context.Context, stopVK uint32, out func(RecordedInput)) error

RecordInput installs input hooks and streams enriched events to out until ctx is cancelled or the stop key is pressed. It runs the caller's out on the recorder goroutine; out must not block for long.

It reads live UI state and only works on an interactive desktop; on a host that cannot host UIA the hit-test simply yields empty element info and clicks fall back to coordinates.

func (*Desktop) RecordInputWithMark added in v1.3.0 

func (d *Desktop) RecordInputWithMark(
	ctx context.Context,
	stopVK, markVK uint32,
	out func(RecordedInput),
) error

RecordInputWithMark is RecordInput with an explicit assertion-mark key.

The mark key is what turns a recording into a test. A capture of a human doing a task records the actions and notices nothing about whether they worked; pointing at what matters and pressing the key is the cheapest moment to say so, because it is the moment the author is already looking at it.

func (*Desktop) RecordingStatus 

func (d *Desktop) RecordingStatus() (RecordingStatus, bool)

RecordingStatus returns the session recorder's status and whether recording is active.

func (*Desktop) ResizeWindow 

func (d *Desktop) ResizeWindow(titleSubstr string, x, y, width, height int) (WindowInfo, error)

ResizeWindow moves and resizes the first window matching titleSubstr.

func (*Desktop) Resolve added in v1.3.0 

func (d *Desktop) Resolve(spec SelectorSpec) (label, candidates int, err error)

Resolve returns the label the spec designates, and how many elements it matched. The count is returned even on failure, so a caller can report that a selector was ambiguous rather than merely unsatisfied.

func (*Desktop) RootInfo 

func (d *Desktop) RootInfo() (name string, childCount int, err error)

RootInfo returns the UI Automation desktop root element's name and its direct child count. It is a lightweight smoke test confirming that COM, DPI awareness, and the UIA client are all working on the engine thread.

func (*Desktop) RunPowerShell 

func (d *Desktop) RunPowerShell(ctx context.Context, command string, timeout time.Duration) (PowerShellResult, error)

RunPowerShell executes a PowerShell command and returns its combined output and exit code. The command is passed via -EncodedCommand so no escaping is needed. It is bounded by timeout (a non-positive timeout means no limit) and also honors ctx cancellation.

The child process runs with an environment reconstructed from the registry (see winenv.go), so commands see the full user PATH/vars even when the MCP host launched the server with a stripped environment. The interpreter is resolved to an absolute path against that reconstructed PATH.

func (*Desktop) RunWindowsPowerShell 

func (d *Desktop) RunWindowsPowerShell(
	ctx context.Context,
	command string,
	timeout time.Duration,
) (PowerShellResult, error)

RunWindowsPowerShell runs a command specifically under Windows PowerShell 5.1 (powershell.exe), required for WinRT APIs such as toast notifications that PowerShell 7 (pwsh) does not expose.

func (*Desktop) Screenshot 

func (d *Desktop) Screenshot() (pngData []byte, width, height int, scaleDenom int, err error)

Screenshot captures the entire virtual desktop (all monitors) as a PNG. Returns the PNG bytes and the encoded image dimensions. Coordinates from Snapshot are in physical pixels; if the returned image was downscaled, its dimensions differ from the physical desktop (the caller reports the scale).

func (*Desktop) Scroll 

func (d *Desktop) Scroll(x, y, wheelClicks int, direction string) error

Scroll moves the cursor to (x,y) and scrolls the wheel. direction is "up", "down", "left", or "right"; wheelClicks is the number of wheel notches. Horizontal scrolling is emulated by holding Shift over the vertical wheel, matching the Python implementation.

func (*Desktop) SelectLabel 

func (d *Desktop) SelectLabel(label int) error

SelectLabel selects the labeled item via the SelectionItem pattern (list items, radio buttons, tabs).

func (*Desktop) SendShortcut 

func (d *Desktop) SendShortcut(keys []string) error

SendShortcut presses a key chord, e.g. []string{"ctrl","shift","esc"}. Keys are pressed in order and released in reverse.

func (*Desktop) SetValueLabel 

func (d *Desktop) SetValueLabel(label int, value string) error

SetValueLabel sets the labeled control's value via the Value pattern (text fields), without synthesizing keystrokes.

func (*Desktop) ShowSecurityBanner 

func (d *Desktop) ShowSecurityBanner(text string)

ShowSecurityBanner raises a persistent, human-visible on-screen banner for a security event and marks the recording timeline so the event is captured on video. Best-effort: a no-op if the overlay manager is unavailable.

func (*Desktop) Snapshot 

func (d *Desktop) Snapshot(opts SnapshotOptions) (*DesktopState, error)

Snapshot captures the current desktop state: windows, and the labeled interactive UI tree of the foreground window (and optionally all windows). It stores the result for subsequent label-based interaction.

func (*Desktop) ToggleLabel 

func (d *Desktop) ToggleLabel(label int) error

ToggleLabel toggles the labeled control via the Toggle pattern (checkboxes, switches).

func (*Desktop) TopLevelWindows 

func (d *Desktop) TopLevelWindows() ([]WindowInfo, error)

TopLevelWindows returns the visible, titled top-level windows.

func (*Desktop) TypeText 

func (d *Desktop) TypeText(text string) error

TypeText types Unicode text at the current focus. Newlines and tabs are sent as Enter and Tab virtual keys; all other characters are sent as Unicode keystrokes so no keyboard-layout mapping is needed.

func (*Desktop) WriteCredential 

func (d *Desktop) WriteCredential(spec CredentialSpec) error

WriteCredential installs a credential into the calling user's credential set. The secret is copied into a UTF-16 blob, handed to CredWrite, and the blob is zeroed before returning.

type DesktopState 

type DesktopState struct {
	// Foreground is the current foreground window (zero value if none).
	Foreground WindowInfo
	// Windows lists visible, titled top-level windows.
	Windows []WindowInfo
	// Interactive lists the labeled interactive elements, in label order.
	Interactive []LabeledElement
	// TreeText is the human-readable semantic UI tree.
	TreeText string
}

DesktopState is the captured perception of the desktop returned by Snapshot. It is also stored on the engine so that label-based interaction (Click/Type by label) can resolve labels to screen coordinates.

type DiskInfo 

type DiskInfo struct {
	Drive   string
	Volume  string
	SizeGB  float64
	FreeGB  float64
	UsedPct int
}

DiskInfo summarizes a logical disk.

type DisplayInfo 

type DisplayInfo struct {
	Index    int
	Primary  bool
	Bounds   Rect // full monitor rectangle (virtual-screen coords)
	WorkArea Rect // usable area excluding taskbar
	DPI      int  // effective DPI (96 = 100%)
	ScalePct int  // scale percentage (e.g. 150)
}

DisplayInfo describes one monitor.

type ElementInfo 

type ElementInfo struct {
	Name          string
	ControlType   string
	ControlTypeID int32
	ClassName     string
	AutomationID  string
	Rect          Rect
	Enabled       bool
	Offscreen     bool
	ProcessID     int32
	// IsPassword marks a field that masks its input (a password box). The journey
	// recorder reads it to redact keystrokes typed into such a field.
	IsPassword bool
}

ElementInfo is a snapshot of a UI Automation element's key properties, read off the element while it is alive so the element handle can then be released.

type ElementState added in v1.3.0 

type ElementState struct {
	Name        string
	ControlType string
	Value       string
	Enabled     bool
	Checked     bool
	Selected    bool
	Focused     bool
	Expanded    bool

	// The Has* flags are pattern availability: what this control can actually do.
	// They are what separates a checkbox from a button without guessing from the
	// control type, which is why the recorder reads them at every hit-test.
	HasValue          bool
	HasToggle         bool
	HasSelection      bool
	HasInvoke         bool
	HasExpandCollapse bool
}

ElementState is what an assertion can read about one element. It is a snapshot of the properties and patterns that carry state, taken together on the STA thread so the fields describe one consistent moment rather than several.

The Has* flags distinguish "false" from "the control does not have this kind of state": asserting that a Button is unchecked should report that a Button has no toggle state, not that it is unchecked.

type HostFacts 

type HostFacts struct {
	PartOfDomain bool
	Domain       string
	OSSKU        uint32
	OSCaption    string
	Hostname     string
	Serial       string
}

HostFacts is device domain-join, OS-edition, and identity information used by the guardrails layer.

type LabeledElement 

type LabeledElement struct {
	Label   int
	Info    ElementInfo
	CenterX int
	CenterY int
	// contains filtered or unexported fields
}

LabeledElement is an interactive element assigned a stable label within a snapshot, together with its click point (the center of its bounding rect).

type Options 

type Options struct {
	// Overlay enables visual-feedback overlays (green highlight around the
	// active window, orange flash at click points) for screen capture and video
	// recording.
	Overlay bool

	// SecurityOverlay starts the overlay manager even when Overlay is false, so
	// the security-event banner can be shown regardless of the decorative
	// overlay setting. Set by the security layer.
	SecurityOverlay bool

	// Record, when its Dir is set, records the whole session to a video file so
	// every session is tracked regardless of persona.
	Record RecorderOptions
}

Options configures the engine.

type Point 

type Point struct{ X, Y int }

Point is a screen coordinate in physical pixels.

type PowerShellResult 

type PowerShellResult struct {
	// Output is the combined stdout+stderr text.
	Output string
	// ExitCode is the process exit code (0 on success).
	ExitCode int
	// TimedOut reports whether the command was killed for exceeding its timeout.
	TimedOut bool
}

PowerShellResult is the outcome of a PowerShell invocation.

type ProcessInfo 

type ProcessInfo struct {
	PID            uint32
	Name           string
	WorkingSetKB   uint64
	ThreadCount    uint32
	ParentPID      uint32
	ExecutablePath string
}

ProcessInfo summarizes a running process.

type RecordedInput added in v1.1.0 

type RecordedInput struct {
	Kind    string      // "click" | "char" | "key" | "assert"
	X, Y    int         // click point, or the cursor position at a mark
	Element ElementInfo // the element under a click or mark (resolved via UIA)
	// State is what that element can do and what it currently holds. It is read at
	// the same hit-test, which is the only moment both the element and its tree are
	// in hand — and it is what lets a click be recorded as the verb it meant rather
	// than as a click.
	State  ElementState
	Button string // left | right | middle
	Double bool   // a double-click
	Char   rune   // a typed character
	Secure bool   // the char was typed into a password field → redact
	Key    string // a named non-text key (Enter, Tab, …)
}

RecordedInput is one enriched user action the recorder emits. It is engine-level and holds no journey types; the caller maps it to a journey event.

type RecorderOptions 

type RecorderOptions struct {
	// Dir is the output directory for session recordings. Empty disables it.
	Dir string
	// FPS is the capture rate (default 4).
	FPS int
	// MaxWidth downscales frames wider than this (default 1280; 0 = full res).
	MaxWidth int
	// Quality is the MJPEG JPEG quality 1-100 (default 60).
	Quality int
	// Codec selects the video encoder: "h264" or "h265" use ffmpeg when
	// available (small files) and fall back to pure-Go MJPEG-AVI when not;
	// "mjpeg" forces the pure-Go writer. Default "h264".
	Codec string
	// Stamp is the shared session stamp used to name the output. Empty means the
	// recorder mints its own from the clock. When set, the recording file
	// (session-<stamp>.*) correlates by name with the audit file the same stamp
	// named, which is what lets an evidence bundle pair them.
	Stamp string
}

RecorderOptions configures session video recording.

type RecordingStatus 

type RecordingStatus struct {
	Recording   bool
	VideoPath   string
	MarkersPath string
	Frames      int
	DurationSec float64
	FPS         int
}

RecordingStatus reports the recorder's state.

type Rect 

type Rect struct {
	Left, Top, Right, Bottom int
}

Rect is a screen rectangle in physical pixels (the engine is per-monitor-DPI aware, so these are true device coordinates).

func (Rect) Center 

func (r Rect) Center() (x, y int)

Center returns the rectangle's center point.

func (Rect) Empty 

func (r Rect) Empty() bool

Empty reports whether the rectangle has no area.

func (Rect) Height 

func (r Rect) Height() int

Height returns the rectangle height.

func (Rect) Width 

func (r Rect) Width() int

Width returns the rectangle width.

type SelectorSpec added in v1.3.0 

type SelectorSpec struct {
	AutomationID string
	Name         string
	ControlType  string
	// NameMatch qualifies Name. Empty means MatchExact.
	NameMatch string
	// Occurrence is "unique" (default), "first", or a 0-based index.
	Occurrence string
}

SelectorSpec identifies a UI element. Exactly one of AutomationID or Name identifies it; ControlType may narrow either.

AutomationID is the top of the stability ladder: it is developer-assigned and, unlike the accessible name, survives translation.

func (SelectorSpec) Describe added in v1.3.0 

func (s SelectorSpec) Describe() string

Describe renders the spec for an error message.

func (SelectorSpec) Empty added in v1.3.0 

func (s SelectorSpec) Empty() bool

Empty reports whether the spec identifies nothing.

type ServiceInfo 

type ServiceInfo struct {
	Name        string
	DisplayName string
	State       string
	StartMode   string
	PID         uint32
}

ServiceInfo summarizes a Windows service.

type SnapshotOptions 

type SnapshotOptions struct {
	// AllWindows, when true, walks every visible window's tree (subject to the
	// global node budget). When false, only the foreground window is walked,
	// which is faster and usually sufficient.
	AllWindows bool
}

SnapshotOptions controls what a Snapshot captures.

type SystemInfo 

type SystemInfo struct {
	Hostname      string
	OSCaption     string
	OSVersion     string
	OSArch        string
	BuildNumber   string
	LastBoot      string
	Manufacturer  string
	Model         string
	CPUName       string
	PhysicalCores uint32
	LogicalCPUs   uint32
	TotalMemoryMB uint64
	FreeMemoryMB  uint64
	Disks         []DiskInfo
}

SystemInfo is a snapshot of OS and hardware inventory.

type WindowInfo 

type WindowInfo struct {
	Handle       uintptr
	Title        string
	ClassName    string
	Rect         Rect
	ProcessID    uint32
	Minimized    bool
	IsForeground bool
}

WindowInfo describes a top-level window.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.