How to get pixel coordinates on macOS, Windows, and Linux

HiDPI screens have two coordinate systems: physical pixels (the screenshot’s grid) and logical points (what most click APIs use on macOS). On a 2× Retina display a screenshot pixel at (1624, 880) is the logical point (812, 440). Divide by the display’s scale factor — or use a tool that records the scale per monitor and converts for you.

Press Cmd+Shift+4 — the screenshot crosshair shows live cursor coordinates in logical points; press Esc to cancel without capturing. For coordinates you can save, verify, and hand to a script, freeze the screen with pixelcoords and mark the spot.

Wayland’s security design withholds global coordinates and window geometry from applications, so live trackers can’t work. A frozen-snapshot overlay is the model Wayland permits: pixelcoords freezes via the desktop portal, and --pick marks a single window with window-relative coordinates.

pyautogui speaks logical points on macOS and physical pixels on Windows and X11. pixelcoords emit --format pyautogui applies the right convention per monitor from the recorded DPI scale. The Windows pyautogui convention follows its documented behavior and has not yet been verified on hardware.

Mark the region once, then run pixelcoords assert --session <dir> --point X,Y --expect name. The exit code is the API: 0 hit, 1 miss, 2 malformed question — CI and computer-use agents can gate on it. To score a whole run, --stdin reads one point per line and answers in a single process.

pixelcoords resolve --label name --units auto returns the click point in the space your input API expects: logical points on macOS, physical pixels on Windows and X11. Each region converts through its own monitor’s DPI scale, so a mixed-DPI desktop comes out right without the caller knowing it was mixed.

pixelcoords wait --label dialog --for match blocks until that region matches its saved crop again, and --for change blocks until one stops matching. It exits 0 when the condition holds and 1 on timeout, so a script can tell "it never appeared" from "the question was malformed". Two honest limits: matching is contrast-normalized, so --for change cannot see a region that dims uniformly behind a modal, and a blinking cursor inside a watched region is enough to trip it on its own — lower --min-score when that happens.

pixelcoords mcp serves the same commands over the Model Context Protocol on stdio — point an MCP client at the binary and the model gets six read-only tools: list sessions, resolve a click point, assert a point, wait, find, diff. Asking a marked session where to click reads a file and sends no image at all, where the alternative is a screenshot per step. It cannot mark regions — a human does that once in the overlay, and the model asks about it from then on. A miss or a timeout comes back as an ordinary result with ok false, not as an error, so the model reacts to it instead of retrying.

pixelcoords diff compares each marked region against the screen now, or against stored screenshots with --against, and reports how many of that region’s pixels changed. It is scoped to the regions you marked rather than whole screenshots, so a clock in the corner is not a failure, and shaped selections compare by their true silhouette. --tolerance defaults to exact: anti-aliasing usually means CI wants a small nonzero value, but a diff tool that rounds by default is lying by default.