Skip to content

Mirror: draw the mouse pointer, and show where you tapped - #120

Merged
arancool3000 merged 1 commit into
mainfrom
claude/mirror-cursor-clicks
Aug 8, 2026
Merged

arancool3000 merged 1 commit into
mainfrom
claude/mirror-cursor-clicks

Conversation

@arancool3000

Copy link
Copy Markdown
Owner

"i cannot see the mouse on the mirror and cannot click on buttons on the mirror. unless i use the mouse i can't see."

⚠⚡ mss.grab() does not capture the cursor

That's true of mss on every platform, so the mirror has never once shown a pointer. Tapping moved a cursor nobody could find, which makes aiming at a button guesswork. The mirror was not missing click support — it was missing the ability to aim.

The pointer is now drawn onto each captured frame from pyautogui.position(), which is the same coordinate space the existing click mapping already uses, so what you see is exactly where a tap will land. Proved rather than assumed: the test maps the drawn arrow back through the client's own tap formula at four positions, including both corners.

  • ⚠ Drawn black inside a white halo, not one flat colour. This is looked at on an e-ink reader, where a white arrow vanishes on a light window and a black one vanishes on a dark one. Asserted against both backgrounds.
  • ⚠ Scaled with the frame. Frames are downscaled to ~1680px for the wire, so a true-size cursor would be a handful of pixels on a 6-inch screen.
  • ⚠ It can never break the mirror. If the pointer cannot be read — no display, pyautogui unavailable — the frame is returned untouched. A failure costs the cursor, not the screen. Pinned by a test that makes position() throw.

Immediate feedback on tap

The drawn pointer only arrives with the next frame, which is a few hundred ms away and longer on the "Lite" speed — so until then a tap still produced nothing visible. A mark now appears where you touched.

⚠ A static mark that appears once and disappears once, not an animated ripple: on e-ink every frame of an animation is a full panel repaint, and the ripple would cost more flashes than the mirror itself.

The hint under the mirror said "Tap the mirror to click", which was true and useless without a visible pointer. It now says the arrow shows where the mouse is.

Tests

test_mirror_cursor.py (13): position accuracy at four points, off-screen and zero-size ignored, a throwing position() leaving the frame untouched, and legibility on both a white and a dark window.

⚠ The page's inline JS is syntax-checked separately — ast.parse on the Python file proves nothing about a string that happens to contain a script.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LTqV95KXLxBZuJhKULbmyp


Generated by Claude Code

"i cannot see the mouse on the mirror and cannot click on buttons on the
mirror. unless i use the mouse i can't see."

⚠⚡ mss.grab() DOES NOT CAPTURE THE CURSOR. That is true of mss on every
platform, so the mirror has never once shown a pointer — tapping moved a
cursor nobody could find, which makes aiming at a button guesswork. The mirror
was not missing click support; it was missing the ability to AIM.

- The pointer is now drawn onto each captured frame from pyautogui's position,
  which is the SAME coordinate space the existing click mapping uses — so what
  you see is exactly where a tap will land. Proved, not assumed: the test maps
  the drawn arrow back through the client's own tap formula at four positions
  including both corners.
- ⚠ Drawn black inside a white halo, not one flat colour. This is looked at on
  an e-ink reader, where a white arrow vanishes on a light window and a black
  one vanishes on a dark one. Asserted against both backgrounds.
- ⚠ Scaled with the frame. Frames are downscaled to ~1680px for the wire, so a
  true-size cursor would be a handful of pixels on a 6-inch screen.
- ⚠ It can never break the mirror. If the pointer cannot be read — no display,
  pyautogui unavailable — the frame is returned untouched. A failure costs the
  cursor, not the screen. Pinned by a test that makes position() throw.

And immediate feedback on tap: the drawn pointer only arrives with the NEXT
frame, which is a few hundred ms away and longer on the "Lite" speed, so until
then a tap still produced nothing visible. A mark now appears where you
touched.
⚠ A static mark that appears once and disappears once, NOT an animated ripple:
on e-ink every frame of an animation is a full panel repaint, and the ripple
would cost more flashes than the mirror itself.

The hint under the mirror said "Tap the mirror to click", which was true and
useless without a visible pointer. It now says the arrow shows where the mouse
is.

test_mirror_cursor.py (13): position accuracy at four points, off-screen and
zero-size ignored, a throwing position() leaving the frame untouched, and
legibility on both a white and a dark window. The page's inline JS is
syntax-checked separately — ast.parse on the Python proves nothing about a
string containing a script.

Claude-Session: https://claude.ai/code/session_01LTqV95KXLxBZuJhKULbmyp
@arancool3000
arancool3000 merged commit e12a0dc into main Aug 8, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants