Repository navigation
Mirror: draw the mouse pointer, and show where you tapped - #120
Merged
Merged
Conversation
"i cannot see the mouse on the mirror and cannot click on buttons on the mirror. unless i use the mouse i can't see." ⚠⚡ mss.grab() DOES NOT CAPTURE THE CURSOR. That is true of mss on every platform, so the mirror has never once shown a pointer — tapping moved a cursor nobody could find, which makes aiming at a button guesswork. The mirror was not missing click support; it was missing the ability to AIM. - The pointer is now drawn onto each captured frame from pyautogui's position, which is the SAME coordinate space the existing click mapping uses — so what you see is exactly where a tap will land. Proved, not assumed: the test maps the drawn arrow back through the client's own tap formula at four positions including both corners. - ⚠ Drawn black inside a white halo, not one flat colour. This is looked at on an e-ink reader, where a white arrow vanishes on a light window and a black one vanishes on a dark one. Asserted against both backgrounds. - ⚠ Scaled with the frame. Frames are downscaled to ~1680px for the wire, so a true-size cursor would be a handful of pixels on a 6-inch screen. - ⚠ It can never break the mirror. If the pointer cannot be read — no display, pyautogui unavailable — the frame is returned untouched. A failure costs the cursor, not the screen. Pinned by a test that makes position() throw. And immediate feedback on tap: the drawn pointer only arrives with the NEXT frame, which is a few hundred ms away and longer on the "Lite" speed, so until then a tap still produced nothing visible. A mark now appears where you touched. ⚠ A static mark that appears once and disappears once, NOT an animated ripple: on e-ink every frame of an animation is a full panel repaint, and the ripple would cost more flashes than the mirror itself. The hint under the mirror said "Tap the mirror to click", which was true and useless without a visible pointer. It now says the arrow shows where the mouse is. test_mirror_cursor.py (13): position accuracy at four points, off-screen and zero-size ignored, a throwing position() leaving the frame untouched, and legibility on both a white and a dark window. The page's inline JS is syntax-checked separately — ast.parse on the Python proves nothing about a string containing a script. Claude-Session: https://claude.ai/code/session_01LTqV95KXLxBZuJhKULbmyp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
⚠⚡
mss.grab()does not capture the cursorThat's true of mss on every platform, so the mirror has never once shown a pointer. Tapping moved a cursor nobody could find, which makes aiming at a button guesswork. The mirror was not missing click support — it was missing the ability to aim.
The pointer is now drawn onto each captured frame from
pyautogui.position(), which is the same coordinate space the existing click mapping already uses, so what you see is exactly where a tap will land. Proved rather than assumed: the test maps the drawn arrow back through the client's own tap formula at four positions, including both corners.position()throw.Immediate feedback on tap
The drawn pointer only arrives with the next frame, which is a few hundred ms away and longer on the "Lite" speed — so until then a tap still produced nothing visible. A mark now appears where you touched.
⚠ A static mark that appears once and disappears once, not an animated ripple: on e-ink every frame of an animation is a full panel repaint, and the ripple would cost more flashes than the mirror itself.
The hint under the mirror said "Tap the mirror to click", which was true and useless without a visible pointer. It now says the arrow shows where the mouse is.
Tests
test_mirror_cursor.py(13): position accuracy at four points, off-screen and zero-size ignored, a throwingposition()leaving the frame untouched, and legibility on both a white and a dark window.⚠ The page's inline JS is syntax-checked separately —
ast.parseon the Python file proves nothing about a string that happens to contain a script.🤖 Generated with Claude Code
https://claude.ai/code/session_01LTqV95KXLxBZuJhKULbmyp
Generated by Claude Code