Add wireless Rokid–iPhone MiniCPM conversations and Windows IPA workflow

[MengzhangLI]

Rokid can send camera frames and microphone audio directly to an iPhone over its Personal Hotspot or a shared Wi-Fi network. OpenGlass runs Apple on-device ASR, MiniCPM-V 4.6, and system TTS on the phone. With models present, opening the app starts reception automatically; a spoken question is finalized after a pause, answered, read aloud, and followed by automatic listening for the next question. Each inference clears the KV cache.

The Rokid collector discovers and verifies the receiver through Bonjour or the Wi-Fi gateway, remembers the selected phone, and uses one endpoint for both streams. Automatic mode does not fall back to USB. Reconnect backoff, a stalled-write watchdog, and generation-checked camera callbacks handle interruptions. The existing explicit PC/USB address path remains available. A separate OpenGlass WiFi Android variant allows installation when the original collector's signing key is unavailable. Its glasses display now shows one dim, centered status line instead of the diagnostic dashboard; connection settings remain available by touch long press or Menu. Normal taps and confirmation keys only resume a paused session, while an explicit menu action pauses capture; confirmation key-down is consumed to prevent the focused HUD from opening settings on a held key. The HUD disables the full-screen focus highlight and the settings dialog has an explicit Close button.

The iOS overlay builds against pinned official MiniCPM-V-Apps sources in GitHub Actions. Windows scripts provide artifact verification, USB relay, and optional Wi-Fi configuration. The local-only Windows signing workflow has successfully installed the iPhone app; credentials and model weights are excluded from CI. Updates retain the existing model files.

Continuous listening treats an empty final as a normal task rotation, avoiding the old 0.5–8 second retry gaps during silence. Short noise cannot end a task before any words are recognized. Audio arriving during endAudio/finalization is held briefly for the next task only after an empty result; a nonempty result, error, stop or interruption clears it. ASR metadata in /health distinguishes accepted samples from received network packets without exposing audio or transcripts. Real recognition errors retain bounded backoff, and inference/TTS keep their existing input gates.

Validation:

• iOS source 0877663 passed the device Release build: https://github.com/OpenSQZ/OpenGlass/actions/runs/39057876255. The run passed 1,399 protocol checks, 2,087 voice-loop/timing assertions, seven preparation tests, and seven relay tests. Downloaded IPA metadata and SHA256 were verified.
• Android assembleWifi, testWifiUnitTest (seven tests), and assembleDebug passed. APK manifest and signature were verified. Eight desktop Rokid regression tests and PowerShell parsing passed.
• Device queries confirm iPhone OpenGlass build 5 and Rokid OpenGlass WiFi 0.2.0-wifi/code 5 are installed; the phone's two model files remain present with their expected sizes. The older Rokid collector remains installed.
• Live iPhone-hotspot test on 2026-10-18: Rokid logs showed direct image POSTs and an audio WebSocket to the phone. With the glasses USB disconnected (no ADB device), phone image/audio counters continued growing and completed_turns reached 3. The user then confirmed continuous question/answer/TTS without tapping buttons, with both USB cables disconnected. Hotel Wi-Fi has not been tested.
• Wireless transport source commit c439fa3 also passed CI: https://github.com/OpenSQZ/OpenGlass/actions/runs/39058773419. Its iOS Swift sources are identical to installed build 5.
• The user confirmed code 7 displays a single line and completes the first ASR/TTS turn, then reported recurring empty recognition on subsequent turns. The earlier successful wireless test is not evidence of sustained reliability. The current iOS speech change requires an updated IPA and a new multi-turn device test.
• Android code 8 removes an unreliable vendor silenced flag from the HUD; that flag appeared even with clear nonzero microphone samples. assembleWifi, manifest/signature verification, and an in-place install passed; camera/microphone permissions remain granted.
• Source 57589c0 passed macOS CI: https://github.com/OpenSQZ/OpenGlass/actions/runs/39061505661. All 2,176 voice-loop/diagnostics/handoff assertions, 1,399 wire-protocol checks, seven preparation tests, seven relay tests and the device Release build passed. Downloaded IPA build 11 passed manifest, ZIP integrity, Info.plist and SHA256 verification (1B9839531088356F914B6510C33377A971CFBF6EF162915106DC196413305B6A). Signing/install and the new multi-turn test await the user's device step.

The iPhone app must stay in the foreground. ASR pauses during inference and TTS, with a shared echo hold before resuming. Temporary recognition failures retry automatically; missing permissions/language resources and model failures remain explicit. A user pause is respected across callbacks and foreground restoration. The app does not provide simultaneous speaking/listening or speech interruption of TTS, and hotel Wi-Fi must permit peer-to-peer traffic.