Repository navigation
feat(voice): one voice, Crow, on the phone overlay, kiosk and glasses, with its own card and editor in Bot Builder (kiosk 0.4.0, meta-glasses 0.3.0, artifacts 0.6.4) - #533
Merged
Conversation
…eir confirm entries Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
…a pi-form confirm also gates the voice turn Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
The shared voice turn resolves opts.botId, then deps.voiceBotId (the Crow voice bot), then the device's bound bot, after the fast paths. A missing Crow voice bot is ensured once per runner and answers the same turn. Escalation off acts as noEscalate, timings.surface is recorded, and a bot-defs change drops the cached bot. The boot and turn ensures share one builder. The glasses session tests now pin the voice bot. Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
…w asks before deleting, changing or sending Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
…le back on, and writes, uploads and revokes ask first Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
… as Crow, in Crow's own voice (kiosk 0.4.0, meta-glasses 0.3.0) Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
…y Answers as Crow Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
… a preset switch keeps Memory tools off, and the no-assistant link opens Crow's editor Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
…the Crow voice bot alone (artifacts 0.6.4) Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
… never confirms, and a corrupt voice row says so
- Memory tools off drops every crow-memory/ entry (a carried read-only one kept the whole
crow_memory proxy); the switch and a preset switch read any memory entry; on adds the three back.
- A negation word in the first four words ("claro que no", "of course not", "yes, no wait")
is never a yes and always a no.
- A corrupt crow.voice row warns once from the boot and the turn-time ensure and is left
untouched; the voice editor says the settings can't be read (en + es) instead of
"not set up yet", and a save on it is refused with voice_corrupt.
- Kiosk and glasses no-assistant lines point at Crow's voice in Bot Builder; the room hint
names Reset to preset.
- Hello warms Crow's STT profile, not the device's own.
- The memory server's voice-bot botcron refusal is isError.
- A voice bot's notification settings change needs a spoken yes (direct or proxied; get stays free).
- The confirm question reads a proxied call's params and says "delete schedule N" for a delete.
- A test pins the voice system message byte-identical across turns.
Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
…roxied call whose top-level and params values disagree is refused - crow_notification_settings: the deciding action field comes from the tool (direct: args.action, crow_memory proxy: params.action), and any other present action field that is not "get" also asks. - The confirm question describes only what the executor passes on (params for a category proxy, the call's own args for a direct tool). - A voice bot's proxied call whose top-level copy of a described field (id, delete, section_key, task, ...) disagrees with params is refused outright, before any question. Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
…uestion names what runs, and a disagreeing or flat-only voice delete is refused - One shared module (servers/gateway/ai/proxy-args.js) holds the proxy matcher and crow_tools' parameter normalization; the executor and the voice gate both use it, so they cannot drift. - The confirm question reads the object the executor runs (params, the arguments unwrap, the flat fallback) for every proxy, crow_tools included. - For a voice bot, a proxied call whose top level, params or params.arguments disagrees with what runs is refused; so is a crow_tools call that needs a yes and relies on the flat fallback (the remote route never runs those values). "3" and 3 are the same id. Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Each voice surface answered as a bot chosen on that device: the phone overlay and kiosk as the dashboard bot, the glasses as their own bound bot. This PR replaces that with one voice, Crow. Crow is a reserved bot (
crow.voice) with its own settings, answering the same way on all three surfaces. Hand-offs to other bots come in a later PR. This PR ships the voice bot itself, so it is useful on its own.What changes
The voice bot is created once, and nothing else is edited (
servers/shared/voice-bot.js,servers/gateway/voice/ensure-voice-bot.js,servers/gateway/boot/post-listen.js)crow.voice(display name "Crow") if no such row exists, usingINSERT … ON CONFLICT DO NOTHING. If a row already exists, whether enabled, disabled or unreadable, it is never rewritten. An unreadable row gets a log warning instead.migrated_fromand nothing else is touched. That covers the effective dashboard bot ("Automatic" resolves as the kiosk does) and each device's bound bot. Device records, settings and other bots' definitions are left as they are.migrated_from.skipped_toolswith the reason. Write tools (e.g. a calendar create) are not carried; Crow's own routine tools arrive in the next PR.Every voice turn answers as Crow (
servers/gateway/voice/turn.js)crow.voiceinstead of the device's bound bot.opts.botIdstill overrides it, for the turn check and the evaluation harness.One set of voice rules, and a yes before changes (
servers/gateway/voice/crow-voice.js,voice-lines.js,action-claims.js,turn-helpers.js)pipeline:) is refused from voice, and so is re-enabling a schedule without naming its task.crow_tools), it reads the arguments the executor passes on. The proxy rules now live in one shared module (servers/gateway/ai/proxy-args.js), which the executor and the voice gate both import. On voice, a proxied call is refused when its top-level and executed values disagree, and so is acrow_toolscall that needs a yes but puts its arguments outsideparams.Surfaces, Crow's speaking voice and speech recognition (
bundles/kiosk,bundles/meta-glasses)Bot Builder: Crow's card and editor (
servers/gateway/dashboard/panels/bot-builder/voice-bot-card.js,voice-bot-editor.js,i18n.js)Guards carried from #528
crow_create_schedulerefusespipeline:botcron:crow.voice, returning an error.No schema change, no
SCHEMA_GENERATIONbump, no DELETE or DROP. The AI Companion, which is being retired, is not touched.Tests
voice-bot-ensure: create once, never overwrite, the migration record, the carry-over rules, and two gateways at once.voice-turn-crow: answers as Crow; fast path with no bot; missing, disabled or unreadable bot; escalation off; surface; spoken yes; Spanish; refusals; a byte-stable system message.crow-voice-options.bot-builder-voice-card.crow.voiceonce.Deploy
#528 is already merged. Pull
mainon each host and restart the gateway(s). The kiosk, meta-glasses and artifacts installed copies refresh because of the version bumps.After the restart:
~/.crow/bundles/kiosk/manifest.jsonreads 0.4.0;crow.voice'smigrated_from.toolsandmigrated_from.skipped_toolsto see which old voice tool selections were carried and which were not.Rollback
Revert this PR only, never #528. Then disable the row; this is a disable, never a delete:
UPDATE pi_bot_defs SET enabled = 0 WHERE bot_id = 'crow.voice'Devices still hold their old
bound_bot_idand the old setting is untouched, so the previous behaviour returns.https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp