Skip to content

fix(prompt): a hosted model gets the machine's layout on demand, not in every prompt - #853

Open
OAraLabs wants to merge 1 commit into
mainfrom
fix/anatomy-on-demand-for-hosted
Open

OAraLabs wants to merge 1 commit into
mainfrom
fix/anatomy-on-demand-for-hosted

Conversation

@OAraLabs

Copy link
Copy Markdown
Owner

Why

The anatomy section (hardware, services, network, tailnet peers) went into every system prompt, whichever model served the turn. On an isolated test rig, the first reply from a hosted model listed the box's GPU and its tailnet peers by name.

What

  • The machine-and-network section is fenced when the prompt is built, on both build paths: profile bootstrap files and the legacy toggles.
  • A request to a hosted model gets a one-line pointer instead: "This machine's hardware, services and network are not in this prompt. Call the anatomy tool when a task needs them." The tool was already registered, and cloud tiers send every tool.
  • Local routes (llama_cpp, ollama, lm_studio, vllm) are unchanged. Anything else, including an unknown provider, counts as hosted.
  • The decision is per request:
    • a /claude-style session is judged by its route;
    • a hosted fallback for a local primary loses the section for that round.
  • New key anatomy.include_in_hosted_prompt: false (template + generated reference). Only a literal true sends the full section to hosted models on every turn.

Verification

  • New tests/test_anatomy_hosted_routes.py (34). It covers:

    • the hosted/local table and the literal-true rule;
    • both build paths, byte-identical outside the section, and idempotence;
    • the request a hosted model actually receives (no host, IP, peer or path), and that a local one is unchanged;
    • the route override, a fallback pin, and a wiring pin for all three loops.

    6 fail without the loop change.

  • Full suite compared with a clean main checkout on the same Mac: no new failures. Main's 20 failures and 131 errors are this Mac's environment; two tests flaky under the parallel run pass alone. Config guards (defaults equality, drift, placement, sdist) pass.

  • Ruff clean.

🤖 Generated with Claude Code

…in every prompt

The anatomy section (hardware, services, network, tailnet peers) went into
every system prompt, whichever model served the turn. On a test rig the first
reply from a hosted model listed the box's GPU and its tailnet peers by name.

- The section is fenced when the prompt is built (both build paths).
- A request to a hosted model (anything but llama_cpp, ollama, lm_studio,
  vllm; an unknown provider counts as hosted) gets a one-line pointer to the
  anatomy tool instead, which the model calls when a task needs the layout.
- Judged per request: a /claude-style session by its route, and a hosted
  fallback for a local primary for its round.
- anatomy.include_in_hosted_prompt (false; only a literal true) sends the full
  section to hosted models on every turn. Local routes are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant