feat(agent): Responses API transport for Responses-only models - #267
darylfung96 wants to merge 3 commits into
Conversation
…efix) Some gateways serve models exclusively on /v1/responses and 503 /v1/chat/completions. A KB can now opt in with e.g. model: openai-responses/muse-spark-1.3-contributor. - New openkb/responses.py: message/tool/response_format translation, Chat-Completions-shaped adapter, Agents-SDK OpenAIResponsesModel builder - compiler: route _llm_call/_llm_call_async through litellm.responses - query/chat/lint/skills/decks: Agent constructors + run-config builder use the Responses model (creds from bundle, else env) - indexer: clear error for long-PDF PageIndex path (needs completions) - Docs + 12 unit tests
…through - Trim module/function docstrings to the why; fix stale blocks claim - Extract _translate_content shared by both message paths (also fixes untranslated blocks on assistant+tool_calls messages) - _to_dict: explicit continue on failed coercion - Top-level third-party imports (no cycle); SDK wiring test
The openai-responses/ prefix works with any model id on any OpenAI-compatible Responses endpoint; examples now use neutral placeholders instead of a specific model.
darylfung96
changed the title
Release risk — 48 / 100 · MEDIUMinto main · Note What's changingOpt-in Responses transport for Responses-only LLM gateways. Prefix Default Chat Completions path is untouched. Long-PDF PageIndex indexing fails fast if the KB model uses the prefix. Findings
Before merge
Release plan
Blast radius — 7 services touched (1 new), 1 impactedCompile agent touched (~2) · Query / Chat agent touched (~2) · Wiki linter touched (~1) · Skills touched (~4) · KB store & indexing touched (~1) · KB config touched (files only) · Responses new (+16) · HTTP API & CLI impacted · tests: +14 entities in 1 file
Compile agent · touched · 1 files · ~2
Query / Chat agent · touched · 1 files · ~2
Wiki linter · touched · 1 files · ~1
Skills · touched · 2 files · ~4
KB store & indexing · touched · 1 files · ~1
KB config · touched · 2 files · files only
Responses · new · 1 files · +16
HTTP API & CLI · impacted · 5 references into changed code
Deployables / files — secondary
Score breakdown — 8 axes
FireWeave · advisory · blast radius from a fixed service catalog (OpenKB): services are catalog entries, states from sem diff + resolved refs, diagram only when ≥2 services are painted |
🟡 Medium release risk — 47/100🟩🟩🟩🟨🟨⬜⬜⬜⬜⬜ higher = riskier · into local ·
Findings
Before merge
Release plan — Control point: none on the changed paths · Register in FireWeave · During release: Run one internal KB on an openai-responses/ model first and
Blast radius · Services touched — OpenKB REST API, OpenKB CLI reaches it · LLM Responses Gateway, PageIndex API depends
Score breakdown — 8 axes consideredLimited visibility — CI configured but nothing ran on this commit; model listed unknowns. Unscored is not safe.
FireWeave · advisory, never blocks the check · claude-sonnet-5 · 4m 29s · Full report |
🟡 Medium release risk — 47/100🟩🟩🟩🟨🟨⬜⬜⬜⬜⬜ higher = riskier · into local ·
Findings
Before merge
Release plan — Control point: none on the changed paths · Register in FireWeave · During release: Watch debug logs for 'dropping unsupported param' and for em
Blast radius · Services touched — OpenKB CLI, OpenKB API reaches it · OpenAI Responses API depends
Score breakdown — 8 axes consideredLimited visibility — CI configured but nothing ran on this commit; model listed unknowns. Unscored is not safe.
FireWeave · advisory, never blocks the check · claude-sonnet-5 · 4m 42s · Full report |
⏳ Release risk — analysing…into local · FireWeave is reading this change. This comment updates in place when the score is ready. FireWeave · advisory, never blocks the check · prompt b7 |
🟡 Medium release risk — 41/100🟩🟩🟩🟨🟨⬜⬜⬜⬜⬜ higher = riskier · into local ·
Findings
Before merge
Release plan — Control point: none on the changed paths · Register in FireWeave · During release: Have the first KB\(s\) that opt into openai-responses/<id
Blast radius · Services touched — LLM Gateway, PageIndex depends
Score breakdown — 8 axes consideredLimited visibility — CI configured but nothing ran on this commit; model listed unknowns. Unscored is not safe.
FireWeave · advisory, never blocks the check · claude-sonnet-5 · 3m 12s · Full report |
🟡 Medium release risk — 47/100🟩🟩🟩🟨🟨⬜⬜⬜⬜⬜ higher = riskier · into local ·
Findings
Before merge
Release plan — Control point: none on the changed paths · Register in FireWeave · During release: Have the first KB\(s\) that set `model: openai-responses/
|
| Control point | none on the changed paths · Register in FireWeave |
| During release | Have the first KB(s) that set model: openai-responses/<id> be low-traffic, and watch for TruncatedResponseError and JSON-decode failure… · …/skill/evaluator.py:238 |
| After release | Watch openkb add failure logs for the new ValueError on long-PDF indexing after any KB switches its model to a Responses-prefixed one. · openkb/indexer.py:195 |
| Watch | Watch adoption of the openai-responses/ prefix; there is no way to kill it centrally if it misbehaves. · …/agent/compiler.py:429 |
| Already working | Change is opt-in per KB — Every call site branches on is_responses_model(model); a KB whose model: is unchanged (e.g. gpt-5.4) takes the ex… · …/agent/compiler.py:429 |
| Not verified | Whether the adapter has been run against a real /v1/responses gateway anywhere — the repository only shows offline unit tests. · Deployment topology for the API/frontend (single vs multi-replica, how config.yaml changes roll out) — task states 'not mapped'. · Whether any existing KB configs already use a model string that could collide with the openai-responses/ prefix convention. |
Blast radius · Services touched — OpenKB Web UI reaches it · LLM Gateway, PageIndex depends
- LLM Gateway — Called via litellm.responses/AsyncOpenAI instead of Chat Completions ·
openkb/responses.py:267 - PageIndex — Now hard-skipped for Responses-model KBs indexing long PDFs ·
openkb/indexer.py:195 - OpenKB Web UI — Chat/query turns route through the new transport for opted-in KBs ·
…/api/chat.ts:349
Score breakdown — 8 axes considered
Limited visibility — CI configured but nothing ran on this commit; model listed unknowns. Unscored is not safe.
| Axis | Reading | Evidence |
|---|---|---|
| Path criticality | Internal — The changed files are internal LLM-transport plumbing (model-string routing, message translation) with no auth, payment, or infra markers, and the file names a… | openkb/responses.py:50 |
| Blast radius | Crosses services — The changed transport-selection code is reached from a separate web frontend app (frontend/) via the API and from the CLI, both funneling through the same quer… | …/api/chat.ts:349 |
| Reversibility | No state change — The new module only transforms in-memory messages and makes an outbound LLM call; no persisted or in-memory state is written anywhere in the diff. | openkb/responses.py:264 |
| Change size | Medium — 10 files · 638 lines · 4 top-level dirs | 10 files · 4 apps |
| Verification | Partial — CI not-reported (CI exists — github-actions — nothing ran on this commit) · tests changed in this PR · 7 source files | see full report |
| Rollout control | Ungated — The scan found zero FireWeave control points in the repository, and the new transport is selected unconditionally by a model-string prefix check with no flag i… | …/agent/compiler.py:429 |
| Cohort rollout | Not needed — no tier-3 path, no shared-surface blast, not a large tier-2 change | no plan on base |
| Observability | Partial — 3 of 7 changed source files emit a log or metric | see full report |
FireWeave · advisory, never blocks the check · claude-sonnet-5 · 4m 37s · Full report
🟡 Medium release risk — 47/100🟩🟩🟩🟨🟨⬜⬜⬜⬜⬜ higher = riskier · into local ·
Findings
Before merge
Release plan — Control point: none on the changed paths · Register in FireWeave · During release: Watch for empty/truncated answers or exceptions on KBs that
Blast radius · Services touched — OpenKB HTTP API, OpenKB Frontend reaches it · OpenAI Responses API, PageIndex depends
Score breakdown — 8 axes consideredLimited visibility — CI configured but nothing ran on this commit; model listed unknowns. Unscored is not safe.
FireWeave · advisory, never blocks the check · claude-sonnet-5 · 3m 29s · Full report |
This branch has not been deployed
2 participants
native OpenAI Responses API transport via openai-responses/ model prefix
Problem
Some gateways serve certain models exclusively on /v1/responses and answer /v1/chat/completions with 503 Endpoint is unavailable.
OpenKB talks to LLMs only through LiteLLM Chat Completions, so those models are completely unusable — compile, query, chat, lint, and
skills all fail.
Solution
Opt-in native Responses API transport. Prefixing the model id routes every agent call through the Responses API instead:
The prefix is OpenKB-only (stripped before anything reaches LiteLLM). Responses are adapted back to the Chat Completions shape, so all
existing call sites work unchanged — including tool calls and run_streamed sessions.
What's changed
OpenAIResponsesModel builder (per-KB creds, env fallback for CLI)
model; Chat Completions behavior untouched
Usage
Credentials work as usual (LLM_API_KEY + OPENAI_API_BASE, plus litellm.extra_headers if the gateway needs routing headers).
Limitations
untested
Testing