Skip to content

feat: route hosted model calls through the metering gateway - #253

Open
jefft72 wants to merge 1 commit into
codex/token-pr-companion-01-depsfrom
codex/token-pr-companion-02-hosted-gateway
Open

jefft72 wants to merge 1 commit into
codex/token-pr-companion-01-depsfrom
codex/token-pr-companion-02-hosted-gateway

Conversation

@jefft72

@jefft72 jefft72 commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds an opt-in hosted model transport that routes every generation through the authenticated cloud gateway and prevents automatic replacement requests after ambiguous failures.

Reasoning

Pricing cannot be authoritative if compaction, summaries, tool turns, or child agents can bypass the metered transport.

One constrained Anthropic runtime and a private, durably persisted operation ID cover those paths while suppressing retries that could create duplicate billable generations.

The native server starts requiring this transport in the following PR; deployment depends on the matching cloud gateway implementation.

Changes

  • Adds a gateway transport with fixed endpoint, model, credential, and output-cap validation.
  • Persists an operation UUID and request digest before sending, retaining unresolved evidence without storing prompts or credentials.
  • Routes parent, tool-loop, compaction, summary, and child-agent calls through the same transport.
  • Prohibits custom provider, credential, and transport overrides in hosted mode.
  • Propagates a private non-retryable policy through provider retries and automatic compaction, preserving completed output when metadata cleanup fails.
  • Adds regressions for request identity, durable evidence, transport restrictions, restart, and the actual agent call paths.

Testing

  • npm run check
  • npm run build:offline
  • From packages/coding-agent: node ../../node_modules/vitest/dist/cli.js --run test/hosted-model-gateway.test.ts
  • From packages/ai: node ../../node_modules/vitest/dist/cli.js --run test/together-models.test.ts test/generate-models-strict.test.ts: 7 tests passed after incorporating the catalog prerequisite.
  • node --test scripts/vendored-braces-security.test.mjs: 11 tests passed against the inherited dependency patch.
  • Reviewed hosted transport, retry policy, and protocol omission boundaries locally.

@jefft72 jefft72 added enhancement New feature or request coding-agent labels Oct 2, 2026
@jefft72 jefft72 assigned Issacboy123 and Pepps233 and unassigned Issacboy123 Oct 2, 2026
@Pepps233
Pepps233 self-requested a review October 4, 2026 18:02
@Pepps233 Pepps233 assigned jefft72 and unassigned Pepps233 Oct 4, 2026
@Pepps233
Pepps233 added this pull request to stack #256 October 4, 2026 18:34
Route hosted turns, tool loops, compaction, summaries and child agents through one constrained Anthropic gateway. Persist private operation identity and the request digest before sending, reject transport and credential overrides, and suppress replacement generations after ambiguous failures so metering cannot be bypassed or duplicated.
@Pepps233
Pepps233 force-pushed the codex/token-pr-companion-02-hosted-gateway branch from 424026f to f1b79c8 Compare October 4, 2026 18:35

@Pepps233 Pepps233 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sure routing requests through our cloud gateway is not enabled by default. it should only be enabled if the user chooses to auth with us. this pr is approved, disregard the request changes flag

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

coding-agent enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants