Skip to content

feat(operators): serve the Claude tiers through the Anthropic API in local development - #2017

Merged
jfrench9 merged 1 commit into
mainfrom
feature/operator-anthropic-provider
Nov 11, 2026
Merged

jfrench9 merged 1 commit into
mainfrom
feature/operator-anthropic-provider

Conversation

@jfrench9

Copy link
Copy Markdown
Member

Summary

A developer can now run the operators' Claude tiers on their own Anthropic API key instead of Bedrock. It is off by default and for local development only.

Changes

  • config/operators.py: a third provider, ANTHROPIC. With ANTHROPIC_API_ENABLED on, every Claude row keeps its short name, request fields and rate-card key, and changes only its wire id (the short name itself) and provider. The registry refuses the flag when ENVIRONMENT is not dev or test, and refuses it without a key.
  • config/env.py: ANTHROPIC_API_ENABLED and ANTHROPIC_API_KEY.
  • operations/operators/anthropic_api.py (new): translates Converse blocks to the Messages API and back, covering tools, tool results, cache markers, effort and thinking blocks. Retries only refusals that precede a generation; a timeout is not re-sent.
  • operations/operators/ai_client.py: routes a row served by the Anthropic API to the new client.
  • pyproject.toml, uv.lock: anthropic moves from the dev extra to the main dependencies, so it ships in the image. No version changed in the lock.
  • .env.example, docs/technical/Local-Development.md, config/README.md: describe the route.

Worth a close look: the switch is all or nothing. With it on, every Claude row is served by the Anthropic API, and a Bedrock wire id passed literally no longer resolves.

Breaking Changes

None. With the flag off the registry is unchanged.

Testing

  • just test-all: 18,708 passed, 44 skipped, 2 xfailed; lint, format and type checks clean.
  • New tests: the adapter in both directions, retry and refusal handling, routing, and the registry's flag, environment and key checks.
  • Live calls through the adapter against the Anthropic API: a two-turn tool call on Haiku, Sonnet and Opus 5.5, and a Sonnet run whose thinking block was replayed and accepted.
  • Not run: an operator inside the rebuilt local stack with the flag on, and a live prompt-cache read (the test prompts were below the cacheable size).

Certification

  • I have the right to submit this work under the Apache 2.0 license, and do so. Where any part of it is owned by my employer, I have their permission.

🤖 Generated with Claude Code

…local development

ANTHROPIC_API_ENABLED with a key runs every Claude row on the Anthropic API
instead of Bedrock, translating Converse blocks at the edge as the self-hosted
route does. The registry refuses it outside dev and test. The anthropic SDK
moves from the dev extra to the main dependencies.
@github-actions github-actions Bot added the change:normal Normal change: automated review documented on the pull request before merge label Nov 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Change classification: normal — platform or application code changed.

A normal change carries a documented automated review on this pull request before merge, in place of a second human approver. The change-classification job requests that review and passes once it has been posted. Recorded by the change-classification job.

@jfrench9

Copy link
Copy Markdown
Member Author

@claude please review this PR

@claude

claude Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Claude finished @jfrench9's task in 27s —— View job


Review of #2017: Anthropic API provider for local development

Verdict: looks good to merge. I found no blocking issues. The gating is tight and the adapter is clear. I read the diff of the source, config and dependency changes. I did not read tests/operations/test_anthropic_api.py or tests/config/test_operators.py, and I did not run anything.

  • Read the diff
  • Check config gating, adapter translation and routing
  • Post the review

What I checked

  • Gating (config/operators.py): with the flag off, anthropic_api_models returns an unchanged copy of _BEDROCK_MODELS. With it on, the registry refuses any ENVIRONMENT other than dev or test, and refuses an empty key. The refusal sits in the registry, so it runs at import and fails the boot. replace(spec, ...) keeps the request fields and rate-card keys, so the meter keeps working.
  • Key handling (config/env.py): ANTHROPIC_API_KEY is read only when the flag is on, so a stray key is inert when it is off. It goes through the centralized env config, as the repo requires.
  • Routing (ai_client.py): the Anthropic client is built lazily. A row served by the Anthropic API while the flag is off raises AIProviderError instead of falling through to Bedrock. The import in anthropic_api.py is circular but safe, because ai_client imports the module lazily and under TYPE_CHECKING.
  • Retry policy: only 429, 503 and 529 are retried, with max_retries=0 on the SDK. A timeout is surfaced and not re-sent. This matches the stated billing rationale.
  • Dependency move: anthropic is now a main dependency, which the lock reflects.

Minor suggestions (non-blocking)

  1. Tool results drop non-text parts. In to_messages_content (anthropic_api.py), a toolResult keeps only {"text": ...} parts. A Converse {"json": ...} part would be silently dropped, so the model would see an empty result. If no tool in the codebase returns json parts, a comment saying so would help. Otherwise serialize them to text.
  2. Cache marker on a non-cacheable last block. build_messages_request puts cache_control on the last block of the last message. If that block were a thinking or redacted_thinking block, the API rejects it. Last messages are normally user turns, so this is unlikely, but a guard that skips those types is cheap.
  3. Empty content after translation. A message made only of cachePoint blocks, or only of unsupported blocks, becomes content: [], and the API returns a 400. This is an edge case. Dropping such messages, or raising a clearer error, would be friendlier.
  4. Retry backoff. asyncio.sleep(2**attempt) gives 2s and then 4s. It ignores the retry-after header on a 429, so a real rate limit may outlast all three attempts. Reading e.response.headers.get("retry-after") would be more accurate.
  5. Connection-error message. The APIConnectionError text says the call "failed before reaching the model". A connection can also drop mid-response, so "failed at the connection" is safer wording.
  6. A client per call. This is a sound choice for the event-loop reason in the comment, and it costs a new TLS handshake per call. That is fine for local development.

Things to confirm

  • Direct Bedrock callers. The PR says a literal Bedrock wire id no longer resolves when the flag is on. Check that nothing outside AIClient, such as embeddings, search or other direct bedrock-runtime calls, depends on a Claude Bedrock id. I did not search for these.
  • Local stack and cache read. As the PR body notes, neither an operator in the rebuilt local stack with the flag on nor a live prompt-cache read has been run. Both are worth one manual pass before relying on this route.

@jfrench9
jfrench9 merged commit efa4551 into main Nov 11, 2026
10 checks passed
@jfrench9
jfrench9 deleted the feature/operator-anthropic-provider branch November 11, 2026 05:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change:normal Normal change: automated review documented on the pull request before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant