Skip to content

feat(sdk): record the provider's error code in error.type - #220

Merged
aikins01 merged 3 commits into
mainfrom
feat/sdk-provider-error-codes
Nov 10, 2026
Merged

aikins01 merged 3 commits into
mainfrom
feat/sdk-provider-error-codes

Conversation

@aikins01

@aikins01 aikins01 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

A failed model call now records the provider's error code in error.type and exception.type when its class name says nothing about the cause. Before, an exhausted openai quota and a rate limit were both RateLimitError, and an Anthropic stream that failed with overloaded_error, a Gemini 429, and an AI SDK RetryError gave generic class names. telemetry.dev groups failures and alert incidents by error type, so these failures could not be told apart.

The TypeScript SDKs and the Python SDK use the same rule, in errorTypeOf (packages/otel/src/errors.ts) and _error_type (sdks/python/src/telemetry_dev/_spans.py):

  • An error with an HTTP status from 400 to 599 and a code that is not only digits records the code, such as insufficient_quota or context_length_exceeded on an openai error.
  • A generic provider error class records its code, else its HTTP status as a string (429), else the provider's error type (overloaded_error). The generic classes are listed in docs/sdk-conformance.md. In Python, a boto ClientError gives its response.Error.Code.
  • The AI SDK's AI_RetryError records the type of the last error that it retried.
  • Any other error records its class name, as before.

Numeric codes, such as Z.ai's "1302", are not used, because a number names no cause and the class (RateLimitError) does. A getter on a third-party error that throws gives the fallback type and does not break the span.

The core SDKs, the AI SDK v6 and v7 integrations (also the media and batch spans, which set no error.type before), and TanStack AI call the shared function, so integrations that end spans through the core SDK, such as LangChain, get the rule too.

Failures that a provider reports in a response body or a stream event, not as an HTTP error, recorded Error or RuntimeError. The integrations now build them with the provider's code and a type that names the failure:

Failure error.type
openai or OpenRouter response.failed, stream error event, openai Realtime error the code, such as rate_limit_exceeded, else response.failed or response.error
OpenRouter numeric code the code as an HTTP status, such as 502
LiteLLM Responses failure the code, else response.failed
failed Google video operation the google.rpc.Code name, such as RESOURCE_EXHAUSTED
Bedrock stream exception in Python ModelStreamErrorException, as in TypeScript, in place of EventStreamError

A TypeScript openai image stream that ends with a failed event now marks its span failed, as the Python SDK does.

After the release, an alert that groups incidents by error type sees the new values as new error types one time, for example insufficient_quota in place of RateLimitError.

Tests through the real provider clients with a fake network: openai (TypeScript and Python) for insufficient_quota, context_length_exceeded, a 500 with no code, and a numeric code; Anthropic (TypeScript and Python) for a stream that fails with overloaded_error; Google GenAI (TypeScript 400, Python 429); and the AI SDK v7 for RetryError, an APICallError 529, and a stream provider error. The existing tests of each body or stream failure now check the code, and new tests cover a failed image stream, a failed Realtime response, and a failed Google video operation in TypeScript.

I ran vp test run in the changed packages (openai also with OPENAI_SDK_VERSION=6), vp check, and pytest, ruff, and pyright in each Python package.

A failed model call recorded its error class, which often names no cause:
openai's RateLimitError is also an exhausted quota, Anthropic's APIError
from a stream's error event, @google/genai's ApiError, and the AI SDK's
APICallError and RetryError say nothing about why the call failed.

Both SDKs now record, in error.type and exception.type:

- the error's code when it has an HTTP status and a code that is not
  only digits (insufficient_quota, context_length_exceeded);
- for a generic provider error class, its code, else its HTTP status
  (429), else the provider's error type (overloaded_error);
- for the AI SDK's RetryError, the type of the last error it retried;
- otherwise the class name, as before.

The core SDK, the AI SDK and TanStack AI integrations, and the Python
SDK share the rule. The comments and the conformance doc no longer say a
propagated gen_ai.conversation.id makes the ingest guess "llm"; the
ingest stops doing that.
@aikins01 aikins01 self-assigned this Nov 10, 2026
A failure that a provider reports in a response body or a stream event,
not as an HTTP error, recorded error.type Error or RuntimeError. This
covered openai and OpenRouter response.failed and stream error events,
OpenRouter chat stream errors, openai Realtime errors and failed image
streams, LiteLLM Responses failures, and failed Google video operations.

The integrations now build these failures with the provider's code and a
type that names the failure, so error.type is the code (such as
rate_limit_exceeded), an OpenRouter numeric code as an HTTP status
(502), the google.rpc.Code name of a video operation (RESOURCE_EXHAUSTED),
or else the failure's kind (response.failed). A TypeScript image stream
that ends with a failed event now marks its span failed, as in Python.

The AI SDK media and batch spans now set error.type with the same rule.
In Python, a Bedrock stream's EventStreamError records the exception
name, ModelStreamErrorException, as the TypeScript SDK does.
@aikins01
aikins01 merged commit 853e4aa into main Nov 10, 2026
10 checks passed
@aikins01
aikins01 deleted the feat/sdk-provider-error-codes branch November 10, 2026 21:16
@aikins01 aikins01 mentioned this pull request Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant