Repository navigation
Conversation
Contributor License AgreementThe following contributors need CLA coverage: |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Devin Review found 1 potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| requiredEnvVars: ['YOU_API_KEY'], | ||
| host: 'api.you.com', | ||
| modelsPath: '/v1/models', | ||
| modelListFormat: 'openai', |
There was a problem hiding this comment.
🟡 You.com context limits disappear from index
When You.com lists models with max_input_tokens, modelListFormat: 'openai' produces entries without context limits. extractContextLength ignores that field, so the model index omits those limits.
Learn more
The model-index benchmark fetches each configured gateway's model list and normalizes entries into shared fields. You.com's model objects expose max_input_tokens for their input limit, but extractContextLength recognizes only four other field names. With modelListFormat: 'openai', normalizeModels calls normalizeModel and stores contextLength as undefined for these entries. The index omits a limit available in the response.
Example: A listed You.com model with { "id": "kimi-k3", "max_input_tokens": 128000 } becomes an index entry with no contextLength instead of 128000.
Recommended fix: Teach extractContextLength to read max_input_tokens, and validate the normalized You.com model list against an example response.
Was this helpful? React with 👍 or 👎 to provide feedback.
Add You.com Inference Router to AI gateway benchmarks
Adds
you-dot-com(https://you.com/docs/guides/inference-router/quickstart) as an AI gateway participant.What's included
claude-sonnet-5/v1/messages+anthropic-versiongpt-5.6-luna/v1/chat/completions(router has no Responses API)kimi-k3/v1/chat/completionsGET /v1/modelsNot added to the Gemini family: the router's catalog has no Gemini model.
Model change for the Anthropic and OpenAI families
The router doesn't serve older model: Claude Haiku 4.5 or
gpt-5.4-mini, so to keep every participant on the same model:gpt-5.4-mini→gpt-5.6-lunaExisting results in
results/ai-gateway-latency/anthropic/and.../openai/are for the old models, so trend lines across this change won't be like-for-like.Other changes
YOU_API_KEYadded to.env.example,provider-vars.json, and the AI gateway workflow's vault-secret patternyou-dot-comadded to the workflow provider choices, PR-comment labels, SVG labels, andbench:*:you-dot-comscriptsx-ydc-inference-timing-*response headers are captured in each iteration'sreceiptsAI_GATEWAYS.mdupdated for the model change, plus a You.com sectionTesting
Ran
you-dot-comalongsideanthropic-directandopenai-directlocally with--iterations 2 --dry-run. All requests succeeded on the Anthropic, OpenAI and Kimi routes. No results are committed in this PR.For maintainers
YOU_API_KEYto the vaultworkflow_dispatch, familyanthropic,openai,kimi) to produce results with the full roster. This also checks the other gateways' new Sonnet 5 / gpt-5.6-luna model ids, which haven't been run yet.