Repository navigation
feat: add UberLLM provider - #137
PSA-Team-source wants to merge 2 commits into
Conversation
|
🔍 OpenCodeReview found 7 issue(s) in this PR.
|
| label: 'UberLLM (OpenAI-compatible)', | ||
| local: false, | ||
| creds: [ | ||
| { key: 'UBERLLM_API_KEY', label: 'UberLLM API key', secret: true }, |
There was a problem hiding this comment.
The API key environment variable name 'UBERLLM_API_KEY' follows an inconsistent naming convention compared to other providers in this file. Most providers use the provider name directly (e.g., 'OPENAI_API_KEY', 'AZURE_OPENAI_API_KEY'). Consider renaming to 'UBERLLM_API_KEY' to match the provider key 'uberllm'.
Suggestion:
| { key: 'UBERLLM_API_KEY', label: 'UberLLM API key', secret: true }, | |
| { key: 'UBERLLM_API_KEY', label: 'UberLLM API key', secret: true }, |
| // UberLLM configuration (OpenAI-compatible Chat Completions) | ||
| const uberllmApiKey = process.env.UBERLLM_API_KEY?.trim() || null; | ||
| const uberllmModel = process.env.UBERLLM_MODEL?.trim() || "deepseek/deepseek-v4.1-flash"; | ||
| const uberllmEndpoint = process.env.UBERLLM_ENDPOINT?.trim() || "https://api.uberllm.dev/v1/chat/completions"; |
There was a problem hiding this comment.
The UberLLM endpoint URL is duplicated in multiple files (config/index.js, providers-handler.js, databricks.js, orchestrator/index.js). This creates maintenance risk if the endpoint changes. Consider extracting this to a single constant in config/index.js and importing it where needed.
Suggestion:
| const uberllmEndpoint = process.env.UBERLLM_ENDPOINT?.trim() || "https://api.uberllm.dev/v1/chat/completions"; | |
| // In config/index.js (line ~146) | |
| const UBERLLM_DEFAULT_ENDPOINT = "https://api.uberllm.dev/v1/chat/completions"; | |
| const uberllmEndpoint = process.env.UBERLLM_ENDPOINT?.trim() || UBERLLM_DEFAULT_ENDPOINT; | |
| // In other files, import and use: | |
| const { UBERLLM_DEFAULT_ENDPOINT } = require('../config'); | |
| const endpoint = config.uberllm.endpoint || UBERLLM_DEFAULT_ENDPOINT; |
| providers.push({ | ||
| name: "uberllm", | ||
| type: "uberllm", | ||
| baseUrl: config.uberllm.endpoint || "https://api.uberllm.dev/v1/chat/completions", |
There was a problem hiding this comment.
The UberLLM provider registration lacks URL format validation for the endpoint. This differs from other providers like openscale which use the same pattern, but the validation may be deferred to the invoke* function.
Suggestion: If URL validation is expected at the provider registration level, consider adding it here for consistency. Otherwise, ensure the invokeUberLLM function properly handles invalid endpoint URLs.
| convertAnthropicMessagesToOpenRouter | ||
| } = require("./openrouter-utils"); | ||
|
|
||
| const endpoint = config.uberllm.endpoint || "https://api.uberllm.dev/v1/chat/completions"; |
There was a problem hiding this comment.
The invokeUberLLM function uses hardcoded fallback values for both endpoint and model. If the config is missing these values, requests will default to the hardcoded endpoint and model, which may lead to unexpected billing or behavior.
Suggestion: Add explicit validation to ensure the API key is configured, and consider requiring the endpoint and model to be explicitly configured before enabling the provider (similar validation exists in other invoke* functions).
Suggestion:
| const endpoint = config.uberllm.endpoint || "https://api.uberllm.dev/v1/chat/completions"; | |
| const endpoint = config.uberllm?.endpoint; | |
| if (!endpoint) { | |
| throw new Error("UberLLM endpoint is not configured."); | |
| } |
| }, "UberLLM"); | ||
|
|
||
| if (!response.ok && response.status === 429) { | ||
| const err = new Error(`UberLLM rate-limited: ${String(response.json?.error?.message || '').slice(0, 120)}`); |
There was a problem hiding this comment.
The response.json?.error?.message access could be null if performJsonRequest fails to parse the response body. The error message will then be an empty string, which may not be informative for debugging.
Suggestion: Add a fallback that uses the raw response text when JSON parsing fails.
Suggestion:
| const err = new Error(`UberLLM rate-limited: ${String(response.json?.error?.message || '').slice(0, 120)}`); | |
| const err = new Error(`UberLLM rate-limited: ${String(response.json?.error?.message || response.text?.slice(0, 120) || '').slice(0, 120)}`); |
| ]; | ||
|
|
||
| const REASONING_CONTENT_PROVIDERS = new Set(["moonshot", "openrouter", "edenai", "openai", "azure-openai", "atlas", "cheaperinference", "atmorouter", "runcrate", "openscale", "orcarouter"]); | ||
| const REASONING_CONTENT_PROVIDERS = new Set(["moonshot", "openrouter", "edenai", "openai", "azure-openai", "atlas", "cheaperinference", "atmorouter", "runcrate", "openscale", "uberllm", "orcarouter"]); |
There was a problem hiding this comment.
The uberllm provider is added to REASONING_CONTENT_PROVIDERS but lacks corresponding entries in other capability sets like NATIVE_THINKING_PROVIDERS. This may lead to inconsistent feature detection for the provider.
Suggestion: Review other provider registrations (e.g., openrouter, openai) to ensure consistent capability coverage, and add uberllm to any missing capability sets that it supports.
| const localProviders = ["ollama", "llamacpp", "lmstudio"]; | ||
| if (localProviders.includes(fallbackProvider)) { | ||
| throw new Error(`FALLBACK_PROVIDER cannot be '${fallbackProvider}' (local providers should not be fallbacks). Use cloud providers: databricks, azure-anthropic, azure-openai, openrouter, edenai, openai, atlas, cheaperinference, atmorouter, runcrate, openscale, bedrock, orcarouter`); | ||
| throw new Error(`FALLBACK_PROVIDER cannot be '${fallbackProvider}' (local providers should not be fallbacks). Use cloud providers: databricks, azure-anthropic, azure-openai, openrouter, edenai, openai, atlas, cheaperinference, atmorouter, runcrate, openscale, uberllm, bedrock, orcarouter`); |
There was a problem hiding this comment.
The fallback error message at line 458 hardcodes a list of valid cloud providers. This list is duplicated here and in the SUPPORTED_MODEL_PROVIDERS set at the top of the file. If the list of supported providers changes, this error message will quickly become outdated, leading to user confusion. Consider extracting this list into a constant or deriving it dynamically from the set of supported providers.
Suggestion:
| throw new Error(`FALLBACK_PROVIDER cannot be '${fallbackProvider}' (local providers should not be fallbacks). Use cloud providers: databricks, azure-anthropic, azure-openai, openrouter, edenai, openai, atlas, cheaperinference, atmorouter, runcrate, openscale, uberllm, bedrock, orcarouter`); | |
| const CLOUD_PROVIDERS = [...SUPPORTED_MODEL_PROVIDERS].filter(p => !['ollama', 'llamacpp', 'lmstudio'].includes(p)); | |
| // ... | |
| throw new Error(`FALLBACK_PROVIDER cannot be '${fallbackProvider}' (local providers should not be fallbacks). Use cloud providers: ${CLOUD_PROVIDERS.join(', ')}`); |
| const localProviders = ["ollama", "llamacpp", "lmstudio"]; | ||
| if (localProviders.includes(fallbackProvider)) { | ||
| throw new Error(`FALLBACK_PROVIDER cannot be '${fallbackProvider}' (local providers should not be fallbacks). Use cloud providers: databricks, azure-anthropic, azure-openai, openrouter, edenai, openai, atlas, cheaperinference, atmorouter, runcrate, openscale, bedrock, orcarouter`); | ||
| throw new Error(`FALLBACK_PROVIDER cannot be '${fallbackProvider}' (local providers should not be fallbacks). Use cloud providers: databricks, azure-anthropic, azure-openai, openrouter, edenai, openai, atlas, cheaperinference, atmorouter, runcrate, openscale, uberllm, bedrock, orcarouter`); |
There was a problem hiding this comment.
In src/config/index.js, the fallback provider error message hardcodes a list of valid cloud providers (line 458) that duplicates the SUPPORTED_MODEL_PROVIDERS set at the top of the file. This duplication means if the provider list changes, this error message may become outdated. Consider extracting this list into a constant or deriving it dynamically from the supported providers set.
| uberllm: { | ||
| apiKey: uberllmApiKey, | ||
| model: uberllmModel, | ||
| endpoint: uberllmEndpoint, | ||
| }, |
There was a problem hiding this comment.
The configuration object structure for UberLLM follows the same pattern as other providers (apiKey, model, endpoint), which is good. However, there's no default value validation for the endpoint if it's explicitly set but malformed. Consider adding endpoint format validation similar to other critical configurations.
Summary
Adds UberLLM as a first-class provider (
MODEL_PROVIDER=uberllm). One UberLLM key reaches Claude, GPT, Gemini, DeepSeek, GLM, Kimi and Qwen models (~263 chat models, public list atGET https://api.uberllm.dev/v1/models, no key needed), so a Lynkr tier setup can put a cheap model onTIER_SIMPLEand a Claude model onTIER_REASONINGwith a single provider entry. Mirrors #133 and #135 file for file: same 15 files, same invoker shape, same tests.Changes
src/config/index.js:UBERLLM_API_KEY,UBERLLM_MODEL(defaultdeepseek/deepseek-v4.1-flash),UBERLLM_ENDPOINT(defaulthttps://api.uberllm.dev/v1/chat/completions); added toSUPPORTED_MODEL_PROVIDERS, the primary key check, the fallback checks andreloadConfig().src/clients/databricks.js:invokeUberLLM()with the same Anthropic to Chat Completions conversion, tool handling, no replay of billable POSTs and 429 escalation asinvokeOpenScale(); registered inPROVIDER_INVOKERSand exported.src/routing/index.js,src/routing/model-tiers.js,src/orchestrator/index.js,src/orchestrator/sse-transformer.js,src/clients/provider-capabilities.js,src/api/providers-handler.js: same entries as the other OpenAI-compatible providers.bin/lynkr-init.js: UberLLM in the init wizard andPROVIDER_ORDER..env.example,README.md,documentation/providers.md(section 8f): configuration and tier examples.test/uberllm-integration.test.js(mirrorstest/openscale-integration.test.js),test/dispatch-registry.test.js, and the new test added totest:unitinpackage.json.UberLLM also serves an Anthropic-compatible
/v1/messages, but this provider uses the Chat Completions path like the other gateways, so it goes through Lynkr's existing tool conversion and tier routing.Testing
node --test test/uberllm-integration.test.js test/openscale-integration.test.js test/runcrate-integration.test.js test/dispatch-registry.test.js(with thetest:unitenv vars): 23 tests, 23 pass, 0 fail.npx eslinton the 9 touched files undersrc/andbin/plus the new test: no errors.npm run test:unitlocally.https://api.uberllm.dev/v1/chat/completions, callinginvokeUberLLM()from this branch withMODEL_PROVIDER=uberllmandmax_tokens: 30:pong,finish_reason: stop.get_weathertool: HTTP 200, oneget_weathertool call (arguments cut off by the 30-token cap).Tested with deepseek/deepseek-v4.1-flash: replied "pong" (HTTP 200, finish_reason stop) and emitted a get_weather tool call through invokeUberLLM().
Per-model prices: https://uberllm.dev/models. Keys: https://uberllm.dev/signup.
This PR was prepared with AI assistance: an AI coding agent wrote the change and ran the checks above.
I work on UberLLM.