Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,16 @@ OPENSCALE_MODEL=deepseek/deepseek-v4.1-flash
# DESCRIPTION: OpenScale Chat Completions endpoint.
OPENSCALE_ENDPOINT=https://api.openscale.so/v1/chat/completions

# ------------------------------------------------------------------------------
# UberLLM (OpenAI-compatible)
# ------------------------------------------------------------------------------
# DESCRIPTION: UberLLM API key from https://uberllm.dev/signup.
# UBERLLM_API_KEY=your-uberllm-api-key
# DESCRIPTION: Default UberLLM text model.
UBERLLM_MODEL=deepseek/deepseek-v4.1-flash
# DESCRIPTION: UberLLM Chat Completions endpoint.
UBERLLM_ENDPOINT=https://api.uberllm.dev/v1/chat/completions

# ------------------------------------------------------------------------------
# Ollama (local models)
# ------------------------------------------------------------------------------
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,7 @@ Claude Code / Cursor / Codex / Cline / Continue
| **AtmoRouter** | Cloud | Claude, GPT, Gemini, GLM, Kimi, Qwen, DeepSeek, and other OpenAI-compatible models | $-$$ |
| **Runcrate** | Cloud | DeepSeek, GLM, Kimi, Qwen, gpt-oss, and other open-weight models | $-$$ |
| **OpenScale** | Cloud | DeepSeek V4.1 Flash, GLM 5.3 Flash | $ |
| **UberLLM** | Cloud | Claude, GPT, Gemini, DeepSeek, GLM, Kimi, Qwen, and other OpenAI-compatible models | $-$$$ |
| **DeepSeek** | Cloud | DeepSeek R1, Reasoner | $ |
| **Z.ai** | Cloud | GLM-4.7, GLM-4.5-Air | $ |
| **Moonshot AI** | Cloud | Kimi K2.6, Kimi K3 | $ |
Expand Down
13 changes: 12 additions & 1 deletion bin/lynkr-init.js
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,17 @@ const PROVIDERS = {
],
defaultModel: 'deepseek/deepseek-v4.1-flash',
},
uberllm: {
label: 'UberLLM (OpenAI-compatible)',
local: false,
creds: [
{ key: 'UBERLLM_API_KEY', label: 'UberLLM API key', secret: true },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintainability · low
The API key environment variable name 'UBERLLM_API_KEY' follows an inconsistent naming convention compared to other providers in this file. Most providers use the provider name directly (e.g., 'OPENAI_API_KEY', 'AZURE_OPENAI_API_KEY'). Consider renaming to 'UBERLLM_API_KEY' to match the provider key 'uberllm'.

Suggestion:

Suggested change
{ key: 'UBERLLM_API_KEY', label: 'UberLLM API key', secret: true },
{ key: 'UBERLLM_API_KEY', label: 'UberLLM API key', secret: true },

],
extras: [
{ key: 'UBERLLM_ENDPOINT', label: 'Chat Completions endpoint', default: 'https://api.uberllm.dev/v1/chat/completions' },
],
defaultModel: 'deepseek/deepseek-v4.1-flash',
},
openrouter: {
label: 'OpenRouter (100+ models, one key)',
local: false,
Expand Down Expand Up @@ -250,7 +261,7 @@ const PROVIDERS = {

const PROVIDER_ORDER = [
'ollama', 'llamacpp', 'lmstudio',
'azure-anthropic', 'azure-openai', 'openai', 'atlas', 'cheaperinference', 'atmorouter', 'runcrate', 'openscale', 'openrouter', 'edenai',
'azure-anthropic', 'azure-openai', 'openai', 'atlas', 'cheaperinference', 'atmorouter', 'runcrate', 'openscale', 'uberllm', 'openrouter', 'edenai',
'databricks', 'bedrock', 'vertex', 'zai', 'moonshot', 'baidu', 'fireworks', 'orcarouter',
];
const TIERS = ['SIMPLE', 'MEDIUM', 'COMPLEX', 'REASONING'];
Expand Down
36 changes: 36 additions & 0 deletions documentation/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -970,6 +970,42 @@ TIER_REASONING=openscale:deepseek/deepseek-v4.1-flash

---

### 8f. UberLLM (OpenAI-Compatible)

**Best for:** One key for Claude, GPT, Gemini, DeepSeek, GLM, Kimi and Qwen models, so each Lynkr tier can use a different vendor without a second account

#### Configuration

```env
MODEL_PROVIDER=uberllm
UBERLLM_API_KEY=your-uberllm-api-key
UBERLLM_MODEL=deepseek/deepseek-v4.1-flash
UBERLLM_ENDPOINT=https://api.uberllm.dev/v1/chat/completions
```

Create a key at [uberllm.dev/signup](https://uberllm.dev/signup). The endpoint is OpenAI-compatible, so Lynkr converts Anthropic messages and tools to Chat Completions requests and transforms streamed OpenAI SSE events back to Anthropic events.

UberLLM can also be selected per tier:

```env
TIER_SIMPLE=uberllm:deepseek/deepseek-v4.1-flash
TIER_MEDIUM=uberllm:z-ai/glm-5.3-flash
TIER_COMPLEX=uberllm:z-ai/glm-5.3
TIER_REASONING=uberllm:anthropic/claude-sonnet-4.6
```

`UBERLLM_MODEL` is the static-routing default. A model named in `TIER_*` takes precedence for that tier. Model IDs carry the vendor prefix (`anthropic/`, `openai/`, `google/`, `deepseek/`, `z-ai/`, `moonshotai/`, `qwen/`); the full list is public at `GET https://api.uberllm.dev/v1/models` and per-model prices are at [uberllm.dev/models](https://uberllm.dev/models).

#### Behavior

- Supports non-streaming and streaming Chat Completions
- Supports Anthropic-format tools through Lynkr's existing OpenAI conversion path
- Relays model reasoning as `reasoning_content`
- Uses Bearer authentication and accepts a custom `UBERLLM_ENDPOINT`
- Does not automatically replay billable Chat Completions POST requests

---

### 9. LM Studio (Local with GUI)

**Best for:** Local models with graphical interface
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
"start:supervised": "while true; do node index.js 2>&1 | npx pino-pretty --sync; code=$?; echo \"[supervisor] lynkr exited ($code) — restarting in 3s\"; sleep 3; done",
"lint": "eslint src index.js",
"test": "npm run test:unit && npm run test:performance",
"test:unit": "LYNKR_KNN_DIR=/tmp/lynkr-test-knn DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com LOG_FILE_ENABLED=false node --test test/routing.test.js test/hybrid-routing-integration.test.js test/retry-logic.test.js test/sse-transformer.test.js test/passthrough-stream.test.js test/passthrough-mode.test.js test/openrouter-error-resilience.test.js test/format-conversion.test.js test/azure-openai-config.test.js test/azure-openai-format-conversion.test.js test/azure-openai-routing.test.js test/azure-openai-streaming.test.js test/azure-openai-error-resilience.test.js test/azure-openai-integration.test.js test/openai-integration.test.js test/atlas-integration.test.js test/cheaperinference-integration.test.js test/atmorouter-integration.test.js test/runcrate-integration.test.js test/openscale-integration.test.js test/toon-compression.test.js test/gcf-compression.test.js test/llamacpp-integration.test.js test/resilience.test.js test/telemetry-routing.test.js test/memory/store.test.js test/memory/surprise.test.js test/memory/extractor.test.js test/memory/search.test.js test/memory/retriever.test.js test/memory/distiller.test.js test/memory/distiller-freeze.test.js test/memory/wiki.test.js test/memory/skills-cache.test.js test/memory/tencentdb-launcher.test.js test/distill.test.js test/large-payload.test.js test/prompt-cache-injection.test.js test/risk-analyzer.test.js test/interaction-block.test.js test/preflight.test.js test/token-reduction.test.js test/session-affinity.test.js test/cache-state.test.js test/cache-switch-cost.test.js test/lens-recommendations.test.js test/model-registry-cost.test.js test/output-format-guard.test.js test/tier-fallback.test.js test/wrap.test.js test/init.test.js test/tool-call-response-metadata.test.js test/degradation.test.js test/routing-telemetry-columns.test.js test/sticky-routing.test.js test/knn-ambiguous-escalate.test.js test/deescalator.test.js test/client-profiles.test.js test/openai-ingress-tool-injection.test.js test/tool-result-compression-gate.test.js test/strip-internal-fields.test.js test/complexity-tool-subtraction.test.js test/bandit.test.js test/routing-propensity.test.js test/reward-pipeline.test.js test/knn-cold-start.test.js test/calibration.test.js test/feedback-loop.test.js test/session-fingerprint.test.js test/side-request-guards.test.js test/verifier.test.js test/intent-score.test.js test/difficulty-classifier.test.js test/classifier-setup.test.js test/usage-stats.test.js test/loop-guard.test.js test/moonshot-model-mapping.test.js test/baidu-model-mapping.test.js test/tenant-policy-ingress-parity.test.js test/decide.test.js test/embeddings-degradation.test.js test/health-probe.test.js test/stuck-detector.test.js test/onnx-embedder.test.js test/ope.test.js test/hierarchical-budget.test.js test/token-rate-limit.test.js test/otel-export.test.js test/mcp-broker.test.js test/compression-budget.test.js test/gpt-utils.test.js test/dedup-observe-only.test.js test/context-window-header.test.js test/token-budget-auto.test.js test/opencode-setup.test.js test/auth-mode-first-party.test.js test/harness-envelope.test.js test/task-ledger.test.js test/jev-router.test.js test/jev-routing.test.js test/force-patterns.test.js test/upstream-fidelity.test.js test/tool-schema-compression.test.js test/tool-strict-preservation.test.js test/passthrough-route.test.js test/quota-ledger.test.js test/shortfall-semantic.test.js test/routing-decisions.test.js test/routing-corpus.test.js test/switch-gate-grounding.test.js",
"test:unit": "LYNKR_KNN_DIR=/tmp/lynkr-test-knn DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com LOG_FILE_ENABLED=false node --test test/routing.test.js test/hybrid-routing-integration.test.js test/retry-logic.test.js test/sse-transformer.test.js test/passthrough-stream.test.js test/passthrough-mode.test.js test/openrouter-error-resilience.test.js test/format-conversion.test.js test/azure-openai-config.test.js test/azure-openai-format-conversion.test.js test/azure-openai-routing.test.js test/azure-openai-streaming.test.js test/azure-openai-error-resilience.test.js test/azure-openai-integration.test.js test/openai-integration.test.js test/atlas-integration.test.js test/cheaperinference-integration.test.js test/atmorouter-integration.test.js test/runcrate-integration.test.js test/openscale-integration.test.js test/uberllm-integration.test.js test/toon-compression.test.js test/gcf-compression.test.js test/llamacpp-integration.test.js test/resilience.test.js test/telemetry-routing.test.js test/memory/store.test.js test/memory/surprise.test.js test/memory/extractor.test.js test/memory/search.test.js test/memory/retriever.test.js test/memory/distiller.test.js test/memory/distiller-freeze.test.js test/memory/wiki.test.js test/memory/skills-cache.test.js test/memory/tencentdb-launcher.test.js test/distill.test.js test/large-payload.test.js test/prompt-cache-injection.test.js test/risk-analyzer.test.js test/interaction-block.test.js test/preflight.test.js test/token-reduction.test.js test/session-affinity.test.js test/cache-state.test.js test/cache-switch-cost.test.js test/lens-recommendations.test.js test/model-registry-cost.test.js test/output-format-guard.test.js test/tier-fallback.test.js test/wrap.test.js test/init.test.js test/tool-call-response-metadata.test.js test/degradation.test.js test/routing-telemetry-columns.test.js test/sticky-routing.test.js test/knn-ambiguous-escalate.test.js test/deescalator.test.js test/client-profiles.test.js test/openai-ingress-tool-injection.test.js test/tool-result-compression-gate.test.js test/strip-internal-fields.test.js test/complexity-tool-subtraction.test.js test/bandit.test.js test/routing-propensity.test.js test/reward-pipeline.test.js test/knn-cold-start.test.js test/calibration.test.js test/feedback-loop.test.js test/session-fingerprint.test.js test/side-request-guards.test.js test/verifier.test.js test/intent-score.test.js test/difficulty-classifier.test.js test/classifier-setup.test.js test/usage-stats.test.js test/loop-guard.test.js test/moonshot-model-mapping.test.js test/baidu-model-mapping.test.js test/tenant-policy-ingress-parity.test.js test/decide.test.js test/embeddings-degradation.test.js test/health-probe.test.js test/stuck-detector.test.js test/onnx-embedder.test.js test/ope.test.js test/hierarchical-budget.test.js test/token-rate-limit.test.js test/otel-export.test.js test/mcp-broker.test.js test/compression-budget.test.js test/gpt-utils.test.js test/dedup-observe-only.test.js test/context-window-header.test.js test/token-budget-auto.test.js test/opencode-setup.test.js test/auth-mode-first-party.test.js test/harness-envelope.test.js test/task-ledger.test.js test/jev-router.test.js test/jev-routing.test.js test/force-patterns.test.js test/upstream-fidelity.test.js test/tool-schema-compression.test.js test/passthrough-route.test.js test/quota-ledger.test.js test/shortfall-semantic.test.js test/routing-decisions.test.js test/routing-corpus.test.js test/switch-gate-grounding.test.js",
"test:memory": "LYNKR_KNN_DIR=/tmp/lynkr-test-knn DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node --test test/memory/store.test.js test/memory/surprise.test.js test/memory/extractor.test.js test/memory/search.test.js test/memory/retriever.test.js test/memory/distiller.test.js test/memory/distiller-freeze.test.js test/memory/wiki.test.js test/memory/skills-cache.test.js test/memory/tencentdb-launcher.test.js",
"test:new-features": "LYNKR_KNN_DIR=/tmp/lynkr-test-knn DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node --test test/passthrough-mode.test.js test/openrouter-error-resilience.test.js test/format-conversion.test.js",
"test:performance": "LYNKR_KNN_DIR=/tmp/lynkr-test-knn DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node test/hybrid-routing-performance.test.js && DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node test/performance-tests.js",
Expand Down
13 changes: 13 additions & 0 deletions src/api/providers-handler.js
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,19 @@ async function getConfiguredProviders() {
});
}

// Check UberLLM (OpenAI-compatible Chat Completions)
if (config.uberllm?.apiKey) {
providers.push({
name: "uberllm",
type: "uberllm",
baseUrl: config.uberllm.endpoint || "https://api.uberllm.dev/v1/chat/completions",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintainability · low
The UberLLM provider registration lacks URL format validation for the endpoint. This differs from other providers like openscale which use the same pattern, but the validation may be deferred to the invoke* function.

Suggestion: If URL validation is expected at the provider registration level, consider adding it here for consistency. Otherwise, ensure the invokeUberLLM function properly handles invalid endpoint URLs.

enabled: true,
models: [
{ id: config.uberllm.model || "deepseek/deepseek-v4.1-flash", name: "Configured Model" },
]
});
}

// Check OpenRouter
if (config.openrouter?.apiKey) {
providers.push({
Expand Down
68 changes: 68 additions & 0 deletions src/clients/databricks.js
Original file line number Diff line number Diff line change
Expand Up @@ -1998,6 +1998,72 @@ async function invokeOpenScale(body) {
return response;
}

async function invokeUberLLM(body) {
if (!config.uberllm?.apiKey) {
throw new Error("UberLLM API key is not configured.");
}

const {
convertAnthropicToolsToOpenRouter,
convertAnthropicMessagesToOpenRouter
} = require("./openrouter-utils");

const endpoint = config.uberllm.endpoint || "https://api.uberllm.dev/v1/chat/completions";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintainability · medium
The invokeUberLLM function uses hardcoded fallback values for both endpoint and model. If the config is missing these values, requests will default to the hardcoded endpoint and model, which may lead to unexpected billing or behavior.

Suggestion: Add explicit validation to ensure the API key is configured, and consider requiring the endpoint and model to be explicitly configured before enabling the provider (similar validation exists in other invoke* functions).

Suggestion:

Suggested change
const endpoint = config.uberllm.endpoint || "https://api.uberllm.dev/v1/chat/completions";
const endpoint = config.uberllm?.endpoint;
if (!endpoint) {
throw new Error("UberLLM endpoint is not configured.");
}

const headers = {
"Authorization": `Bearer ${config.uberllm.apiKey}`,
"Content-Type": "application/json",
};

const messages = convertAnthropicMessagesToOpenRouter(body.messages || []);
if (body.system) {
messages.unshift({ role: "system", content: body.system });
}

const uberllmBody = {
model: body._suggestionModeModel || body._tierModel || config.uberllm.model || "deepseek/deepseek-v4.1-flash",
messages,
temperature: body.temperature ?? 0.7,
max_tokens: body.max_tokens ?? 16384,
top_p: body.top_p ?? 1.0,
stream: body.stream ?? false,
};

const toolsToSend = Array.isArray(body.tools) ? body.tools : [];

if (toolsToSend.length > 0) {
uberllmBody.tools = convertAnthropicToolsToOpenRouter(toolsToSend);
uberllmBody.parallel_tool_calls = false;
uberllmBody.tool_choice = forwardToolChoice(body);
}

logger.debug({
endpoint,
model: uberllmBody.model,
hasTools: !!uberllmBody.tools,
toolCount: uberllmBody.tools?.length || 0,
temperature: uberllmBody.temperature,
max_tokens: uberllmBody.max_tokens,
}, "=== UBERLLM REQUEST ===");

// Chat completions are billable POSTs, so UberLLM requests are never
// replayed automatically (same policy as invokeAtmoRouter). A 429 is thrown
// so invokeModel's caller can escalate to a different provider/tier.
const response = await performJsonRequest(endpoint, {
headers,
body: uberllmBody,
maxRetriesOverride: 0,
retryableStatusesOverride: [],
}, "UberLLM");

if (!response.ok && response.status === 429) {
const err = new Error(`UberLLM rate-limited: ${String(response.json?.error?.message || '').slice(0, 120)}`);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintainability · low
The response.json?.error?.message access could be null if performJsonRequest fails to parse the response body. The error message will then be an empty string, which may not be informative for debugging.

Suggestion: Add a fallback that uses the raw response text when JSON parsing fails.

Suggestion:

Suggested change
const err = new Error(`UberLLM rate-limited: ${String(response.json?.error?.message || '').slice(0, 120)}`);
const err = new Error(`UberLLM rate-limited: ${String(response.json?.error?.message || response.text?.slice(0, 120) || '').slice(0, 120)}`);

err.status = 429;
throw err;
}

return response;
}

async function invokeLlamaCpp(body, _incomingHeaders = {}) {
if (!config.llamacpp?.endpoint) {
throw new Error("llama.cpp endpoint is not configured.");
Expand Down Expand Up @@ -4029,6 +4095,7 @@ const PROVIDER_INVOKERS = {
atmorouter: invokeAtmoRouter,
runcrate: invokeRuncrate,
openscale: invokeOpenScale,
uberllm: invokeUberLLM,
llamacpp: invokeLlamaCpp,
lmstudio: invokeLMStudio,
bedrock: invokeBedrock,
Expand Down Expand Up @@ -4968,6 +5035,7 @@ module.exports = {
invokeAtmoRouter,
invokeRuncrate,
invokeOpenScale,
invokeUberLLM,
invokeOrcaRouter,
invokeCursor,
PROVIDER_INVOKERS,
Expand Down
2 changes: 1 addition & 1 deletion src/clients/provider-capabilities.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ const NATIVE_THINKING_BEDROCK_MODELS = [
"claude-haiku",
];

const REASONING_CONTENT_PROVIDERS = new Set(["moonshot", "openrouter", "edenai", "openai", "azure-openai", "atlas", "cheaperinference", "atmorouter", "runcrate", "openscale", "orcarouter"]);
const REASONING_CONTENT_PROVIDERS = new Set(["moonshot", "openrouter", "edenai", "openai", "azure-openai", "atlas", "cheaperinference", "atmorouter", "runcrate", "openscale", "uberllm", "orcarouter"]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintainability · low
The uberllm provider is added to REASONING_CONTENT_PROVIDERS but lacks corresponding entries in other capability sets like NATIVE_THINKING_PROVIDERS. This may lead to inconsistent feature detection for the provider.

Suggestion: Review other provider registrations (e.g., openrouter, openai) to ensure consistent capability coverage, and add uberllm to any missing capability sets that it supports.


function supportsNativeThinking(providerType, model) {
if (NATIVE_THINKING_PROVIDERS.has(providerType)) return true;
Expand Down
Loading
Loading