Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions documentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Connect Lynkr to your development tools:
- **[Claude Code CLI Setup](claude-code-cli.md)** - Configure Claude Code CLI to use Lynkr
- **[Claude Desktop Setup](claude-desktop.md)** - Route the Claude Desktop app through Lynkr (macOS gateway profile, model picker as tier selector)
- **[Tier Pinning](tier-pinning.md)** - Use any desktop client's model picker as a routing tier selector (the generalized technique behind the Claude Desktop and Codex integrations)
- **[Switching Models](model-switching.md)** - Pick a tier from any client with the `lynkr-*` model ids, see which model served each request (status line, badge, headers, dashboard), and give each CLI its own model
- **[opencode Integration](opencode.md)** - Per-tier context windows via `lynkr run opencode` (picker doubles as tier pin; compaction always matches the serving model)
- **[OpenMausBot Integration](openmausbot.md)** - Run OpenMausBot bots through Lynkr's OpenAI-compatible endpoint (engine config, client profile, no tool injection)
- **[Context Window Header](context-window-header.md)** - `X-Lynkr-Context-Window`: the served model's real context window on every response, so clients can compact against the model actually serving them
Expand Down
27 changes: 27 additions & 0 deletions documentation/codex-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,33 @@ enabled = true

---

## Pinning a Routing Tier

With tier routing enabled (all four `TIER_*` set), the `model` Codex sends
can pin a tier instead of leaving the choice to content scoring:

```toml
# ~/.codex/config.toml
model = "lynkr-reasoning" # or lynkr-simple / lynkr-medium / lynkr-complex
```

Or per run: `codex -m lynkr-simple`.

| `model` | Served by |
|---|---|
| `lynkr-auto` | Content-based tier routing (no pin) |
| `lynkr-simple` | `TIER_SIMPLE` |
| `lynkr-medium` | `TIER_MEDIUM` |
| `lynkr-complex` | `TIER_COMPLEX` |
| `lynkr-reasoning` | `TIER_REASONING` |

A `lynkr-*` id wins over the `model_reasoning_effort` Codex sends with each
request, and `lynkr-auto` turns off the effort-based pin entirely. Any other
model id keeps the `(model, effort)` mapping described in
[tier-pinning.md](tier-pinning.md). Both wire APIs behave the same.

---

## Wire API Formats

Codex supports two API formats:
Expand Down
10 changes: 10 additions & 0 deletions documentation/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,16 @@ export FALLBACK_PROVIDER=databricks

---

### How do I switch models, or see which model answered?

- **Switch:** set your client's model to `lynkr-simple`, `lynkr-medium`, `lynkr-complex` or `lynkr-reasoning` to pin that tier, or `lynkr-auto` for automatic routing (e.g. `claude --model lynkr-reasoning`, or `model = "lynkr-simple"` in Codex's `config.toml`).
- **See:** add `lynkr-statusline` to Claude Code, set `LYNKR_VISIBLE_ROUTING=true` for a badge in every reply, or read the `X-Lynkr-Tier` / `X-Lynkr-Model` / `X-Lynkr-Provider` headers on `/v1/messages` responses.
- **Per CLI:** give each client a different `lynkr-*` id and point those tiers at the models you want.

See [Switching Models](model-switching.md) for details.

---

### What is MODEL_PROVIDER and do I still need it?

`MODEL_PROVIDER` sets a single static provider for all requests. When you set `MODEL_PROVIDER=ollama`, every request goes to Ollama regardless of complexity.
Expand Down
155 changes: 155 additions & 0 deletions documentation/model-switching.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# Switching Models and Seeing Which One Served

Lynkr picks a model for every request from the four tiers you configure
(`TIER_SIMPLE`, `TIER_MEDIUM`, `TIER_COMPLEX`, `TIER_REASONING`). This page
answers the everyday questions about that choice:

- [How do I switch model from my client?](#switch-model-from-your-client)
- [How do I see which model answered?](#see-which-model-served-a-request)
- [How do I change the routing rules?](#change-the-routing-rules)
- [How do I use a different model for each CLI?](#a-different-model-for-each-cli)

Picking a tier needs tier routing enabled, i.e. all four `TIER_*`
variables set (see [Routing & Model Tiering](routing.md)).

---

## Switch Model From Your Client

By default every request is scored and sent to the tier that fits it. To
choose the tier yourself, set the client's model to one of Lynkr's tier ids:

| Model id | Served by |
|---|---|
| `lynkr-auto` | Automatic, content-based routing (the default behaviour) |
| `lynkr-simple` | `TIER_SIMPLE` |
| `lynkr-medium` | `TIER_MEDIUM` |
| `lynkr-complex` | `TIER_COMPLEX` |
| `lynkr-reasoning` | `TIER_REASONING` |

The ids work on every endpoint (`/v1/messages`, `/v1/chat/completions`,
`/v1/responses`), so the same trick works in any client that lets you type a
model name:

| Client | How to pick a tier |
|---|---|
| Claude Code | `claude --model lynkr-reasoning`, `/model lynkr-simple` inside a session, or `export ANTHROPIC_MODEL=lynkr-complex` |
| opencode | `lynkr run opencode` writes the five entries; pick one in opencode's model picker ([details](opencode.md)) |
| Codex CLI | `model = "lynkr-medium"` in `~/.codex/config.toml`, or `codex -m lynkr-medium` ([details](codex-cli.md#pinning-a-routing-tier)) |
| Cline, Continue, Kilo Code, curl, SDKs | Use `lynkr-*` as the model name of the OpenAI-compatible provider |
| Claude Desktop | Its model picker is mapped to tiers ([details](claude-desktop.md)) |
| Codex Desktop | The model + effort picker is mapped to tiers ([details](tier-pinning.md)) |

Switch back to automatic routing with `lynkr-auto`.

A tier pin selects the **tier**, and the tier's model comes from your
`TIER_*` configuration. On `/v1/messages` (Claude Code, opencode) a few
explicit signals still override the pin for that one request: force phrases
such as `ultrathink` or `security audit`, high-risk edits, and a follow-up
that scores clearly above the pinned tier. See
[Tier Pinning](tier-pinning.md#scope-and-caveats) for the full scope.

---

## See Which Model Served a Request

**Status line (Claude Code).** One line under the prompt after every turn,
with no token cost. Add to `~/.claude/settings.json`:

```json
"statusLine": { "type": "command", "command": "lynkr-statusline" }
```

```
◆ Opus 4.8 · MEDIUM → minimax-m3:cloud (ollama) · today $0.06 · cache 67%
```

**Badge in the reply (any client).** Set `LYNKR_VISIBLE_ROUTING=true` and
every reply starts with the tier, model and provider that served it:

```
*[Lynkr] COMPLEX → GLM-4.7 (zai) · score 58*
```

**Response headers.** Anthropic-format responses (`/v1/messages`) carry
`X-Lynkr-Tier`, `X-Lynkr-Model` and `X-Lynkr-Provider` (full list in
[Routing Headers](routing.md#routing-headers)); non-streaming
`/v1/chat/completions` responses carry `X-Lynkr-Routed-Model` and
`X-Lynkr-Routed-Provider`:

```bash
curl -si http://localhost:8081/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: dummy" \
-H "anthropic-version: 2023-06-01" \
-d '{"model":"lynkr-auto","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}' \
| grep -i '^x-lynkr'
```

**Dashboard.** `http://localhost:8081/dashboard` shows recent requests,
spend and routing decisions per tier, model and provider.

**Dry run.** `lynkr route --preview` explains how a request *would* route
(every signal, the chosen tier and model) without calling a provider:

```bash
echo '{"messages":[{"role":"user","content":"rename this variable"}]}' \
| lynkr route --preview -
```

---

## Change the Routing Rules

- **Which model a tier uses** — edit the `TIER_*` lines in `.env`. The file
in the directory Lynkr was started from is watched and reloaded on save,
so no restart is needed (disable with `HOT_RELOAD_ENABLED=false`):

```bash
TIER_SIMPLE=ollama:qwen2.5-coder:7b
TIER_MEDIUM=openai:gpt-4o-mini
TIER_COMPLEX=zai:GLM-4.7
TIER_REASONING=openrouter:anthropic/claude-opus-4.1
```

- **Which phrases force a tier** — see [Force Patterns](routing.md#force-patterns).
- **How requests are scored** — see [Routing & Model Tiering](routing.md)
for the scoring modes, thresholds and the tier config file.

---

## A Different Model for Each CLI

Give each client its own tier id, and point those tiers at the models you
want. For example, Claude Code on a local Ollama model and opencode on GLM:

```bash
# .env
# SIMPLE → Claude Code, COMPLEX → opencode
TIER_SIMPLE=ollama:qwen2.5-coder:7b
TIER_MEDIUM=openai:gpt-4o-mini
TIER_COMPLEX=zai:GLM-4.7
TIER_REASONING=openrouter:anthropic/claude-opus-4.1
ZAI_API_KEY=your-zai-api-key
```

```bash
# Claude Code: pinned to the SIMPLE tier (Ollama)
ANTHROPIC_MODEL=lynkr-simple claude

# opencode: pick "lynkr-complex" in the model picker (GLM)
lynkr run opencode
```

Keep in mind:

- There are four tiers, so up to four clients can each have a model of
their own; clients pinned to the same tier share its model.
- Changing a `TIER_*` line changes the model for every client pinned to that
tier, and for automatic routing too.
- Clients set to `lynkr-auto` keep full automatic routing across all four
tiers.
- On `/v1/messages` the overrides described in
[Switch Model From Your Client](#switch-model-from-your-client) still
apply, so an `ultrathink` turn from a client pinned to SIMPLE goes to the
REASONING tier for that request.
32 changes: 28 additions & 4 deletions documentation/tier-pinning.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Unrecognized ids/values always resolve to *no pin* (fall through to normal
routing) — the resolver never guesses (see `documentation/claude-desktop.md`
for the incident-shaped reasoning behind exact-match-only resolution).

## Two client shapes, two mechanisms
## Client shapes and their mechanisms

### 1. Gateway-advertised ids (Claude Desktop) — `src/routing/model-slots.js`

Expand All @@ -52,6 +52,30 @@ shown in the UI as effort labels like "Light"). Lynkr pins on the
`src/api/openai-router.js` on both `/v1/chat/completions` and
`/v1/responses`.

### 3. Lynkr's own tier ids (any client) — `src/routing/model-slots.js`

For clients whose model id you set yourself (a CLI flag, a config file, a
custom-model entry in a picker), Lynkr accepts five virtual ids on **every**
ingress — Anthropic `/v1/messages` and OpenAI `/v1/chat/completions` +
`/v1/responses`:

| `model` | Tier |
|---|---|
| `lynkr-auto` | none — full content-based routing |
| `lynkr-simple` | `SIMPLE` |
| `lynkr-medium` | `MEDIUM` |
| `lynkr-complex` | `COMPLEX` |
| `lynkr-reasoning` | `REASONING` |

Matching is case-insensitive. On the OpenAI routes a virtual id outranks the
Codex `(model, effort)` mapping above: Codex attaches a `reasoning.effort`
to every request, so `lynkr-simple` + effort `high` pins SIMPLE, and
`lynkr-auto` keeps content routing instead of letting the effort pin every
turn. Resolution for the OpenAI routes lives in
`resolveOpenAIRequestPin()` (`src/routing/openai-model-slots.js`). For
per-client setups built on these ids see
[model-switching.md](model-switching.md).

## Generalizing to a new client

The pattern reduces to three questions:
Expand All @@ -69,9 +93,9 @@ The pattern reduces to three questions:
content scoring. A picker pin is advisory-input-turned-explicit — an
unmapped value must degrade to the default, never to a wrong pin.

Both resolvers return `null` for anything unrecognized, and both feed the
same downstream pin mechanism (`_forceProvider`), so a new client mapping is
one small module + one resolver call at its ingress point.
Every resolver returns `null` for anything unrecognized, and all of them
feed the same downstream pin mechanism (`_forceProvider`), so a new client
mapping is one small module + one resolver call at its ingress point.

## Scope and caveats

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
"start:supervised": "while true; do node index.js 2>&1 | npx pino-pretty --sync; code=$?; echo \"[supervisor] lynkr exited ($code) — restarting in 3s\"; sleep 3; done",
"lint": "eslint src index.js",
"test": "npm run test:unit && npm run test:performance",
"test:unit": "LYNKR_KNN_DIR=/tmp/lynkr-test-knn DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com LOG_FILE_ENABLED=false node --test test/routing.test.js test/hybrid-routing-integration.test.js test/retry-logic.test.js test/sse-transformer.test.js test/passthrough-stream.test.js test/passthrough-mode.test.js test/openrouter-error-resilience.test.js test/format-conversion.test.js test/azure-openai-config.test.js test/azure-openai-format-conversion.test.js test/azure-openai-routing.test.js test/azure-openai-streaming.test.js test/azure-openai-error-resilience.test.js test/azure-openai-integration.test.js test/openai-integration.test.js test/atlas-integration.test.js test/cheaperinference-integration.test.js test/atmorouter-integration.test.js test/runcrate-integration.test.js test/openscale-integration.test.js test/standardcompute-integration.test.js test/toon-compression.test.js test/gcf-compression.test.js test/llamacpp-integration.test.js test/resilience.test.js test/telemetry-routing.test.js test/memory/store.test.js test/memory/surprise.test.js test/memory/extractor.test.js test/memory/search.test.js test/memory/retriever.test.js test/memory/distiller.test.js test/memory/distiller-freeze.test.js test/memory/wiki.test.js test/memory/skills-cache.test.js test/memory/tencentdb-launcher.test.js test/distill.test.js test/large-payload.test.js test/prompt-cache-injection.test.js test/risk-analyzer.test.js test/interaction-block.test.js test/preflight.test.js test/token-reduction.test.js test/session-affinity.test.js test/cache-state.test.js test/cache-switch-cost.test.js test/lens-recommendations.test.js test/model-registry-cost.test.js test/output-format-guard.test.js test/tier-fallback.test.js test/wrap.test.js test/init.test.js test/tool-call-response-metadata.test.js test/degradation.test.js test/routing-telemetry-columns.test.js test/sticky-routing.test.js test/knn-ambiguous-escalate.test.js test/deescalator.test.js test/client-profiles.test.js test/openai-ingress-tool-injection.test.js test/tool-result-compression-gate.test.js test/strip-internal-fields.test.js test/complexity-tool-subtraction.test.js test/bandit.test.js test/routing-propensity.test.js test/reward-pipeline.test.js test/knn-cold-start.test.js test/calibration.test.js test/feedback-loop.test.js test/session-fingerprint.test.js test/side-request-guards.test.js test/verifier.test.js test/intent-score.test.js test/difficulty-classifier.test.js test/classifier-setup.test.js test/usage-stats.test.js test/loop-guard.test.js test/moonshot-model-mapping.test.js test/baidu-model-mapping.test.js test/tenant-policy-ingress-parity.test.js test/decide.test.js test/embeddings-degradation.test.js test/health-probe.test.js test/stuck-detector.test.js test/onnx-embedder.test.js test/ope.test.js test/hierarchical-budget.test.js test/token-rate-limit.test.js test/otel-export.test.js test/mcp-broker.test.js test/compression-budget.test.js test/gpt-utils.test.js test/dedup-observe-only.test.js test/context-window-header.test.js test/token-budget-auto.test.js test/opencode-setup.test.js test/auth-mode-first-party.test.js test/harness-envelope.test.js test/task-ledger.test.js test/jev-router.test.js test/jev-routing.test.js test/force-patterns.test.js test/upstream-fidelity.test.js test/tool-schema-compression.test.js test/passthrough-route.test.js test/quota-ledger.test.js test/shortfall-semantic.test.js test/routing-decisions.test.js test/routing-corpus.test.js test/switch-gate-grounding.test.js",
"test:unit": "LYNKR_KNN_DIR=/tmp/lynkr-test-knn DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com LOG_FILE_ENABLED=false node --test test/routing.test.js test/hybrid-routing-integration.test.js test/retry-logic.test.js test/sse-transformer.test.js test/passthrough-stream.test.js test/passthrough-mode.test.js test/openrouter-error-resilience.test.js test/format-conversion.test.js test/azure-openai-config.test.js test/azure-openai-format-conversion.test.js test/azure-openai-routing.test.js test/azure-openai-streaming.test.js test/azure-openai-error-resilience.test.js test/azure-openai-integration.test.js test/openai-integration.test.js test/atlas-integration.test.js test/cheaperinference-integration.test.js test/atmorouter-integration.test.js test/runcrate-integration.test.js test/openscale-integration.test.js test/standardcompute-integration.test.js test/toon-compression.test.js test/gcf-compression.test.js test/llamacpp-integration.test.js test/resilience.test.js test/telemetry-routing.test.js test/memory/store.test.js test/memory/surprise.test.js test/memory/extractor.test.js test/memory/search.test.js test/memory/retriever.test.js test/memory/distiller.test.js test/memory/distiller-freeze.test.js test/memory/wiki.test.js test/memory/skills-cache.test.js test/memory/tencentdb-launcher.test.js test/distill.test.js test/large-payload.test.js test/prompt-cache-injection.test.js test/risk-analyzer.test.js test/interaction-block.test.js test/preflight.test.js test/token-reduction.test.js test/session-affinity.test.js test/cache-state.test.js test/cache-switch-cost.test.js test/lens-recommendations.test.js test/model-registry-cost.test.js test/output-format-guard.test.js test/tier-fallback.test.js test/wrap.test.js test/init.test.js test/tool-call-response-metadata.test.js test/degradation.test.js test/routing-telemetry-columns.test.js test/sticky-routing.test.js test/knn-ambiguous-escalate.test.js test/deescalator.test.js test/client-profiles.test.js test/openai-ingress-tool-injection.test.js test/openai-virtual-tier-pin.test.js test/tool-result-compression-gate.test.js test/strip-internal-fields.test.js test/complexity-tool-subtraction.test.js test/bandit.test.js test/routing-propensity.test.js test/reward-pipeline.test.js test/knn-cold-start.test.js test/calibration.test.js test/feedback-loop.test.js test/session-fingerprint.test.js test/side-request-guards.test.js test/verifier.test.js test/intent-score.test.js test/difficulty-classifier.test.js test/classifier-setup.test.js test/usage-stats.test.js test/loop-guard.test.js test/moonshot-model-mapping.test.js test/baidu-model-mapping.test.js test/tenant-policy-ingress-parity.test.js test/decide.test.js test/embeddings-degradation.test.js test/health-probe.test.js test/stuck-detector.test.js test/onnx-embedder.test.js test/ope.test.js test/hierarchical-budget.test.js test/token-rate-limit.test.js test/otel-export.test.js test/mcp-broker.test.js test/compression-budget.test.js test/gpt-utils.test.js test/dedup-observe-only.test.js test/context-window-header.test.js test/token-budget-auto.test.js test/opencode-setup.test.js test/auth-mode-first-party.test.js test/harness-envelope.test.js test/task-ledger.test.js test/jev-router.test.js test/jev-routing.test.js test/force-patterns.test.js test/upstream-fidelity.test.js test/tool-schema-compression.test.js test/passthrough-route.test.js test/quota-ledger.test.js test/shortfall-semantic.test.js test/routing-decisions.test.js test/routing-corpus.test.js test/switch-gate-grounding.test.js",
"test:memory": "LYNKR_KNN_DIR=/tmp/lynkr-test-knn DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node --test test/memory/store.test.js test/memory/surprise.test.js test/memory/extractor.test.js test/memory/search.test.js test/memory/retriever.test.js test/memory/distiller.test.js test/memory/distiller-freeze.test.js test/memory/wiki.test.js test/memory/skills-cache.test.js test/memory/tencentdb-launcher.test.js",
"test:new-features": "LYNKR_KNN_DIR=/tmp/lynkr-test-knn DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node --test test/passthrough-mode.test.js test/openrouter-error-resilience.test.js test/format-conversion.test.js",
"test:performance": "LYNKR_KNN_DIR=/tmp/lynkr-test-knn DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node test/hybrid-routing-performance.test.js && DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node test/performance-tests.js",
Expand Down
Loading
Loading