Skip to content

feat(insights): rank optional context when sources exceed the budget - #864

Merged
izadoesdev merged 2 commits into
stagingfrom
codex/insights-context-ranking
Sep 27, 2026
Merged

izadoesdev merged 2 commits into
stagingfrom
codex/insights-context-ranking

Conversation

@izadoesdev

@izadoesdev izadoesdev commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

When the final profile and recalled-context merge exceeds the native source budget, relevant page excerpts can be omitted before an investigation sees them. This change uses one Jev evaluation to rank competing optional pages for the current subject, then applies the existing native packing rules.

The profile, canonical replies, original homepage, source contents, provenance, and budget stay under native control. Contexts that fit skip evaluation. Oversized input, unavailable billing, provider errors, invalid answers, and the one-second provider deadline return the exact native fallback. Returned usage is recorded before answer validation. The implementation covers fresh investigations and analysis replies; shadow runs require explicit injection, and clarifications/deterministic verification keep their current behavior.

Validation: repository lint and all 33 type-check tasks pass; 699 Insights tests pass (6 existing live tests skipped); 20 shared context/usage tests pass (1 existing service-dependent test skipped); the new ranking tests also pass an explicit TypeScript check. A frozen 48-investigation synthetic baseline/candidate comparison is running; this draft will be updated with the full quality and latency results. No production data, deployment, or merge is included.

Based directly on current staging; no unmerged PR dependency. Known overlaps: #849 shares the Gateway dependency, Jev price mapping, and nearby generation wiring; #842 shares Gateway lock entries; #751 owns broader business-context changes. This slice only changes final-merge page prioritization and reuses native packing.


Summary by cubic

Ranks competing optional pages with a single Jev call so the final profile/recall merge no longer drops relevant excerpts when it exceeds the native source budget.

  • Reuses the existing Jev fetch call from candidate relevance instead of adding a new gateway dependency.
  • One bounded evaluation (with a 1-second deadline) runs only when optional pages compete; contexts that fit skip evaluation entirely.
  • Oversized input, unavailable billing, provider errors, invalid answers, and timeouts return the exact native fallback, with usage recorded before answer validation.
  • Native packing still protects the profile, canonical replies, and original homepage; clarifications and deterministic verification are unchanged.
  • Applies to fresh investigations and analysis replies; shadow runs require explicit injection.
  • Adds typesafe-ai/jev pricing to agent credits.

Written for commit de17aa8. Summary will update on new commits.

Review in cubic

@vercel

vercel Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
dashboard Ready Ready Preview Sep 27, 2026 4:15pm UTC
databuddy-status Ready Ready Preview Sep 27, 2026 4:15pm UTC
documentation Ready Ready Preview Sep 27, 2026 4:15pm UTC

@coderabbitai

coderabbitai Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 6d72068f-9370-4e5d-b0b6-0accbd282ea6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Candidate relevance already calls Jev with a plain fetch, and ai@6 pins a
gateway provider without evaluation models. Export that call as
evaluateWithJev and reuse it for context ranking instead of adding
@ai-sdk/gateway. Ranking usage now goes through recordUsage, since
billUsage was removed on staging.
@unkey-deploy

unkey-deploy Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Unkey Deploy

Name Status Preview Inspect Updated (UTC)
links (preview) Ready Visit Preview Inspect Sep 27, 2026 4:13pm

@greptile-apps

greptile-apps Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Medium risk] Adds optional context ranking to investigation planning.

The review found no blocking behavioral failure, but the explicit repository typing requirement must be satisfied before merging.

Findings

  1. P2 Unknown types in Jev contract ▶
  2. P2 Analysis ranking lacks test coverage ▶

Summary

The PR adds a bounded Jev evaluation to prioritize optional business-context pages when the native merge omits sources, and wires it into fresh investigations and analysis replies.

  • Native packing and fallback remain responsible for the source budget and protected context.
  • Jev usage receives a model price mapping. The new reply branch needs direct test coverage, and the shared Jev types need to meet the repository’s explicit-type requirement.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Profile and recalled context] --> B[Native merge and budget check]
  B -->|All sources fit| D[Native context]
  B -->|Optional pages compete| C[Jev relevance ranking]
  C -->|Valid ranking| E[Native packing of reordered pages]
  C -->|Unavailable, invalid or late| D
  E --> F[Investigation or analysis reply]
  D --> F
Loading

Reviews (1) · Last reviewed commit: "refactor(insights): rank context through..."

Comment thread apps/insights/src/business-aware-selection.ts
Comment thread apps/insights/src/resume.ts
@izadoesdev
izadoesdev merged commit 1c97e01 into staging Sep 27, 2026
22 checks passed
@izadoesdev
izadoesdev deleted the codex/insights-context-ranking branch September 27, 2026 16:25
izadoesdev added a commit that referenced this pull request Sep 27, 2026
…review-gaps

test(insights): close review gaps from #864 and #851

This branch was successfully deployed

4 active deployments
Preview – dashboard — de17aa80 Deployed Sep 27, 2026 by vercel[bot]
Preview – documentation — de17aa80 Deployed Sep 27, 2026 by vercel[bot]
Preview – databuddy-status — de17aa80 Deployed Sep 27, 2026 by vercel[bot]
links - preview — de17aa80 Deployed Sep 27, 2026 by unkey-deploy[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant