fix(search): harden live search accuracy, provider queries, and request cost by waleedlatif1 · Pull Request #8226 · simstudioai/sim

waleedlatif1 · 2026-09-24T04:32:57Z

Summary

GitHub: batch code search by bytes under its 1,000-byte query limit (default searches were getting 400s), send date bounds as one updated:start..end range (two updated: qualifiers are ORed), keep qualifiers outside the grouped text (they returned 0/422 inside parentheses), and leave dateless code out of dated searches
Calendar: keep one verified copy of a meeting shared across calendars, and order dated agendas by start instead of calendar by calendar
Clean Gmail snippets (entities, invisible preheader padding), Calendar descriptions, and Confluence excerpts with html-to-text; report Google quota 403s as rate_limited instead of "reconnect"
partial now means degraded coverage only; more matches stay on nextCursor, and an account whose results were cut from the merge drops its cursor so continuing can't skip them
Search and read share one account session: filter before verifying, verify in parallel, pass GitLab search evidence instead of refetching each issue, reuse the accounts listed for the search, and run independent DB loads in parallel
Pin DNS on the request as well as the agent, since Bun ignores an agent-level lookup; reuse pinned keep-alive connections per search (~85 ms vs ~430 ms per provider call) with compressed responses, still capped at 4 MB decoded
Previews center on the query match; reads honor limit; Atlassian sites run in parallel; Gmail reads only the metadata fields it uses; the Drive service verification budget caps the page
Route the Search MCP chat tool to the assistant chat endpoint (the worker's execute route only accepts the block payload)

Type of Change

Bug fix

Testing

Type-check, Biome, and all 49 audits pass. Ran 8,719 tests across search, knowledge, security, connectors, and MCP. New tests fail against the previous code. Verified GitHub query limits and date ranges, Google/Atlassian response shapes, and Bun connection pinning against live provider APIs.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…st cost - GitHub: batch code search by bytes under its 1,000-byte query limit, send date bounds as one updated:start..end range, keep qualifiers outside grouped text, and exclude dateless code from dated searches - Calendar: keep one verified copy of a meeting shared across calendars and order dated agendas by start - Clean Gmail, Calendar, and Confluence text; report Google quota 403s as rate limits - Report partial only for degraded coverage; drop cursors that would skip merged-out results - Share one account session between search and read; filter before verifying, verify in parallel, pass GitLab evidence, and reuse listed accounts - Pin DNS on the request as well as the agent (Bun ignores agent lookups), and reuse pinned keep-alive connections with compressed responses - Route the Search MCP chat tool to the assistant chat endpoint

vercel · 2026-09-24T04:33:02Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Sep 24, 2026 5:40am UTC

greptile-apps · 2026-09-24T04:37:13Z

Confidence Score: 5/5

The PR appears safe to merge; the latest changes fully address the remaining card-matcher finding without introducing an actionable regression.

Summary

This PR hardens live provider search behavior, authorization verification, pagination semantics, response processing, and request efficiency.

Corrects GitHub batching and date-query behavior, Calendar merging and ordering, and provider-specific text cleanup.
Shares account sessions between search and read while parallelizing independent account, policy, and verification work.
Adds DNS-pinned connection reuse and bounded decompression for provider requests.
Separates degraded coverage from ordinary pagination and preserves account cursors only when continuation is safe.
Routes Search MCP chat through the assistant endpoint and strips only validated interactive-card payloads from text responses.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Live search request] --> B[List allowed accounts]
  B --> C[Resolve account credentials]
  C --> D[Open shared account session]
  D --> E[Provider search]
  E --> F[Apply filters]
  F --> G[Verify candidates in parallel]
  G --> H[Merge and rank results]
  H --> I[Return results and safe cursors]
  D --> J[Read selected result]
  J --> K[Re-check current source boundary]

_{Reviews (5) · Last reviewed commit: "fix(search): strip an MCP answer card on..."}

cubic-dev-ai

Review completed against the latest diff

_{Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic}

…d MCP chat - Keep invisible-character cleanup linear, table rows and definition lists readable, meaningful joiners intact, and deeply nested markup readable - Strip interactive Chat tags from MCP chat answers with the helper the Slack assistant shares - Report empty continuable pages and short dated listings as partial; dedupe after date filtering - Merge GitHub batches within each kind, give repository search the larger query budget, keep the 1,000-result hint, and skip code search when the query leaves no room for a repository - Keep memoized metadata failures for the request, re-read metadata after a document read, reload the Coda grant on every provider load, and request only gzip or brotli

waleedlatif1 · 2026-09-24T05:01:40Z

@greptile

waleedlatif1 · 2026-09-24T05:01:42Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-24T05:01:47Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 38 files

_{Requires human review: Auto-approval blocked because this review re-detected 3 unresolved issues already reported by Cubic.
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Fix all with cubic | Re-trigger cubic}

…ies, and answer tags - Report any page with more matches no cursor can reach as partial, and an empty continuable page as partial - Keep live reads on the 8,000-character page budget indexed reads use - Leave boolean GitHub queries as written and explain the 256-character search text limit - Strip only closed interactive tags from complete MCP answers so prose after an unclosed opener survives - Skip the wire-size Content-Length precheck for encoded bodies; the decoded stream stays capped - Build the meaningful-mark pattern without a joiner inside a character class

waleedlatif1 · 2026-09-24T05:17:01Z

@greptile

waleedlatif1 · 2026-09-24T05:17:02Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-24T05:17:08Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

waleedlatif1 · 2026-09-24T05:17:39Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-24T05:17:43Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 39 files

_{Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Fix all with cubic | Re-trigger cubic}

cubic-dev-ai

All reported issues were addressed across 39 files

_{Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Fix all with cubic | Re-trigger cubic}

…ed prose - Remove interactive cards by their JSON payload so closing markers inside strings cannot end a card and tag-shaped prose stays - Keep the Slack stream's existing tag withholding unchanged - Require valid element syntax in looksLikeHtml so an address like <a@acme.com> stays text - Keep verified results when verification is rate-limited and name the rate limit

waleedlatif1 · 2026-09-24T05:30:53Z

@greptile

waleedlatif1 · 2026-09-24T05:30:54Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-24T05:31:02Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 39 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic}

waleedlatif1 · 2026-09-24T05:40:43Z

@greptile

waleedlatif1 · 2026-09-24T05:40:45Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-24T05:40:49Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 39 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic}

greptile-apps Bot reviewed Sep 24, 2026

View reviewed changes

Comment thread apps/sim/lib/sim-search/live/github.ts Outdated

Comment thread apps/sim/lib/sim-search/live/coda-mcp.ts Outdated

Comment thread apps/sim/lib/core/security/secure-fetch-connection-pool.server.test.ts

fix(search): keep document spacing when cleaning provider text

459e93c

vercel Bot temporarily deployed to Preview September 24, 2026 04:39 Inactive