Skip to content

fix(insights): complete measured funnel-step investigations - #850

Merged
izadoesdev merged 2 commits into
stagingfrom
codex/insights-step-completion
Sep 27, 2026
Merged

izadoesdev merged 2 commits into
stagingfrom
codex/insights-step-completion

Conversation

@izadoesdev

@izadoesdev izadoesdev commented Sep 20, 2026 •

Copy link
Copy Markdown
Member

Funnel-step investigations with valid native measurements were always marked incomplete: the completion gate excluded funnel_step, and its citation filter expected the nonexistent get_funnel_step_analytics tool. Resolve the validated parent/step and accept the existing parent funnel analytics only when both exact comparison windows establish that step's population.

The gate checks tenant, definition, filters, unique predecessor/target rows, ordered counts, native two-decimal conversion and endpoint totals. It rejects clipped or conflicting reads from either window, including earlier uncited reads. Step conversion uses the preceding step's visitors, not the whole-funnel entrant count. Definition actions, saved verification checks, publication validation and billing rules are unchanged; supported step findings can now complete and follow the existing charge settlement path.

Validation:

  • 27 focused boundary cases, including valid cases that failed before the fix; 717 Insights tests passed, six existing skips.
  • Root lint, all 33 workspace typecheck tasks and all 29 repository test tasks passed. The initial sandboxed whole-repo run could not bind existing Slack/RPC HTTP test servers; rerunning with local server permission passed.
  • Direct typechecking of the normally excluded test file yields the same 46 existing diagnostics on baseline and candidate, with none added.
  • Independent code review found no actionable findings.

A fresh bounded synthetic native-agent comparison completed all eight attempts on September 21 using unchanged detector input/prompt and identical tools. Full-window native completion improved 0/2 to 2/2; useful full-window findings were already 2/2 in both arms. Both versions still published unsupported clipped-window comparisons despite remaining incomplete, exposing a separate publication-validation defect. This PR adds no model calls or prompt instructions, and no speed or overall usefulness gain was demonstrated (aggregate observed time 49.238s baseline versus 54.053s candidate, with mocked tool I/O). The prior Gateway-failed round is preserved separately. Keep draft while the publication defect is addressed and maintainer review remains pending.

Scope/dependencies: independent slice based on current staging; no dependency on an unmerged PR. #751 overlaps agent.ts and SPEC.md for business context and should be rechecked before landing. #849 is the separate Jev selector experiment; this PR does not include its selector or the rejected funnel-objective/revenue/claim-veto experiments.

AI disclosure: Codex implemented the fix, ran checks and conducted automated review. Human maintainer verification remains pending.

Current remote checks: 18 successful contexts and one neutral, with no pending or failed contexts. Configured review has not completed: CodeRabbit skipped auto-review, and cubic reached its monthly quota. All general comments/reviews/threads were read; there is no actionable code feedback.

The separate publication failure is addressed by draft #851, an independent slice from staging. Its frozen eight-run confirmation retained full-window publication 2/2 in both arms and changed correctly private clipped results from 0/2 to 2/2. It does not depend on this completion fix.

@unkey-deploy

unkey-deploy Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Unkey Deploy

Name Status Preview Inspect Updated (UTC)
links (preview) Ready Visit Preview Inspect Sep 27, 2026 4:06pm

@vercel

vercel Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
dashboard Ready Ready Preview Sep 27, 2026 4:08pm UTC
databuddy-status Ready Ready Preview Sep 27, 2026 4:08pm UTC
documentation Ready Ready Preview Sep 27, 2026 4:08pm UTC

@coderabbitai

coderabbitai Bot commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: af617b64-4554-4b78-8135-51374e2760c8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@izadoesdev

Copy link
Copy Markdown
Member Author

@greptile review

Please review the current head 45338a636102b8daa0675a24b9190fa81dd079fa for correctness, security, and regressions as part of the requested review of all open PRs. This PR should remain a draft; this request is for review only.

@greptile-apps

greptile-apps Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Critical risk] Changes how funnel-step investigations are marked complete and billed.

The completion change has no new actionable finding, but the outstanding repository-style requirement should be satisfied before merging.

Findings

  1. P2 Integer index access ▶

Summary

This PR allows measured funnel-step investigations to complete when cited parent-funnel analytics establish the step population in both exact comparison windows.

  • It validates the definition, counts, conversion rate, and conflicting reads before completion.
  • It adds boundary tests and clarifies the completion requirement in the spec.

Reviews (2) · Last reviewed commit: "fix(insights): narrow funnel-step rows w..."

Comment thread apps/insights/src/agent.ts Outdated

This branch was successfully deployed

4 active deployments
Preview – dashboard — bfe4b020 Deployed Sep 27, 2026 by vercel[bot]
Preview – documentation — bfe4b020 Deployed Sep 27, 2026 by vercel[bot]
Preview – databuddy-status — bfe4b020 Deployed Sep 27, 2026 by vercel[bot]
links - preview — bfe4b020 Deployed Sep 27, 2026 by unkey-deploy[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant