Skip to content

Add cookbook: pausing agents for captcha telemetry - #547

Open
dprevoznik wants to merge 12 commits into
mainfrom
hypeship/captcha-telemetry-wait-cookbook
Open

Add cookbook: pausing agents for captcha telemetry#547
dprevoznik wants to merge 12 commits into
mainfrom
hypeship/captcha-telemetry-wait-cookbook

Conversation

@dprevoznik

@dprevoznik dprevoznik commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • New cookbook: browsers/telemetry/pausing-for-captcha-solves.mdx — pauses a browser-loop agent when captcha_solve_started fires and resumes once captcha_solve_result / captcha_challenge_result reports a terminal outcome, reusing the abort-and-resume pattern from the existing Playwright/computer-use fallback cookbook.
  • Wired into docs.json under Telemetry, and cross-linked from the fallback cookbook's Next Steps.
  • Documents the @onkernel/sdk version conflict between browser-loop's pinned dependency (predates the telemetry API) and the SDK version the script needs, with a verified overrides fix.

Test plan

  • docs.json and the doc's JSON snippet parse as valid JSON
  • The full TypeScript example type-checks against the real @onkernel/browser-loop, @onkernel/sdk, and @earendil-works/pi-agent-core packages (installed at the exact versions browser-loop depends on, with the overrides fix applied) — caught and fixed a real API mismatch (InMemorySessionRepo/AgentHarnessEvent from a newer pi-agent-core than browser-loop pins) and a missing-telemetry-on-old-SDK issue before they shipped
  • Not run end-to-end against a live Kernel browser (no API key in this environment) — the script is a straightforward extension of the already-published fallback cookbook's pattern, but hasn't been observed solving a real captcha

Note

Low Risk
Documentation-only changes (new MDX page, nav entry, and cross-link) with no runtime or API behavior changes in this repo.

Overview
Adds a Telemetry cookbook that shows how to pause a @onkernel/browser-loop agent while Kernel’s captcha solver runs, using captcha telemetry plus a DOM page probe instead of prompt-only “wait for solve” behavior.

The guide walks through a captcha-gate.ts example: stream captcha_* events with order-independent joins on task_id/challenge_id, hold tool calls via harness.on("tool_call") with { block: true, reason }, and surface outcomes that distinguish solver task success from challenge-level results. It also documents setup (including pinning @earendil-works/pi-agent-core), verdict messaging, and limits.

Navigation: registers browsers/telemetry/pausing-for-captcha-solves in docs.json and links it from the Playwright computer-use fallback doc’s Next steps.

Reviewed by Cursor Bugbot for commit c694177. Bugbot is set up for automated code reviews on this repo. Configure here.

Shows how to use browser-loop's abort/resume pattern with the
captcha_solve_started / captcha_solve_result / captcha_challenge_result
telemetry events, so an agent stops acting while Kernel's solver is
working and resumes once it reports a terminal outcome.
@mintlify

mintlify Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
Kernel 🟢 Ready View Preview Sep 4, 2026, 5:00 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

cursor[bot]
cursor Bot approved these changes Sep 4, 2026
Comment thread browsers/telemetry/pausing-for-captcha-solves.mdx Outdated
Comment thread browsers/telemetry/pausing-for-captcha-solves.mdx
Comment thread browsers/telemetry/pausing-for-captcha-solves.mdx Outdated
Comment thread browsers/telemetry/pausing-for-captcha-solves.mdx Outdated
Buffers a terminal telemetry event that arrives before waitForOutcome
is called, clears tracked IDs on timeout so the next captcha still
triggers a pause, stops a same-challenge task from overwriting the
tracked task_id, and distinguishes a task-level success from a
challenge actually clearing.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD 3712d40 against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +252 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after approval: 3712d40 only edits the cookbook example — outcome buffering, distinct challenge_solved vs task_success, same-episode ID handling, and timeout ID reset. Same files, still documentation and nav config.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example systemPrompt / cookbook text lives in user-facing docs, not in model-instruction or production prompt files.
  • Incremental commit narrows example-code correctness; it does not expand blast radius.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

Comment thread browsers/telemetry/pausing-for-captcha-solves.mdx Outdated
Comment thread browsers/telemetry/pausing-for-captcha-solves.mdx Outdated
If an episode's outcome buffers before waitForOutcome is called and a
new episode starts before the driver gets back to it, the buffer is
now cleared so the driver waits on the current episode instead of
resuming on the stale one. Also breaks the dense gate-behavior
paragraph into bullets for scannability.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD 9df87e2 against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +261 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after the last assessment: 9df87e2 only edits the cookbook — clears a stale buffered outcome when a new captcha episode opens, and splits the gate-behavior paragraph into bullets. Same files, still documentation and nav config.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example systemPrompt / cookbook text lives in user-facing docs, not in model-instruction or production prompt files.
  • Incremental commit tightens example-code correctness; it does not expand blast radius.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

Comment thread browsers/telemetry/pausing-for-captcha-solves.mdx Outdated
Delivery is unordered, so a captcha_solve_started for an episode can
arrive after that episode's own terminal result already buffered.
The gate now remembers which task_id/challenge_id a buffered outcome
belongs to, so a late duplicate for that same episode is recognized
and left alone instead of being mistaken for a new episode (which
previously discarded the buffer and re-triggered a spurious pause).

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD d7f6e03 against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +286 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after the last assessment: d7f6e03 only edits the cookbook example — keeps the buffered outcome's task_id / challenge_id so a late duplicate captcha_solve_started is ignored instead of clearing the buffer. Same files, still documentation and nav config.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example systemPrompt / cookbook text lives in user-facing docs, not in model-instruction or production prompt files.
  • Incremental commit tightens example-code correctness; it does not expand blast radius.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

Starts on duckduckgo.com, navigates to 2captcha.com's public
Cloudflare Turnstile demo, and has the agent summarize the page's
explanation of how Turnstile is solved once the challenge clears.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD 0ed73cc against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +290 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after the last assessment: 0ed73cc only edits cookbook demo text — switches the example target from the public Google reCAPTCHA v2 demo to duckduckgo.com → 2captcha.com's Cloudflare Turnstile demo, and updates the example TASK_PROMPT / comment to match. Same files, still documentation and nav config.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example TASK_PROMPT / systemPrompt text lives in user-facing docs, not in model-instruction or production prompt files.
  • Incremental commit is demo-URL and example-prompt wording only; it does not expand blast radius.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

Removes inline references to the Playwright/computer-use fallback
cookbook from the body text (kept only in Next Steps), and replaces
the concrete duckduckgo/2captcha demo task with a placeholder the
reader fills in with their own task.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD 911955e against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +287 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after the last assessment: 911955e only edits cookbook copy — drops inline links to the Playwright/computer-use fallback page from the body (kept in Next Steps) and replaces the concrete duckduckgo/2captcha demo TASK_PROMPT with a placeholder. Same files, still documentation and nav config.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example TASK_PROMPT / systemPrompt text lives in user-facing docs, not in model-instruction or production prompt files.
  • Incremental commit is wording and a placeholder prompt; it does not expand blast radius.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

Quick-reference table of the three captcha events and their possible
terminal outcomes, right after they're introduced in prose.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD 924f68a against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +293 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after the last assessment: 924f68a only adds a markdown table in the cookbook intro listing the three captcha event types and their outcomes. Same files, still documentation and nav config.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example TASK_PROMPT / systemPrompt text lives in user-facing docs, not in model-instruction or production prompt files.
  • Incremental commit is a reference table; it does not expand blast radius.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

browser-loop's exact @onkernel/sdk pin is relaxed to a caret range
(kernel/browser-loop#94), matching the same removal already done for
the Playwright/computer-use fallback cookbook (#548).

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD 738ec65 against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +279 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after the last assessment: 738ec65 only deletes a <Warning> callout about a stale @onkernel/sdk version-pin / overrides workaround. Same files, still documentation and nav config. Surface area shrank slightly.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example TASK_PROMPT / systemPrompt text lives in user-facing docs, not in model-instruction or production prompt files.
  • Incremental commit is copy removal only; it does not expand blast radius.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

The previous gate paused on captcha_solve_started, joined challenge
results only when a start had carried a challenge_id, and had no
fallback when an event never arrived. On a real reCAPTCHA that dropped
the challenge-level outcome entirely and re-prompted the agent once per
solver task.

The gate now pairs tasks on task_id only, records every challenge result
and marks whether it can be attributed, bounds every wait, and falls
back to a read-only page probe. It holds at the tool_call hook instead
of aborting the turn, so an action is stopped before it reaches the
page and no re-prompt is needed.

Split into four named steps with the complete file in an accordion, and
pin pi-agent-core to the version browser-loop depends on -- the previous
install line resolved a newer one that the script doesn't compile
against.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD 4d7645f against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +522 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after the last assessment: 4d7645f rewrites the cookbook around captcha correlation rules — task-id joins, bounded waits, a read-only page probe, and a tool_call hold instead of abort-and-resume. Same three files; still documentation and nav config. No new paths, CI, or runtime code.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example systemPrompt / cookbook text lives in user-facing docs, not in model-instruction or production prompt files.
  • Incremental rewrite enlarges the example but does not expand blast radius beyond this docs site.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

/** A task the solver accepted has no terminal result yet. */
holding: () => openTasks().length > 0,
/** A terminal outcome is recorded that the agent hasn't been told about. */
pending: () => challenges.size > 0 || [...tasks.values()].some((t) => t.status && t.status !== "success"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Success outcomes skipped between turns

High Severity

pending treats only non-success task statuses as a terminal outcome, so a captcha_solve_result of success that lands between tool calls never enters resolve or reset. Those closed tasks stay in the maps and can be reused as the verdict for a later captcha, and the documented Turnstile task-success message never fires on this path.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 4d7645f. Configure here.

async function resolve(): Promise<Verdict> {
// A challenge result covers every task under it, so stop waiting once one lands.
await until(() => openTasks().length === 0 || Boolean(joinedResult()), TASK_SETTLE_MS);
if (joinable.size > 0 && !joinedResult()) await until(() => Boolean(joinedResult()), CHALLENGE_GRACE_MS);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up solver tasks get discarded

High Severity

resolve ends its wait when the tasks it already knows about finish, then watches only for captcha_challenge_result during the grace period. A later solver task that starts in that window is not waited on, and reset drops it, so the agent can act while Kernel’s solver is still working.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 4d7645f. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD f8d1708 against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +522 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after the last assessment: f8d1708 is a merge of main into this branch. The captcha cookbook is unchanged vs 4d7645f. Incoming main edits (payments/vaults nav, fallback-cookbook warning removal) are already on main and are not new surface from this PR. Same three files vs main; still documentation and nav config.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example systemPrompt / cookbook text lives in user-facing docs, not in model-instruction or production prompt files.
  • Merge from main does not expand this PR's blast radius.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

Was in a Note callout, which undersold content essential to
understanding the cookbook's design rather than a skippable aside.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD dbdabaa against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +518 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after the last assessment: dbdabaa only folds a <Note> callout into body prose in the cookbook intro. Same three files; still documentation and nav config. No new paths, CI, or runtime code.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example systemPrompt / cookbook text lives in user-facing docs, not in model-instruction or production prompt files.
  • Incremental edit is copy presentation only; it does not expand blast radius.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk assessment: Very Low

Verdict: Risk did not increase. Prior approval stands (not re-approving).

Re-evaluated HEAD c694177 against the previously approved commit 8e9d21e from the file diffs only (not the PR description).

Current scope (3 files, +518 / -1 vs main):

File Change
browsers/telemetry/pausing-for-captcha-solves.mdx New Mintlify cookbook (example TypeScript + setup notes)
docs.json Adds that page under the existing Telemetry nav group
browsers/playwright-computer-use-fallback.mdx One Next-steps cross-link

What changed after the last assessment: c694177 is a merge of main into this branch. The captcha cookbook, nav entry, and cross-link are unchanged vs dbdabaa. Incoming main edits (info/pricing.mdx, apps/invoke.mdx) are already on main and are not new surface from this PR. Same three files vs main; still documentation and nav config.

Why still Very Low

  • Documentation and Mintlify nav only. No application, API, auth, billing, or infrastructure code.
  • No CODEOWNERS file in this repo, so no required owner review.
  • Example systemPrompt / cookbook text lives in user-facing docs, not in model-instruction or production prompt files.
  • Merge from main does not expand this PR's blast radius.

Approval from the prior Very Low assessment remains in effect.

Open in Web View Automation 

Sent by Cursor Automation: Assign PR reviewers

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c694177. Configure here.

tasks.clear();
challenges.clear();
joinable.clear();
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Late start reopens a finished episode

Medium Severity

reset() drops every task_id and challenge_id after a verdict, so a captcha_solve_started that arrives after its already-consumed result is treated as a brand-new open task. The next tool call then holding()-waits the full TASK_SETTLE_MS for a result that was already seen and discarded.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit c694177. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant