Skip to content

feat(harness): add a Gemini CLI harness (tap, turn, and init templates) - #516

Open
michaelxu2288 wants to merge 1 commit into
scaleapi:nextfrom
michaelxu2288:feat/gemini-cli-harness
Open

feat(harness): add a Gemini CLI harness (tap, turn, and init templates)#516
michaelxu2288 wants to merge 1 commit into
scaleapi:nextfrom
michaelxu2288:feat/gemini-cli-harness

Conversation

@michaelxu2288

@michaelxu2288 michaelxu2288 commented Sep 10, 2026

Copy link
Copy Markdown

What

Adds Gemini CLI as a framework harness, mirroring the Claude Code and Codex ones:

  • convert_gemini_cli_to_agentex_events (src/agentex/lib/adk/_modules/_gemini_cli_sync.py): maps the CLI's stream-json events (init, message deltas, tool_use, tool_result, error, result; schema from packages/core/src/output/types.ts in google-gemini/gemini-cli) onto the canonical StreamTaskMessage* stream. Assistant deltas open one text slot that closes on the next tool event, the result, or end of stream, so every Start has a Done; tool requests and results pair by tool_id; a finally closes the source iterator on cancellation like the other taps.
  • GeminiCliTurn (_gemini_cli_turn.py): HarnessTurn wrapper exposing session_id and model from init and normalising result.stats into TurnUsage (tokens, cached, duration, tool calls; cost is not reported by the CLI).
  • Both exported from agentex.lib.adk.
  • agentex init templates sync-gemini-cli, default-gemini-cli, temporal-gemini-cli, registered in TemplateType, the file map and the menus. Prompt passed via -p with stdin closed (in headless mode the CLI reads stdin to EOF and appends it), optional GEMINI_MODEL, GEMINI_API_KEY credential, npm install -g @google/gemini-cli in the Dockerfile. Turns are independent prompts because the CLI's --resume takes latest/index rather than a session id; the Temporal template keeps the reported session_id for observability only and says so.

Companion docs page: scaleapi/scale-agentex#429.

Test

  • tests/lib/adk/test_gemini_cli_sync.py (12), tests/lib/adk/test_gemini_cli_turn.py (12), tests/lib/core/harness/test_harness_gemini_cli_sync.py (5): pass.
  • tests/lib/cli/test_init_templates.py: 29 passed, including the three new templates; all rendered projects byte-compile.
  • ruff check and format clean on the new files.
  • Offline fixtures follow the CLI's published event schema; I have not run a live smoke test against the gemini binary (needs a Gemini API key). Happy to add a recorded live fixture if a maintainer can share one, or run it myself once I have a key.

https://claude.ai/code/session_01HCVKnA7LeJZ44nxZz1uzF3

Adds Gemini CLI as a framework harness alongside Claude Code and Codex:

- convert_gemini_cli_to_agentex_events: maps the CLI's stream-json events
  (init, message deltas, tool_use, tool_result, error, result; schema per
  packages/core/src/output/types.ts in google-gemini/gemini-cli) onto the
  canonical StreamTaskMessage* stream. Assistant deltas open one text slot
  that closes on the next tool event, the result, or end of stream, so every
  Start has a Done; tool requests and results pair by tool_id.
- GeminiCliTurn: HarnessTurn wrapper exposing session_id and model from the
  init event and normalising result.stats into TurnUsage.
- Both exported from agentex.lib.adk.
- agentex init templates sync-gemini-cli, default-gemini-cli and
  temporal-gemini-cli (registered in TemplateType, file map and menus),
  cloned from the Claude Code templates: prompt passed via -p with stdin
  closed (the CLI reads stdin to EOF in headless mode), optional GEMINI_MODEL,
  GEMINI_API_KEY credential, npm install -g @google/gemini-cli in the
  Dockerfile. Turns are independent prompts: the CLI's --resume takes
  latest/index, not a session id.
- Tests: tap (text deltas, whole messages, tools, errors, callbacks,
  source close on cancel), turn (usage mapping, protocol), harness end to
  end through UnifiedEmitter with span derivation; template suite covers the
  three new templates.

Offline tests only; a live smoke run needs a Gemini API key.

Claude-Session: https://claude.ai/code/session_01HCVKnA7LeJZ44nxZz1uzF3
Comment on lines +262 to +267
elif evt_type == "result":
done = _close_text()
if done is not None:
yield done
if on_result is not None:
await on_result(evt)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Terminal failures appear successful

A terminal Gemini result can have status: "error", but this branch closes the text stream and invokes the normal result callback without checking the status or error payload. UnifiedEmitter therefore treats authentication, turn-limit, and other terminal failures as successful completion, potentially returning missing or partial output instead of surfacing the error. Standalone error events are also logged without entering the canonical stream.

Knowledge Base Used:

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agentex/lib/adk/_modules/_gemini_cli_sync.py
Line: 262-267

Comment:
**Terminal failures appear successful**

A terminal Gemini `result` can have `status: "error"`, but this branch closes the text stream and invokes the normal result callback without checking the status or error payload. `UnifiedEmitter` therefore treats authentication, turn-limit, and other terminal failures as successful completion, potentially returning missing or partial output instead of surfacing the error. Standalone error events are also logged without entering the canonical stream.

**Knowledge Base Used:**
- [Agent framework integrations](https://app.greptile.com/scale-ai/-/custom-context/knowledge-base/scaleapi/scale-agentex-python/-/docs/agent-framework-integrations.md)
- [Harness delivery and metrics](https://app.greptile.com/scale-ai/-/custom-context/knowledge-base/scaleapi/scale-agentex-python/-/docs/harness-and-metrics.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Cursor Fix in Claude Code Fix in Codex

Comment on lines +121 to +126
if proc.returncode is None:
try:
proc.terminate()
except ProcessLookupError:
pass
await proc.wait()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Cancellation can hang indefinitely

If the Gemini CLI or an active tool subprocess does not exit after SIGTERM, this unbounded proc.wait() blocks generator cleanup indefinitely. As a result, request cancellation or Temporal activity cancellation can remain hung. The same cleanup pattern also appears in the sync template and temporal-gemini-cli/project/activities.py.j2; each path needs a bounded graceful shutdown followed by forced termination.

Knowledge Base Used: Temporal execution

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agentex/lib/cli/templates/default-gemini-cli/project/acp.py.j2
Line: 121-126

Comment:
**Cancellation can hang indefinitely**

If the Gemini CLI or an active tool subprocess does not exit after SIGTERM, this unbounded `proc.wait()` blocks generator cleanup indefinitely. As a result, request cancellation or Temporal activity cancellation can remain hung. The same cleanup pattern also appears in the sync template and `temporal-gemini-cli/project/activities.py.j2`; each path needs a bounded graceful shutdown followed by forced termination.

**Knowledge Base Used:** [Temporal execution](https://app.greptile.com/scale-ai/-/custom-context/knowledge-base/scaleapi/scale-agentex-python/-/docs/temporal-execution.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Cursor Fix in Claude Code Fix in Codex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant