Skip to content

fix(core): omit inline image data from compaction prompt - #381

Open
esecoder wants to merge 1 commit into
lessweb:mainfrom
esecoder:fix/compact-prompt-inline-images
Open

esecoder wants to merge 1 commit into
lessweb:mainfrom
esecoder:fix/compact-prompt-inline-images

Conversation

@esecoder

@esecoder esecoder commented Oct 22, 2026 •

Copy link
Copy Markdown

Problem - Conversation compacting is crashing!

When a session contains images, getCompactPrompt() serializes every message's contentParams verbatim. Inline image content is stored as data:image/...;base64,..., so the summary request counts the raw base64 payload as text tokens.

A real session with 18 screenshots in the summarized range contained about 2.5 MB of base64 data. The compaction request was 2,135,942 tokens against a 1,048,576-token model limit, so it failed with HTTP 400. Because compaction aborts before marking any messages as compacted, the session was permanently stuck above the auto-compaction threshold and could not continue.

Normal chat requests are unaffected because they send images as image_url parts, which are counted as image tokens rather than text.

Fix

  • Replace inline data: image URLs with [image data omitted from compaction prompt] when building the summary prompt.
  • Preserve remote image URLs and other image metadata (for example detail).
  • Leave non-image contentParams untouched.
  • Add a regression test covering inline and remote image URLs.

Verification

  • node --import tsx --test src/tests/prompt.test.ts — 25/25 pass
  • tsc -p ./ --noEmit — pass
  • eslint on changed files — pass
  • prettier --check on changed files — pass
  • Full core suite: 410 pass, 2 skipped, 3 fail

The 3 failures are permission tests (computeToolCallPermissions, UnderstandImage, ReadImage). They also fail on a clean checkout of HEAD and are unrelated to this change.

@esecoder

Copy link
Copy Markdown
Author

Compaction loop follow-up

Heads-up: this fix is correct and should not be blocked, but it can expose a pre-existing runaway compaction loop on sessions that already accumulated many summary messages.

I reproduced it on a real session after applying the equivalent change locally: 139 summary system messages (~1.25M chars), a new summary every 30–60 seconds, and later summaries degenerating into "transcript truncated / no user turns".

The loop is in compactSession() on current main: startIndex is the first non-system message, so summary system messages before it are never included in a later summary prompt and never compacted. The main request still includes them; if activeTokens stays above autoCompactWindow, the next iteration compacts again and appends another summary.

I opened #383 with the evidence, a minimal reproduction, and suggested guards (merge/prune summaries, include them in the compactable range, or stop after one compaction when activeTokens did not drop). No code changes in this PR — keeping it image-only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant