Repository navigation
Keep Turns running when event text contains NUL - #817
Merged
Merged
Conversation
PostgreSQL jsonb rejects \u0000, so any Harness output with a NUL byte, such as `grep -a` or `/proc/*/cmdline` command output, failed the whole journal batch and the Turn with event_persistence_failed. The journal now stores U+FFFD in place of U+0000 in every string of an observation and of the execution outcome, as decoding already does for invalid UTF-8, and a failed journal write is logged with its Session, Turn and first ordinal.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PostgreSQL
jsonbcannot hold\u0000. When a Harness emits output with a NUL byte,InsertTurnEventBatchrejects the whole batch (SQLSTATE 22P05), and the Turn fails withevent_persistence_failedafter the Harness has already finished. Common commands trigger it, includinggrep -a,caton a binary file, reading/proc/*/cmdlineandfind -print0. It reproduces with Codex and Claude Code. Core logged nothing.Change
sessions.journalPayloadnormalizes each journal observation and the execution outcome. It replaces U+0000 with U+FFFD in every string and key, as decoding already does for invalid UTF-8. Payloads without\u0000keep the existing single pass.execution.journal.flushlogs a failed journal write with its Session, Turn, first ordinal and event count. It does not log the raw driver error, which can contain submitted values.Checks
TestTurnJournalStoresNULInObservedTextAsReplacementCharacter(PostgreSQL). It fails on main withunsupported Unicode escape sequence (SQLSTATE 22P05)and passes with this change.sessions,executionandsessionpg(journal and execution), along withgo vet ./...andmake check-docs.Need help on this PR? Tag
@codesmith-botwith what you need.