Skip to content

Keep tail-transcript -f from dropping a line caught mid-write - #259

Merged
dmccoystephenson merged 2 commits into
mainfrom
fix/transcript-follow-partial-lines
Nov 15, 2026
Merged

dmccoystephenson merged 2 commits into
mainfrom
fix/transcript-follow-partial-lines

Conversation

@dmccoystephenson

Copy link
Copy Markdown
Member

Summary

  • iter_pretty_lines (gardener/transcript.py), the reader behind gardener tail-transcript, no longer drops a transcript line caught mid-write under -f. At EOF readline() returns a partial line without its newline. That fragment, and its tail on the next poll, were each parsed as JSON, failed, and were skipped, so the event was lost silently. An unterminated tail is now held and joined to the next read, the same whole-lines rule live.py's TranscriptCache applies to the same files. Without -f, a final line with no newline is still parsed.
  • The file is now opened with encoding="utf-8", errors="replace" instead of the locale's encoding, matching live.py, dashboard.py, run_log.py and trace_client.py. Claude Code writes UTF-8 whatever the locale is.
  • print_transcript catches OSError from a file that exists but can't be read, and returns 1 with a one-line gardener: error instead of a traceback. BrokenPipeError is an OSError subclass, but it is still caught first by the existing clause and keeps exiting 0.
  • docs/TESTING.md's test_transcript.py paragraph describes the new coverage.

The lost-event case was found by reading the code, not observed in a real transcript. The guard live.py already carries for these files shows the case had been judged worth handling once before.

Closes #258

Test plan

  • python3 -m unittest tests.test_transcript: 49 tests pass locally.
  • Verify by removal: the fix was committed first, then git checkout origin/main -- gardener/transcript.py was run, and a Grep for the fix's pending token returned nothing. Three of the four new tests then failed (mid-write join, UTF-8 decode, unreadable file). test_dump_mode_still_parses_a_final_line_with_no_newline passes on both sides because it is a regression guard for dump mode, not a test of the fix. After git checkout HEAD -- gardener/transcript.py, git status --porcelain was empty.
  • python3 -m compileall gardener
  • Full tests/ and integration/ offline runs: the sandbox's python3 is 3.8, below requires-python >=3.10, so CI on this PR is the anchor of record (see the checks below).
  • Manual dispatch verification is not required: dispatch.py, dev_loop.py and the prompt templates are untouched.

This PR description was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).

🤖 Generated with Claude Code


drafted by Claude on behalf of Daniel Stephenson

dmccoystephenson and others added 2 commits November 14, 2026 23:36
Follow mode now holds an unterminated tail until its newline arrives,
the file is decoded as UTF-8 regardless of locale, and an unreadable
file gets a one-line error instead of a traceback.

Closes #258

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dmccoystephenson

Copy link
Copy Markdown
Member Author

Self-review: no blocking issues found.

The code-review skill was run on this diff and reported no correctness findings. A hand review against this repo's conventions followed:

  • gardener/transcript.py:316 (follow branch): an unterminated tail makes the loop continue, the next readline() returns "" at EOF, and the loop sleeps as before. No busy loop is added. Without -f, pending is consumed on every chunk, so a final line with no newline is still parsed.
  • gardener/transcript.py:316: known limit, not a defect. If a followed file ends for good on an unterminated line, -f never prints that line. This matches live.py's TranscriptCache, which also consumes only whole lines, and a dump without -f still prints it.
  • gardener/transcript.py:345: the new except OSError follows the existing (KeyboardInterrupt, BrokenPipeError) clause. BrokenPipeError is an OSError subclass, so it still exits 0.
  • tests/test_transcript.py: the UTF-8 test simulates a non-UTF-8 locale by wrapping Path.open instead of patching locale.getpreferredencoding, because on Python 3.11+ open() resolves the locale encoding in C and would ignore that patch. Each test that should fail without the fix did fail once gardener/transcript.py was reverted to origin/main.
  • Scope against tail-transcript -f drops a transcript line caught mid-write and decodes with the locale's encoding #258: all three gaps the issue names (mid-write line, locale encoding, unreadable-file traceback) are covered. live.py was re-read to confirm the whole-lines rule the issue cites.
  • Safety invariants: dispatch.py, dev_loop.py, MODE_SPECS and the prompt templates are untouched, so no manual dispatch verification is owed. No dependencies were added (stdlib only).
  • Docs: docs/TESTING.md is updated. docs/USAGE.md's -f description ("keep reading as the file grows, like tail -f") is still accurate.
  • CI on the PR head: test (3.10), test (3.x), city-still, fleetcity, garden3d all pass. This is the anchor of record, because the local python3 is 3.8.

This review was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

@dmccoystephenson

Copy link
Copy Markdown
Member Author

Correction to the self-review above: two line references are off. The follow-branch notes refer to gardener/transcript.py:320 (not :316), and the except OSError note refers to gardener/transcript.py:348 (not :345). The findings themselves are unchanged.

This comment was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

@dmccoystephenson
dmccoystephenson merged commit 49a15b8 into main Nov 15, 2026
5 checks passed
@dmccoystephenson
dmccoystephenson deleted the fix/transcript-follow-partial-lines branch November 15, 2026 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tail-transcript -f drops a transcript line caught mid-write and decodes with the locale's encoding

1 participant