Repository navigation
changelog: stop when a Claude call fails - #2893
Merged
Merged
Conversation
generate-changelog-input.sh ran every claude call as "$(claude ... 2>&1) || true". When a call fails (an expired login, an invalid key, a model that is not available), the CLI exits with status 1 and prints its error message to stdout, so the message became the generated text and the script went on. The later safety nets then hid the failure: the reference check found no references and, after a second failing call, built the entry from the raw commit subjects; the stray-line filter removed the error message. With --pr the result was committed and opened as a pull request, and the run reported success. For multi-batch tags the entry also lacked its version header, so it ended up as part of the previous version in CHANGELOG.md. All calls now go through run_claude, which treats a non-zero exit status as failure, reports the output on stderr and makes the script exit. Matching the CLI's error messages is not needed; its exit status is reliable. Two checks on the generated entry make the script exit with an error as well, for output that comes back with exit status 0 but is not a changelog entry: - The merged entry has no version header. Before, the output was used as is. - More than half of the expected references are still missing after the fix attempt. Before, the direct reference repair filled them in; it is meant to patch a few references the model dropped, not to replace the entry with the commit subjects. A failing run used to leave the entries of the tags processed so far in CHANGELOG.md, which a later --auto run takes as done. An exit trap now restores CHANGELOG.md to its state before the run whenever the script fails before the commit. It stops at the commit because restoring the file after it would leave the working tree out of step with the commit. The new tests run the script against a repository with a fake claude on PATH; four of them fail without this change. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Roger Luethi <luethi@osism.tech>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
generate-changelog-input.shignored failedclaudecalls (|| true). Because the CLI prints its error message to stdout, that message became the generated text, and the script's later safety nets covered it up: the direct reference repair built the entry from the raw commit subjects, and the stray-line filter removed the error text.In the changelog round for 11.0.3 the
claudelogin had expired. All 15 component runs with--auto --pr --tags-onlyreportedokand opened PRs that held a raw commit list. For multi-batch repos the version heading was missing as well, for example:Changes
run_claude, which treats a non-zero exit status as failure and makes the script exit.CHANGELOG.mdto its state before the run when the script fails before the commit, so an aborted multi-tag run leaves no partial entries for a later--autorun to take as done.Why the exit status and not the error text
I tested
claude2.1.322 with-p. A missing login, an invalid API key, an invalid OAuth token (401) and an unknown model all exit 1, and a successful call exits 0. In every failure case the error went to stdout and stderr was empty. Matching error messages would be fragile, and the exit status makes it unnecessary. I could not reproduce the exact 11.0.3 message ("this login reached its time limit and cannot be renewed") without breaking a real login; the 401 case is the closest one I could test.A preflight call is not needed either: nothing is committed before the first real call, and an auth failure returns within seconds.
Testing
tests/test_generate_changelog_input.pyruns the script against a test repository with fakeclaudeandghcommands onPATH. It covers a successful run, an auth failure, a failure on a later tag (restore), a failure while writing the commit message with--commit, a reply without references, and--no-run. Four of the six tests fail againstmain. The full suite passes.🤖 Generated with Claude Code