Retire the last nine cadence claims, and say the gap got worse - #90
Merged
Merged
Conversation
The second review on #86 found nine statements still describing a weekly benchmark and a monthly matrix as current. Six were prose; three were source comments justifying behaviour that still exists for a different reason. Two of them matter beyond tidiness, and both are now honest rather than merely updated. `reference/design-tokens.md` and the page brief said cells in one grid can be "up to four weeks" apart, with a weekly against a monthly as the ceiling. That ceiling is gone. A dispatched benchmark means a column is as old as the last run that covered it, so the problem the design has no treatment for is larger than it was, not resolved. Saying "no bound" is the point; quietly deleting the number would have hidden it. The three source comments justified `--merge` and `ranAt` by the twins refreshing monthly. The justification survives the cadence: a dispatched run covers the experiments that run asked for, so a partial run's snapshot still has holes without the merge, and two cells side by side can still be far apart. The plan's triggers table listed two benchmark crons; it now lists the regression cron and dispatch-against-a-bucket. Its "what the schedule actually runs" table is kept as the record of 31 August to 28 September, labelled as such, because the costs below it are measurements of that period. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MQzUoMAwEBJWpEGVvVzSjK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows the second review on #86, which listed nine statements still describing a weekly benchmark and a monthly matrix as current fact.
reference/design-tokens.md-no-skillstwins run monthly".plans/evals-page-brief.md.plans/delivery-plan.mdschedulerows; "eval-refreshruns on two schedules"apps/framework/lib/provenance.ts-no-skillstwins refresh monthly and everything else weekly"apps/framework/harness/run-eval.tsapps/framework/scripts/export-results.tsTwo of these are not tidying
design-tokens.mdand the page brief used the cadence to bound a real problem: cells in one grid are different ages, and a weekly against a monthly capped the gap at four weeks. That ceiling is gone. A dispatched benchmark means a column is as old as the last run that covered it.So both now say there is no bound, and that the problem is worse rather than resolved. Deleting the number quietly would have hidden a design gap that the page still has no treatment for.
The three source comments
They justified
--mergeand the per-rowranAtby the twins refreshing monthly. The justification survives the change for a different reason, which is what they now say: a dispatched run covers only the experiments it asked for, so a partial run's snapshot still has holes without the merge.Kept deliberately
The plan's "what the schedule actually runs" table stays, relabelled as the record of 31 August to 28 September, because the cost measurements below it are measurements of that period and would be unreadable without it.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MQzUoMAwEBJWpEGVvVzSjK