fix(agent): harden evolution evidence and runtime by Pepps233 · Pull Request #108 · diffexai/diffex

Pepps233 · 2026-08-22T06:18:25Z

Summary

Hardens evolution evidence relevance and reflection execution with semantic lifecycle filtering, durable exclusion accounting, bounded provider and node runtime, task lineage, actionable curator validation, and end-to-end lifecycle coverage.

Reasoning

Routine startup, shutdown, resume, and provider-switch activity could create trace evidence and affect evolution thresholds even when no user task occurred.

Substantive sessions ending in shutdown also needed their task evidence retained while trailing lifecycle operations were excluded and auditable cursors continued to advance.

Recursive reflection could multiply internal provider rounds, overproduce lessons, and stall without complete budgets or deadlines, while proposals lacked enough task lineage and relevance validation to distinguish actionable harness memory from observer documentation.

Changes

Adds semantic evidence eligibility, lazy trace creation, shutdown-tail stripping, and diagnostic-only lifecycle failure retention.
Persists exclusion decisions, consumes contiguous excluded ranges, and aligns thresholds and progress totals with eligible evidence only.
Separates leaf, synthesis, and final lesson limits while accounting for every internal provider round and node-local tool or refinement budget.
Adds provider, node, and stage deadlines with resumable timeout checkpoints and durable runtime metrics.
Runs independent leaf reflection with deterministic bounded concurrency and node-local provider-call identities.
Propagates stable prompt lineage and requires curator relevance certificates that reject observer-only proposals.
Verifies startup, resume, provider switching, exit-only sessions, substantive shutdown, failed turns, and mixed eligible and excluded evidence ranges.

Testing

npm run check
node ../../node_modules/vitest/dist/cli.js --run test/invocation-tracing.test.ts test/evolution-runtime.test.ts from packages/coding-agent
node ../../node_modules/vitest/dist/cli.js --run test/evidence-eligibility.test.ts test/control-plane.test.ts test/reflector.test.ts test/curator.test.ts from packages/evolution
node ../../node_modules/vitest/dist/cli.js --run test/materializer.test.ts test/harness-trace.test.ts from packages/harness-trace
./test.sh with all workspaces passing

- Keeps startup and shutdown metadata out of evolution inputs - Preserves task messages, tool results, and substantive operations

- Locks idle abort operations out of evolution eligibility - Prevents interface shutdown paths from becoming learned behavior

- Keeps substantive task evidence eligible when shutdown follows - Prevents abort spans and routine footers from reaching reflection

- Buffers session and harness metadata until substantive work occurs - Avoids raw files for routine startup and shutdown sessions

- Keeps standalone runtime failures in diagnostic traces only - Admits error context when it accompanies substantive task evidence

- Records excluded trace identities and semantic reasons atomically - Keeps diagnostic sources auditable without copying them into prompts

- Advances the durable cursor across contiguous excluded evidence - Prevents lifecycle-only files from being reconsidered on every check

- Uses one eligible set for overview counts and token estimates - Freezes the same sources for downstream reflection

- Limits each evidence leaf to three behavior-changing lessons - Reinforces the bound in prompts, schema, and response parsing

- Caps intermediate synthesis nodes at five lessons - Allows up to ten durable final lessons for curation

- Charges each underlying reflector provider call to the durable budget - Bounds internal tool-use loops at ten provider rounds

- Isolates evidence and lesson-inspection budgets per node - Separates two refinement rounds from response correction retries

- Stop stalled provider streams at explicit connection, idle, and absolute deadlines. - Preserve timed-out node checkpoints and incomplete evidence coverage without blocking independent work.

- Capture provider latency, output usage, tool rounds, retries, node duration, and foreground waits. - Store measurements with durable node checkpoints for accurate evolution cost accounting.

- Run independent leaf nodes with a fixed concurrency of two. - Derive provider-call identities from node-local counters so scheduling cannot alter checkpoints.

- Require each curator decision to identify a task, causal behavior, concrete change, and verifiable result. - Exclude observer-only and telemetry-format facts while consolidating behavior-changing lessons.

- Assign every prompt operation a stable task-lineage identifier. - Carry lineage through session segments and materialized evidence so proposal support remains independently countable.

- Exercise exit-only, mixed-range, substantive-shutdown, and failed-turn behavior across tracing and evolution. - Remove completed traces that contain no substantive payload while preserving unclassifiable diagnostics.

- Require substantive messages or failed operations before evolution - Keep lifecycle spans only as context for anchored work

- Delete complete non-substantive traces after clean shutdown - Preserve aborted and incomplete traces for diagnostics while excluding them from evolution

- Allows the first validated mutation batch to initialize a blank EVOLVE.md - Preserves strict destination validation once memory exists

- Keeps model-authored memory concise with a three-sentence validation boundary - Assigns monotonic numbers at persistence so the model cannot choose them

- Gives every session one content-addressed source for the next lesson number - Keeps active EVOLVE.md lessons contiguous after removals without model involvement

- Makes metadata-bearing revisions unambiguous before release - Covers strict schema rejection and repeated live switching

- Prevents UTF-8 encoding from changing content-addressed harness state - Keeps repeated saves deterministic by rejecting lone surrogate code units

Pepps233 · 2026-08-23T16:22:42Z

#93 (review) resolved by d2dc737

- Lets users persistently disable self-evolution from the interactive menu - Prevents disabled sessions from running evolution or retaining invocation traces

- Select the default agent at launch and scope /version to the current directory - Replace raw IDs with reusable semantic harness, job, proposal, and evidence labels

Pepps233 added 18 commits August 22, 2026 01:26

fix(agent): exclude lifecycle-only traces

e1e19bd

- Keeps startup and shutdown metadata out of evolution inputs - Preserves task messages, tool results, and substantive operations

fix(agent): reject shutdown evidence

4cd9fb9

- Locks idle abort operations out of evolution eligibility - Prevents interface shutdown paths from becoming learned behavior

fix(agent): strip shutdown trace metadata

072e03c

- Keeps substantive task evidence eligible when shutdown follows - Prevents abort spans and routine footers from reaching reflection

fix(agent): create traces on demand

0cfd9d2

- Buffers session and harness metadata until substantive work occurs - Avoids raw files for routine startup and shutdown sessions

fix(agent): scope lifecycle diagnostics

a57eae0

- Keeps standalone runtime failures in diagnostic traces only - Admits error context when it accompanies substantive task evidence

fix(agent): persist evidence exclusions

d9e7bfb

- Records excluded trace identities and semantic reasons atomically - Keeps diagnostic sources auditable without copying them into prompts

fix(agent): consume excluded trace ranges

3ab87ad

- Advances the durable cursor across contiguous excluded evidence - Prevents lifecycle-only files from being reconsidered on every check

fix(agent): align eligible evidence totals

6cc2c49

- Uses one eligible set for overview counts and token estimates - Freezes the same sources for downstream reflection

fix(agent): cap leaf reflection lessons

7e8b619

- Limits each evidence leaf to three behavior-changing lessons - Reinforces the bound in prompts, schema, and response parsing

fix(agent): separate lesson limits

75cbb30

- Caps intermediate synthesis nodes at five lessons - Allows up to ten durable final lessons for curation

fix(agent): account provider rounds

0096ceb

- Charges each underlying reflector provider call to the durable budget - Bounds internal tool-use loops at ten provider rounds

fix(agent): bound reflector nodes

47c4939

- Isolates evidence and lesson-inspection budgets per node - Separates two refinement rounds from response correction retries

fix(agent): bound reflection runtime

926b1ee

- Stop stalled provider streams at explicit connection, idle, and absolute deadlines. - Preserve timed-out node checkpoints and incomplete evidence coverage without blocking independent work.

fix(agent): persist reflector runtime

7989a8f

- Capture provider latency, output usage, tool rounds, retries, node duration, and foreground waits. - Store measurements with durable node checkpoints for accurate evolution cost accounting.

fix(agent): parallelize leaf reflection

53056aa

- Run independent leaf nodes with a fixed concurrency of two. - Derive provider-call identities from node-local counters so scheduling cannot alter checkpoints.

fix(agent): require actionable evolution

e16c6d0

- Require each curator decision to identify a task, causal behavior, concrete change, and verifiable result. - Exclude observer-only and telemetry-format facts while consolidating behavior-changing lessons.

fix(agent): propagate prompt lineage

6356080

- Assign every prompt operation a stable task-lineage identifier. - Carry lineage through session segments and materialized evidence so proposal support remains independently countable.

fix(agent): cover lifecycle evolution flows

dd583e0

- Exercise exit-only, mixed-range, substantive-shutdown, and failed-turn behavior across tracing and evolution. - Remove completed traces that contain no substantive payload while preserving unclassifiable diagnostics.

Pepps233 self-assigned this Aug 22, 2026

Pepps233 requested a review from Issacboy123 August 22, 2026 13:24

Pepps233 added the evolution-loop label Aug 22, 2026

Pepps233 added 7 commits August 22, 2026 12:08

fix(agent): anchor evolution evidence

4a744a1

- Require substantive messages or failed operations before evolution - Keep lifecycle spans only as context for anchored work

fix(coding-agent): exclude lifecycle-only traces

6f1095c

- Delete complete non-substantive traces after clean shutdown - Preserve aborted and incomplete traces for diagnostics while excluding them from evolution

fix(agent): bootstrap empty evolution memory

8ab3ba3

- Allows the first validated mutation batch to initialize a blank EVOLVE.md - Preserves strict destination validation once memory exists

feat(agent): number evolution lessons

4a2a747

- Keeps model-authored memory concise with a three-sentence validation boundary - Assigns monotonic numbers at persistence so the model cannot choose them

feat(agent): compact lesson numbering

bd779f6

- Gives every session one content-addressed source for the next lesson number - Keeps active EVOLVE.md lessons contiguous after removals without model involvement

fix(agent): version harness manifests

dc35252

- Makes metadata-bearing revisions unambiguous before release - Covers strict schema rejection and repeated live switching

fix(agent): reject malformed UTF-16

d2dc737

- Prevents UTF-8 encoding from changing content-addressed harness state - Keeps repeated saves deterministic by rejecting lone surrogate code units

feat(coding-agent): add evolution toggle

fff7c67

- Lets users persistently disable self-evolution from the interactive menu - Prevents disabled sessions from running evolution or retaining invocation traces

feat(agent): clarify evolution identities

332eed4

- Select the default agent at launch and scope /version to the current directory - Replace raw IDs with reusable semantic harness, job, proposal, and evidence labels

Pepps233 merged commit f0f9d12 into main Aug 23, 2026

Navigation Menu

Repository navigation

fix(agent): harden evolution evidence and runtime - #108

fix(agent): harden evolution evidence and runtime#108
Pepps233 merged 27 commits into
fix/evolve-unprocessed-materialized-tokensfrom
feat/evolution-relevance-and-runtime-bounds

Pepps233 commented Aug 22, 2026 •

edited

Loading

Uh oh!

Pepps233 commented Aug 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Pepps233 commented Aug 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Reasoning

Changes

Testing

Uh oh!

Pepps233 commented Aug 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Pepps233 commented Aug 22, 2026 •

edited

Loading