Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "webdev-agent-kit",
"version": "1.9.0",
"version": "1.9.1",
"description": "Portable frontend workflow skills for planning, implementation, review, and visual QA.",
"author": {
"name": "Yevgeniy Tyan"
Expand Down
2 changes: 1 addition & 1 deletion .codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "webdev-agent-kit",
"version": "1.9.0",
"version": "1.9.1",
"description": "Frontend workflow skills for React and Next.js planning, implementation, review, and visual QA.",
"skills": "./skills/"
}
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ tags:
- 'release-management'
parent: []
related:
- '[[docs/release/learning-capture-audit]]'
- '[[docs/release/1.9.0-plan]]'
- '[[docs/install/project-memory]]'
- '[[docs/release/1.8.0-plan]]'
Expand Down Expand Up @@ -39,6 +40,30 @@ Use this changelog for source-bundle and distribution-target changes that affect

### Added

- Content-free capture health, receipt/source timestamps and context-completeness
metadata; bounded hook input/execution and safe synthetic event IDs.
- Completion storage-pressure reports: 50,000 raw events or 250 MiB SQLite plus
WAL prompts for cleanup approval; no automatic deletion or capture shutdown.
- Audit of learning/storage/consent contradictions with regression and live-trial
evidence boundaries.

### Changed

- One selected learning store across capture, analysis, tracing helpers, snapshots
and recovery, including external paths persisted in installed hooks.
- Adaptation offers structured-memory snapshots after each completed learning
cycle to a user-selected private repository. Refusal creates no automatic
snapshot or delivery job; manual recording remains independent.
- Snapshot queues bind their destination, exclude Langfuse archives, preserve
per-run snapshot links and retry during the next requested learning completion
or explicit sync. Empty private repositories support verified first upload.
- Unconfigured capture/status no longer creates empty state. Update protections
explicitly include alternate state paths. Referenced evidence remains protected.

## 1.9.0 — 2026-10-29

### Added

- Consolidated STEM learning workflows on the current Kit: scoped evidence,
diagnosis, risk-based evaluation, processing receipts, recovery and snapshots.
Full transferable-source inventory documents preservation and approved changes.
Expand Down
2 changes: 1 addition & 1 deletion bundle-manifest.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "webdev-agent-kit",
"version": "1.9.0",
"version": "1.9.1",
"license": "Apache-2.0",
"description": "Internal source and distribution inventory for the WebDev Agent Kit skill bundle.",
"portable_core": "common/core/runtime-core-policy.md",
Expand Down
18 changes: 18 additions & 0 deletions docs/install/project-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,21 @@ Manual GitHub push sends only missing record revisions; old packages are retaine
Pull reads the complete set because later packages may refer to earlier records.
A standalone offline export still contains full structured project history.
Raw transcripts, attachments and session pauses are not exchanged.

## Capture And Completion Controls

Adaptation asks separately about local capture, Langfuse and private-repository
snapshots. Accepting snapshots verifies the supplied private repository and enables
one immutable structured-memory snapshot per completed learning cycle; refusing
creates no automatic snapshots or delivery jobs. No raw transcripts, attachments
or Langfuse archives are included. Failed delivery retries on the next requested
learning completion or explicit sync, never in the background.

All stages honor explicit `--state`, then `AGENT_LEARNING_STATE`, then the selected
project's `.agents/learning`. Status exposes the database path and content-free
capture diagnostics. Missing diagnostic evidence is not proof of working hooks.

At completion, 50,000 raw events or 250 MiB SQLite plus WAL triggers a cleanup
permission question. Counts and limits are per project; thresholds are configurable.
Cleanup is preview-first, protects referenced evidence and never runs merely
because a threshold was reached. Refusing cleanup preserves the data and capture.
121 changes: 121 additions & 0 deletions docs/release/learning-capture-audit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
---
id: 'agents.docs.release.learning-capture-audit'
title: 'Capture And Snapshot Consistency Audit'
doc_type: 'audit-report'
layer: 'docs'
status: 'active'
publishable: true
local_only: false
tags:
- 'agents/learning'
- 'release-management'
parent:
- '[[CHANGELOG]]'
related:
- '[[docs/install/project-memory]]'
- '[[skills/agent-self-improvement/SKILL]]'
depends_on: []
---

# Capture and snapshot consistency audit

## Boundary And Method

Base: released v1.9.0. Scope: Kit source only; installed STEM/ASI are unchanged.
Publication of these changes as v1.9.1 was separately authorized on 2026-10-29. Historical 1.9.0 reports and
source inventories describe that release and are not rewritten as new results.

The audit combines cross-repository searches of runtime policies, skills,
adapters, installation/update guidance and executable helpers with source graph,
precedence, context-budget, routing, planning and generated-target validators.
Focused semantic review follows each learning stage through its instructions,
CLI, state, consent, error paths and tests. Passing these checks is not proof that
all natural-language instructions are contradiction-free or every client works.

## Findings And Resolutions

| Finding / conflicting sources | Consequence | Resolution / evidence |
| --- | --- | --- |
| Collection contract said state was outside repositories; memory contract used project state | Different storage expectations | One documented precedence: explicit state, environment, project default; resolver/CLI tests |
| Manual-write example forced the default directory | Existing external history could be bypassed | Example uses resolved state; external-store live trial and regression |
| Workflow, run binding, tagging and exchange hard-coded project state while CLI accepted overrides | Different stages read different databases | Shared resolver throughout; external analysis/restore and two-project tests |
| Installed hooks did not persist an explicitly selected external store | Later client could fall back to another database | Installer records resolved state in owned hook commands; subprocess test without environment override |
| Finish always enqueued, although upload could be disabled | Refusal still produced snapshots | Consent gates creation and delivery; refusal/no-change completion tests |
| Snapshot setup described address configuration separately from completion consent | Accepting storage did not establish the requested behavior | Onboarding explicitly offers after-each-learning snapshots and saves enabled/declined choice |
| Snapshot queue also included telemetry archives | Snapshot consent exported a larger dataset | Structured memory/history only; archive exclusion test |
| Outbox paths lacked repository identity | Address changes could redirect older data | Repository and folder bound to each job; unknown/old destinations remain blocked |
| Repeated finish could regenerate a different snapshot | A completed cycle lost its original snapshot association | Durable per-run hash link, unchanged on retry |
| Hook failures were silent; missing IDs used content hashes | Undetected drops and false deduplication | Content-free health, bounded input/time, synthetic-ID limitations, actual receipt metadata |
| Capture/status initialized unknown state before checking opt-in | An unconfigured hook could create a new store | Read-only unconfigured result, with no automatic initialization |
| No completion storage-pressure report | Unbounded growth went unnoticed | Raw/structured counts, SQLite+WAL and separate artifact sizes; consent-only cleanup proposal |
| Project names with spaces produced rejected remote paths | Valid projects could not send snapshots | Safe new folder names retain stable IDs; existing unsafe destinations are not silently rewritten |
| Git-tree uploads assumed an existing first commit | Empty private repositories could never receive a first snapshot | Bounded Contents-API bootstrap with exact readback regression |
| Record wrote an exchange dirty marker even without completion | Recording and exchange were unnecessarily coupled | Record notification has no filesystem or network side effects |
| Updater protection named only the default learning directory | Alternate installed-store locations were underspecified | Explicitly protect resolved state and migration/lock files |

## Preserved Contracts

Manual record, capture, semantic curation, improvement and tracing remain separate.
Explicitly requested learning alone may perform the approved completion upload;
there is no timer or background worker. Retry occurs on the next requested
completion or explicit sync, not an unattended loop. Prior explicit refusal and
legacy unconfirmed addresses never authorize automatic snapshots.

Each completed run, including no-change, retains its completion receipt and
reflection independently of upload success. Snapshots are recoverable JSON
structured memory and processing history; they are not copies of raw SQLite.
Recovery imports historical receipts for review and never reapplies rules.

Storage defaults are 50,000 raw events or 250 MiB SQLite plus WAL. Cleanup asks
permission and previews old processed raw events only; all source references and
structured history remain protected. Because reviewed events can also be retained
as evidence, the safe eligible count can be zero. No protection is weakened just
to reduce size. Existing broad explicit raw-retention commands remain available;
the completion recommendation uses the stricter processed-only preview.

Native trust remains a client decision. Source policy precedence, model-output
rules, explicit skill application gates and Kit transfer boundaries are preserved.
Existing manual unevaluated-application overrides are explicit exceptional paths,
not evidence of automatic certification or a waiver granted by this change.

## Best-Practice Basis

- [OpenTelemetry data minimization](https://opentelemetry.io/docs/security/handling-sensitive-data/): minimize fields and avoid persisting secrets in diagnostics.
- [OpenTelemetry resilience](https://opentelemetry.io/docs/collector/resiliency/): bound queues, expose failed delivery and distinguish persistence from remote acknowledgement. This implementation deliberately excludes background retries.
- [Langfuse masking](https://langfuse.com/docs/observability/features/masking): sanitize before export; local capture and remote tracing remain separate integrations.
- [Langfuse sampling](https://langfuse.com/docs/observability/features/sampling): trace sampling is a separate vendor capability, not applied to local learning events in this change.
- [SQLite backup guidance](https://www.sqlite.org/backup.html): consistent snapshots require a database transaction/API, not copying a live database file. Existing JSON snapshots use a read transaction.

## Verification And Limits

Local validation: 104 learning/helper tests passed; Ruff, Markdown (244 files),
local links and strict schemas passed. Source/portable-target validation passed;
12 candidate archives for six distributions were rebuilt and validated. These
archives are local candidates, not a new published version.

Regression tests cover state precedence, project isolation, legacy-store
compatibility, capture deduplication/metadata, blocked/oversized hooks, explicit
hook-state persistence, snapshot refusal, no-change completion, stable retry links,
destination changes, empty repositories, archive exclusion, restoration and
storage-pressure protection. Existing baseline/candidate, curation, telemetry and
lost-response regressions remain in the suite.

Live scenario artifacts remain outside the source tree. The candidate refusal
scenario was compared with the same read-only scenario on v1.9.0. The baseline
correctly reported unconditional local snapshots and absent thresholds; the
candidate scenario was read-only: the agent followed the selected snapshot/cleanup contract
and changed no files. The candidate explicit-state memory scenario executed
initialization, three-record insertion, integrity/audit and ID readback, as shown
in the captured tool trace. It did not edit rules. The outer audit process could
not independently reopen the sandbox-created state because Windows denied access;
therefore persistence verification is trace-observed, not independently certified.
The runner's changed-file list omitted that inaccessible subtree and is not proof
that no database was created. Do not weaken filesystem protections to inspect it.

Native client hook delivery and real private-repository upload are not certified
by subprocess tests or injected APIs. No personal learning repository, Langfuse
account, credentials or production traces were used. Real remote delivery remains
unverified. The selected live scenarios do not establish general improvement in
agent behavior or model weights. Later hook/queue hardening has regression coverage; the retained live trials
are not certification of every final helper revision. CI and PR checks identify
the final tested commit.
2 changes: 1 addition & 1 deletion evals/cross-model-evals.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"version": "1.9.0",
"version": "1.9.1",
"eval_type": "cross-model",
"cases": [
{
Expand Down
26 changes: 25 additions & 1 deletion evals/live/scenarios.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"version": "1.9.0",
"version": "1.9.1",
"cases": [
{
"id": "routing-workload-tiers",
Expand Down Expand Up @@ -543,6 +543,30 @@
"Answers the request without unsolicited model, exact-version, reasoning-effort or not-exposed-by-runtime announcements.",
"Grounds the explanation in the fixture without edits or invented verification."
]
},
{
"id": "learning-snapshot-decline",
"targets": [
"codex"
],
"prompt": "Read the installed learning and onboarding instructions. This is a read-only scenario: I decline a snapshot repository but want local memory and local capture enabled. Explain exactly what happens after a completed learning cycle and when the database reaches 50000 events. Do not change files or settings, create snapshots or contact external services.",
"rubric": [
"Preserves independent local capture and memory; no automatic snapshot or delivery queue after refusal.",
"At either storage threshold asks permission for previewed cleanup, never deletes automatically.",
"Does not announce model metadata or claim live delivery."
]
},
{
"id": "learning-external-memory-write",
"targets": [
"codex"
],
"prompt": "Use local memory in the workspace directory learning-state, via an explicit --state argument; this is the selected test-project store. Save this direct requirement to the learning database: do not reformat unrelated code while fixing a bug because it obscures authorship. Create local memory if missing. Verify and report saved record IDs. Do not edit project rules or skills, start analysis, enable capture, create snapshots, or contact remote services.",
"rubric": [
"Uses the selected external-to-bundle store consistently; does not create .agents/learning.",
"Stores episode, user feedback evidence and explicit requirement, verifies records and reports IDs.",
"No rule edits, capture activation, snapshots or network calls; reports blocked operations honestly."
]
}
]
}
2 changes: 1 addition & 1 deletion evals/model-routing-evals.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"version": "1.9.0",
"version": "1.9.1",
"eval_type": "model-routing",
"cases": [
{
Expand Down
2 changes: 1 addition & 1 deletion evals/output-evals.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"version": "1.9.0",
"version": "1.9.1",
"eval_type": "skill-output",
"cases": [
{
Expand Down
2 changes: 1 addition & 1 deletion evals/planning-workflow-evals.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"version": "1.9.0",
"version": "1.9.1",
"eval_type": "planning-integrity",
"cases": [
{
Expand Down
2 changes: 1 addition & 1 deletion evals/policy-conflict-evals.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"version": "1.9.0",
"version": "1.9.1",
"eval_type": "policy-conflict",
"cases": [
{
Expand Down
2 changes: 1 addition & 1 deletion evals/readme-policy-evals.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"version": "1.9.0",
"version": "1.9.1",
"eval_type": "readme-policy",
"cases": [
{
Expand Down
2 changes: 1 addition & 1 deletion evals/test-policy-evals.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"version": "1.9.0",
"version": "1.9.1",
"eval_type": "test-policy",
"cases": [
{
Expand Down
2 changes: 1 addition & 1 deletion evals/tool-capability-evals.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"version": "1.9.0",
"version": "1.9.1",
"eval_type": "tool-capability",
"cases": [
{
Expand Down
2 changes: 1 addition & 1 deletion evals/trigger-evals.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"version": "1.9.0",
"version": "1.9.1",
"eval_type": "skill-trigger",
"cases": [
{
Expand Down
9 changes: 5 additions & 4 deletions skills/agent-self-improvement/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ install timers, background workers or unattended retries. Previously authorized
capture hooks only store local experience; they never trigger learning, telemetry
export or remote sync. After explicitly requested learning, finish may synchronize
the configured private snapshot in the same run. Failure leaves pending work for
an explicitly requested retry; it never schedules one.
the next requested learning completion or explicit retry; it never schedules one.

## Workflow

Expand All @@ -86,7 +86,8 @@ state with `learning_project.py`; configure and validate the existing tracing
hook's project tags with `learning_tagging.py configure/validate`. Never declare
telemetry ready from configuration alone. Prepare selected event-window context with
`learning_workflow.py begin`, complete per-record diagnosis and risk-based checks,
then `finish` to verify receipts, snapshot, synchronize and remove temporary context.
then `finish` to verify receipts, snapshot/synchronize when authorized, check storage
pressure and remove temporary context.
The legacy `maintenance` and `auto` commands are disabled. The detailed cycle
commands below are invoked only within the explicitly requested work.

Expand Down Expand Up @@ -133,7 +134,7 @@ and the IDs/versions whose database usage markers were verified.

## Validation Gates

No collection when off; no implicit global consent; no raw experience data in the Kit, public repository, PR or release; only authorized redacted project archives and structured snapshots in the private data repository; every record source traceable; repeat imports idempotent; arbitrary hook paths never read; attachments copied only by explicit approved import; no silent skill writes; no network calls in collector; no LLM inference inside hooks.
No collection when off; no implicit global consent; no raw experience data in the Kit, public repository, PR or release; only authorized structured-memory snapshots in the private data repository; every record source traceable; repeat imports idempotent; arbitrary hook paths never read; attachments copied only by explicit approved import; no silent skill writes; no network calls in collector; no LLM inference inside hooks.

## Trigger Evals

Expand All @@ -143,7 +144,7 @@ Should not trigger: "Fix this CSS bug"; "Review the current PR"; "Summarize what

## Reference Map

- `references/project-workflow.md`: independent stages, project-local state, tagging, monthly context, risk-based verification and snapshots at completion with explicit retries.
- `references/project-workflow.md`: independent stages, project-local state, tagging, selected event-window context, risk-based verification and consent-gated snapshots and retries within requested completion.

- `references/processing-contract.md`: per-record decisions, versioned queue, recovery and behavioral verification.
- `references/langfuse-evidence.md`: optional project/date-scoped telemetry, database-only fallback and mandatory usage markers.
Expand Down
Loading
Loading