jackulau · Nov 12, 2026 · Nov 12, 2026 · Nov 12, 2026 · Nov 12, 2026 · Nov 12, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -16,6 +16,10 @@ grouped by area, with links to the docs for the details.
 - **Lint.** New rules: vendor-literal, site-voice, universal-transition, reduced-motion, global-install and personal-data. `slop.mjs --strict-allowlist` fails on allowlist entries that no longer match anything. [Contributing](https://github.com/jackulau/coding.sh/blob/main/CONTRIBUTING.md)
 - **Doctor.** `coding.sh doctor` counts recent model calls past a physical limit (speed, time to first token, duration) and names the session with the most. [Troubleshooting](https://github.com/jackulau/coding.sh/blob/main/docs/troubleshooting.md)
 - **SDK.** `coding_sh.weights_edit()` in the Python SDK records each weight edit a script makes as one `notification` event with `payload.kind: "weights.edit"`: tensor name, shape, weights changed, L2 and max-abs of the change, and the base and edited sha256. It adds no event kind, and `coding_sh.weights_delta()` computes the counts and norms for small arrays without numpy. [Docs](https://coding.sh/docs/api)
+- **SDK.** `weightsEdit()` in the TypeScript SDK records a weight edit as the same `notification` event the Python SDK's `weights_edit()` sends, and returns `true` only when the event was queued. `weightsDelta()` computes the shape, count and norms for nested arrays and typed arrays. Both SDKs' tests check one fixture of the events an edit must send. [Docs](https://coding.sh/docs/sdk)
+- **Sessions.** A session that recorded weight edits gets `stats.weightEdits` (edits, distinct tensors, weights changed) and the `weights:edited` tag, and is listed even when it holds nothing else. `coding.sh show`, the viewer and MCP `session_get` print one line per edit, and `--where 'weights.changed>1000'` filters by size. Stored sessions are folded again. [Docs](https://coding.sh/docs/api)
+- **OpenTelemetry.** A span or log record named `weights.edit` with `coding_sh.weights.*` attributes is read as a weight edit, and the `otlp` sink sends one `weights.edit` span per edit (so does `[serve.forward]`, for edits posted to `/v1/ingest`). The `invoke_agent` span carries `coding_sh.stats.weight_edits` and `coding_sh.stats.weights_changed`. Metadata mode keeps only the counts and norms. [Docs](https://coding.sh/docs/api)
+- **Evals.** `coding.sh exp run --weights <SHA256>` records which checkpoint a run's task ran (the SHA-256 of the weights file, which you compute). `exp compare`, `exp show`, `exp list`, the viewer and the MCP tools show the stamp, and a comparison warns when both sides ran the same weights or only one recorded them. [Compare an edited model with its base](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#compare-an-edited-model-with-its-base)
 - **Evals.** `coding.sh exp compare A B --given <grader>` compares only the cases where that grader passes in both runs, and prints how many it kept and dropped. With a `terminated` grader this separates "finished more answers" from "got more of the finished answers right". [Compare an edited model with its base](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#compare-an-edited-model-with-its-base)
 - **MCP.** In 2026-07-28 requests, list, read and discover results carry `ttlMs` and `cacheScope`: the tool, prompt and template lists are cached for an hour, `server/discover` for five minutes, and sessions are not cached. [MCP server](https://coding.sh/docs/mcp#protocol)
 
@@ -37,6 +41,8 @@ grouped by area, with links to the docs for the details.
 
 ### Fixed
 
+- **Evals.** `exp metrics` prints a count or score below 0.0005 with two significant digits (`0.0001`) instead of `0.000`, so a small value such as a gain per edited weight no longer reads as zero. [Metric A/B](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#metric-ab-ms-bytes)
+- **SDK.** `coding_sh.weights_edit()` reads a 0-d tensor or array as the number it holds, so `l2=delta.norm()` records the edit. Before, a torch scalar was refused and the edit was dropped with a warning, though the docs said torch scalars count as numbers. [Docs](https://coding.sh/docs/api)
 - **MCP.** A request cancelled with `notifications/cancelled` gets no reply (a cancel for `initialize` is ignored), a cursor on `tools/list`, `prompts/list` or `resources/templates/list` is an error, a request with a null id gets an error instead of no answer (a null-id `notifications/` message is still a notification), and a 2025-03-26 session accepts batches. [MCP server](https://coding.sh/docs/mcp#protocol)
 - **MCP.** Tool answers stay under 25,000 characters: a longer page has fewer rows and `limitUsed`, and a tool with a `cursor` also returns `nextCursor`. `session_replay` pages its files and cuts text and lists only when an answer would not fit, saying so with `truncated` (a range was 5 million characters). [MCP server](https://coding.sh/docs/mcp#tools)
 - **MCP.** `eval_init` and `score_add` are marked destructive, since both can replace stored data, and `score_list`'s error names its `since` argument instead of the CLI's `--since`. [MCP server](https://coding.sh/docs/mcp#tools)

diff --git a/docs/ADAPTERS.md b/docs/ADAPTERS.md
@@ -262,6 +262,9 @@ Apps in production don't use an adapter. They send data over the network to `cod
 | Vercel AI Gateway trace drain | `POST /v1/traces` | `src/otel/map.ts` plus the gateway enricher | `otel…` |
 | Claude Code, Codex and Gemini CLI running remotely with OTel logs on | `POST /v1/logs` | `src/otel/map.ts` log events | `otel…` |
 
+- A span or a log record named `weights.edit` with `coding_sh.weights.*` attributes is stored as
+  the weight edit event the SDKs send (`notification`, `payload.kind: "weights.edit"`); the
+  attributes are listed in [reference/api.md](reference/api.md).
 - Hooks, transcript backfill and `coding.sh run` stay local. Serve does not take spool lines.
 - The ingest key picks the project, and session ids are stored as `<project>/<sessionId>`.
 - SDK sessions that never send `session.end` have no `endedAt` yet; see the open item in

diff --git a/docs/PRODUCTION.md b/docs/PRODUCTION.md
@@ -265,11 +265,11 @@ max_buffer_spans = 20000
 ```
 
 - OTLP posted to `/v1/traces` keeps its traceId, spanId and parent ids. Native `/v1/ingest` batches
-  are sent as `invoke_agent`, `execute_tool` and `chat` spans, the same mapping as the `otlp` sink.
+  are sent as `invoke_agent`, `execute_tool`, `chat` and `weights.edit` spans, the same mapping as the `otlp` sink.
   Every resource gets `coding_sh.project` set to the ingest key's project.
 - `metadata` mode keeps an allowlist: gen_ai operation, provider, model, response id and finish
   reasons, tool name, call id and type, agent and conversation ids, token usage, `service.*`,
-  `telemetry.*`, `coding_sh.*` (except the session title and repo root), and numbers and booleans.
+  `telemetry.*`, `coding_sh.*` (except the session title, the repo root, and a weight edit's tensor name, shape, hashes and note), and numbers and booleans.
   Messages, prompts, tool arguments and results, `input.value`, `output.value`, span event
   attributes other than `exception.type`, and status messages are dropped. `full` mode sends every attribute after `capture.redact`.
 - Spans are sent in batches of up to 1,000, about once a second. A network error, 408, 429 or 5xx

diff --git a/docs/concepts.md b/docs/concepts.md
@@ -62,6 +62,19 @@ test ran. A flag with a colon, such as `skill:pdf`, is a tag, not a failure.
 
 Every flag and what sets it: [Flags and derails](https://coding.sh/docs/query#flags-and-derails).
 
+### Weight edit
+
+A change a script made to a model's weights, recorded as one event: the tensor's name and shape,
+how many weights changed, the size of the change, and the hash of the tensor before and after. A
+session that recorded one has the `weights:edited` tag and counts its edits in `stats.weightEdits`.
+
+- In the viewer: each edit is a row on the session page, and the header counts them.
+- In the CLI: `coding.sh sessions --flag weights:edited` lists the sessions, and
+  `coding.sh show <id>` prints one line per edit.
+- Recorded by the Python SDK's `coding_sh.weights_edit()`, the TypeScript SDK's `weightsEdit()`,
+  or any OpenTelemetry SDK as a span named `weights.edit`. The fields are in the
+  [API reference](https://coding.sh/docs/api).
+
 ### Cost
 
 What a session's model calls cost, from their token counts and each model's price. A `~` marks an

diff --git a/docs/guides/evals.md b/docs/guides/evals.md
@@ -527,10 +527,18 @@ measurement"). To compare two configs graded by different grader versions, regra
 with the newer graders and compare that. `--json` returns the new `experiment` and the `diff`
 (`flips`, `toFail`, `toPass`, `toUngraded`, `fromUngraded`, `graders`).
 
+A regrade keeps the original's [weights stamp](#compare-an-edited-model-with-its-base): the
+outputs came from that checkpoint. `exp compare` of an experiment with its regrade shows the same
+stamp on both sides and does not warn about it.
+
 `coding.sh exp run --arm base=a.json --arm cand=b.json` runs two task configs as one A/B. Cases go
 in a random order, and each case runs in both arms back to back, in random arm order, so drift
 hits both arms the same. Both arms are stored (as `<name>-base` and `<name>-cand`) and share a pair
-id. `--seed` fixes the order, `--max-cases N` runs a random N cases.
+id. `--seed` fixes the order, `--max-cases N` runs a random N cases. With `--arm`, `--weights`
+names the arm each stamp belongs to (`--weights base=<SHA256> --weights cand=<SHA256>`); one arm
+may go without. A bare `--weights <SHA256>` is a usage error there, because two arms rarely run
+one checkpoint. `--replay` with its control arm is an A/B too and refuses `--weights`; with
+`--no-control` it runs one task and takes the stamp.
 
 Add `--stop-early` to stop once the answer is clear. It is a group sequential test with
 O'Brien-Fleming alpha spending at `--looks` (default 4) evenly spaced points of the case budget,
@@ -645,7 +653,8 @@ compare <baseline cell> <cell>` gives the per-case diff of one cell, and treats
 interleaved. `--cell NAME` (repeatable) runs some cells; the baseline always runs. `--seed` fixes
 the order, `--resume` picks up an interrupted matrix, and `--json` returns `matrix`, `plan`,
 `experiments` (cell to id) and `comparisons` (each with `pHolm` and `holmVerdict`). Up to 256
-cells.
+cells. A matrix spec does not take `--weights`: one stamp cannot name the checkpoint of every
+cell. Run each checkpoint as its own `exp run --weights <SHA256>`.
 
 Several matrices against one baseline pay for the baseline each time. `--reuse baseline` copies
 the baseline cell's finished case-trials from earlier experiments instead of running them again
@@ -779,15 +788,29 @@ count such as tokens used has to become a pass or fail before it is a grader.
 {"kind": "json-path", "name": "correct", "path": "$.answer"}
 ```
 
-Run each model, then compare:
+Run each model, then compare. `--weights` records which checkpoint each task ran: the SHA-256 of
+its weights file, 64 hex characters. You compute it, for example with
+`shasum -a 256 model.safetensors`; coding.sh stores the value and hashes nothing.
 
 ```sh
-coding.sh exp run -d math --task base.json -g terminated.json -g correct.json --name base
-coding.sh exp run -d math --task edited.json -g terminated.json -g correct.json --name edited
+coding.sh exp run -d math --task base.json -g terminated.json -g correct.json --name base \
+  --weights 0496e5111097627dd9de19e48925a9143c6e2566b4f29c3f39ecc908f3b424ee
+coding.sh exp run -d math --task edited.json -g terminated.json -g correct.json --name edited \
+  --weights 58899f90572cba67c56b85687d69714292f845c7830f9971cc1b332ee047b2a7
 coding.sh exp compare base edited
 coding.sh exp compare base edited --given terminated
 ```
 
+Both compares name each side's checkpoint by the first 12 characters of its stamp
+(`weights 0496e5111097` on A, `weights 58899f90572c` on B), and `--json` carries the whole value
+as `comparison.a.weights.sha256` and `comparison.b.weights.sha256`. When both sides carry the same
+stamp, the report warns that `B − A` is not a difference between two models. When only one side
+carries a stamp, it says so, and the other side reads `weights not recorded`. `exp show`, the
+`exp run` summary, the viewer and the MCP tools `experiments_list` and `experiment_compare` show
+the same stamp. `exp list` adds a `WEIGHTS` column when a listed run has one, and its `--json`
+rows carry `weights.sha256`. `exp run` without `--weights` stores no stamp, and a comparison of two such runs
+prints nothing about weights.
+
 The first compare prints a paired delta for `terminated` and one for `correct`, each with its
 bootstrap interval. If the edited model only stopped looping, `terminated` is up and `correct`
 is up with it. `--given terminated` keeps the cases where `terminated` passes in both runs, and
@@ -876,6 +899,12 @@ rounds as a run that was never interrupted.
   script a command task runs) and timeouts, not concurrency. If any of them changed, `--resume`
   refuses and names the parts: `Can't resume exp_…: its config changed since it started
   (dataset, task)`. Mixing trials of two configs would give a result of neither.
+- A [weights stamp](#compare-an-edited-model-with-its-base) is part of the config. `--resume <id>`
+  without `--weights` keeps the stamp the run started with, and with a different `--weights` it
+  refuses: the finished trials would be stamped with weights they did not run. `--resume` with no
+  id matches a run by its config, so pass the same `--weights` again; without it the resume
+  refuses and names `weights` as the part that changed. An `--arm` run keeps and checks one stamp
+  per arm in the same way, and its stamps are in the `arms` part of the config.
 - The first Ctrl-C starts no new trials and lets the ones in flight finish and be saved. The second
   stops them. The exit code is 130, and the command to continue is printed (and is the `resume`
   field of `--json` output).
@@ -904,7 +933,8 @@ rounds as a run that was never interrupted.
   kill then leaves a partial record: `exp list` marks it `(partial)` and `exp show` says how many
   case runs it holds. `exp run … --resume <EXPERIMENT>` continues it when no job has that id: the
   finished case-trials are kept and only the rest run, under the same id. It refuses, and names
-  what changed, when the dataset version, task, graders, trials or case selection differ.
+  what changed, when the dataset version, task, graders, trials, case selection or `--weights`
+  differ. Without `--weights` it keeps the stamp the experiment has.
   `--retry-errors` with `--resume <EXPERIMENT>` also runs its task-errored case-trials again, on a
   finished experiment too.
 
@@ -957,7 +987,8 @@ prints one line and its delta against the run before it:
 - When only the graders changed (a rubric edit, a grader file), the stored outputs of the last run
   are graded again with no task calls (`regrade`). The new experiment records `regradeOf`.
 - A spec that fails to parse is reported and watching goes on. Enter reruns, `q` quits, Ctrl-C
-  exits 0. `--watch` does not combine with `--json`, `--resume` or `--arm`.
+  exits 0. `--watch` does not combine with `--json`, `--resume`, `--arm` or `--weights` (a watch
+  session reruns as the task changes, so one stamp would not stay true).
 
 ## Metric A/B (ms, bytes)
 
@@ -983,6 +1014,42 @@ the same, like the byte count of a fixed build, has no noise: a non-zero delta i
 `--json` prints the whole result, runs included. `--fail-on-worse` exits 1 when any metric got
 worse, for CI gates. scripts/perf/ uses it to gate the coding.sh site on Lighthouse metrics.
 
+### An edit's size and gain
+
+To weigh a model edit by what it cost, make each arm one edit, and have its command print how
+many weights the edit changed and what it gained:
+
+```json
+{"edited_weights": 1023, "gain_pp": 29.45, "gain_per_weight": 0.0288}
+```
+
+- `edited_weights`: the number of weights the edit changed. Lower is better, which is the default.
+- `gain_pp`: the gain over the unedited base, in percentage points.
+- `gain_per_weight`: `gain_pp` divided by `edited_weights`.
+
+Your command computes the three numbers. coding.sh treats them like any other metric: it pairs
+the runs and compares them. Name the two gain keys in `--higher`. Without it every metric is
+lower-is-better, and the edit with the larger gain reads as `worse`:
+
+```sh
+coding.sh exp metrics --arm small='python3 edit_report.py small' \
+  --arm wide='python3 edit_report.py wide' --higher 'gain_*'
+```
+
+```
+metric             small     wide                    Δ  95% CI  p (adj)  verdict
+-----------------  -----  -------  -------------------  ------  -------  -------
+edited_weights      1023   294912  +293889 (+28728.2%)  fixed         —  worse
+gain_per_weight ↑  0.029  0.00011      -0.029 (-99.6%)  fixed         —  worse
+gain_pp ↑           29.4     31.2         +1.8 (+5.9%)  fixed         —  better
+```
+
+`↑` marks a metric where higher is better. Here the wide edit changes more weights, gains 1.8
+points more, and gains far less per weight. None of the three names ends in `_ms` or `_bytes`, so
+they print as plain numbers, and a value below 0.0005 keeps two significant digits (`0.00011`)
+and does not print as `0.000`. A command that prints the same numbers on every run has no noise:
+the CI column reads `fixed`, and a non-zero delta is a change.
+
 ## Scores
 
 A score is one typed value on a session, an event, a dataset case or an eval run: a number, true