Repository navigation
feat(sessions): a weights checkpoint links the session that edited a model to the experiments that ran it - #36
Merged
Merged
Conversation
…e an experiment's weights stamp
…ere finds it by its hash A notification with payload.kind weights.checkpoint carries the SHA-256 of the saved weights file (the value exp run --weights takes), and optionally the base file's and a note. - stats.weightEdits.checkpoints lists the distinct hashes in arrival order, at most 16, and the session gets the weights:checkpoint tag. A session with a checkpoint and nothing else is still listed. SESSION_FOLD_VERSION is 21, so stored sessions are folded again. - --where 'weights.checkpoint:<hash or prefix*>' in the SQLite index, the scan, and the Postgres and ClickHouse stores. - show, the viewer and MCP session_get print 'checkpoint 58899f90572c ← 0496e5111097' from one formatter, and the Weight edits row names the checkpoints. - Metadata capture keeps both hashes and drops the note.
…heckpoint span - In: a span or a log record named weights.checkpoint with coding_sh.weights.sha256 (and .base_sha256, .note) is stored as the checkpoint event. Without a 64-hex sha256 it is not read as a checkpoint. - Out: the otlp sink and [serve.forward] send one weights.checkpoint span per event. A session with a checkpoint and no edit carries no coding_sh.stats weight attributes. - Metadata mode keeps the two file hashes, which join a session to an experiment, and drops the note. The forwarder passes the base hash only on a weights.checkpoint span.
…ights file a script saved Both send one notification with payload.kind weights.checkpoint: the SHA-256 of the saved file (the value exp run --weights takes), and optionally the base file's and a note. - Same conventions as weights_edit() and weightsEdit(): true only when the event was queued, an invalid hash drops the event with one warning, a note that is not a string is dropped and the checkpoint still recorded, and nothing raises into the app. - Hashes are sent in lowercase. In Python, base_sha256 and note are keyword-only. - One fixture, checked by both SDKs' tests, holds the payload both must send. Each suite also sends an edit and a checkpoint to a real coding.sh serve and finds the session by the checkpoint's hash. - The SDK reference generator leaves a bare * out of a Python function's heading and table.
…oduced a checkpoint
An experiment's --weights stamp and a session's weights.checkpoint event carry the same
SHA-256, so each side can now name the other.
- exp show and both sides of exp compare print one line per session that recorded the
stamp, with that session's edits:
edits <session id> 2 edits, 2 tensors, 66,560 weights changed
At most 5 sessions per hash, newest first.
- --json carries the ids as weights.sessions (exp compare, exp list, and a top-level
weights object in exp show), and so do MCP experiments_list and experiment_compare.
- coding.sh show <session> adds an Experiments row for each checkpoint an experiment ran,
and checkpointExperiments with --json.
- A stamp no session recorded prints as before. So does a store that cannot answer the
lookup: nothing is added and nothing fails.
- A list with more stamps than one --where query holds (2,000 characters, about 30 hashes)
is looked up 25 hashes at a time.
… and the viewer's Saved weights row is checked in a browser
…and the session links the experiments that ran it
… as two short points
… with where the session and experiment links appear
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
A weights file's SHA-256 now joins the session that saved it with the experiments that ran it, and every surface shows the join the same way.
coding_sh.weights_checkpoint()(Python),weightsCheckpoint()(TypeScript) or an OpenTelemetry span or log namedweights.checkpointrecords the file a script saved its edited weights to. Both SDKs' tests check one shared fixture.stats.weightEdits.checkpoints, gets theweights:checkpointtag, and--where 'weights.checkpoint:<hash or prefix*>'finds it on SQLite, Postgres and ClickHouse.exp showandexp compareprint aneditsline under theweightsline: the session that saved the file and what it changed.--json, MCPexperiments_listandexperiment_compare, and the viewer's/api/compareand/api/experiments/{id}carry the ids asweights.sessions.coding.sh showprints anExperimentsrow per checkpoint an experiment ran.show --json, MCPsession_getand/api/sessions/{id}carrycheckpointExperiments.ran in <name>.otlpsink and[serve.forward]send oneweights.checkpointspan per event.SESSION_FOLD_VERSIONis 21 and the viewer'sMODEL_VERSIONis 3, so stored sessions are folded again.Checked
On the final tree, locally:
packages/coding.sh:npm run checkprinted 6380 tests, 6371 pass, 0 fail, 9 skipped.npm run buildpassed.packages/sdk: 120 of 120.store-postgres: 16 pass, 2 skipped (no live database).store-clickhouse: 16 pass, 1 skipped.CODING_SH_SDK_REQUIRE_ALL=1: 136 tests OK.mypy --strict src: no issues.npm run lint: clean. Site: build, 135 of 135 with a built dist,perf-budget: ok.By hand on a throwaway store: the Python SDK, the TypeScript SDK and an OTLP span each recorded a checkpoint;
sessions --wherefound each;exp compare,exp show,show, the MCP tools and the builtcoding.sh uiserver returned the same ids as each other.Tests only, not run by hand: OTLP out, live Postgres and ClickHouse filters, a real torch model.
Limits
checkpointExperiments: experiments belong to no project.showandsession_getread the newest 200 experiments. One lookup took 18 ms on SQLite with 200 experiments of 100 runs each./api/experiments(the list) carries the stamp inmetadata.weightsand not the session ids; the list page does not show the stamp.weights.tensor:filter: tensor names are stored as a count only.