Build the Elekloader catalogue from per-module records

.gitignore

@@ -54,3 +54,4 @@ docs/HANDOFF-*.md
 # Committing them made every module PR edit the same lines.
 src/catalog/machine-modules.json
 src/community/module-changelogs.json
+vendor/elekloader/catalog/catalog.json

docs/ADD_A_MODULE.md

@@ -233,7 +233,7 @@ Firmware for these machines is built by elekloader's builder, vendored unchanged
 **Writing a new mod:** `npm run module:new -- my-mod --machine digitakt --author <github-login>` creates the folder (`--machine digitone` for Digitone).
 - Write `src/`, and subscribe to core events in `build.json` ([events and budgets](../sdk/machines/digitakt/README.md)).
 - Modwerk compiles the source in its pinned toolchain ([source builds](ELEMOD_SOURCE_BUILDS.md)).
-- The browser builds only from the `.elemod` files in elekloader's catalog, so to appear in the configurator a new mod needs an `.elemod` release, added with `npm run elekloader:update` as above.
+- The browser builds only from the `.elemod` files in elekloader's catalog, so to appear in the configurator a new mod needs an `.elemod` release pinned in its own record, `sdk/imports/elekloader-<id>-<date>.json`, and downloaded with `npm run elekloader:update -- --overlay <that record>`. Never edit `vendor/elekloader/catalog/catalog.json`: it is generated from the upstream catalog and the records.
 - `npm run module:doctor -- <id>` checks the module's manifest, category guide, catalog pin and module rules.
 
 ## Reference

docs/DECISIONS.md

@@ -220,7 +220,7 @@ elekloader packaged its builder as a kit that any website can use, with nothing
 
 - **Vendored kit.** [`vendor/elekloader/kit`](../vendor/elekloader/README.md) is the kit's release zip, unchanged: the engine, the builder worker, the page's client and the catalog format.
 - **The catalog.** [`vendor/elekloader/catalog`](../vendor/elekloader/README.md) holds the same cores and mods, in the kit's catalog format, with revision `e4d8ba8`, so backups keep importing.
-- **The lock.** `vendor/elekloader/elekloader.lock.json` pins every kit file and the catalog. `npm run elekloader:check` checks against it.
+- **The lock.** `vendor/elekloader/elekloader.lock.json` pins every kit file and the upstream catalog (`upstream-catalog.json`). Site-owned modules keep their pins in their own records under `sdk/imports/`, so module PRs never edit the lock; the served `catalog.json` is generated from both. `npm run elekloader:check` checks against it.
 - **What Modwerk keeps.** `src/engine/elekloader/digi-build.ts` maps machine ids to the kit's device keys and starts the kit's worker. Modwerk's own worker and client are gone.
 - **Same builds.** It is the same engine, so the same bytes: the browser check gave the parity record's identities ([verification record](VERIFICATION.md)).
 

docs/MODULE_AUTHOR_UPDATES.md

@@ -55,13 +55,13 @@ Existing followers and eligible prior downloaders receive the normal per-version
 
 1. Rebase your fork onto current `main`. Edit your module under `sdk/<machine>/modules/<id>/` and keep its original authors and licences. Include the fix's changelog update in your module’s `src/community/module-changelogs/<id>.json` in this PR. For automatic publication, increase its semantic version and update its catalog entry and matching changelog entry, including for documentation-only updates. Ordinary documentation/media-only fixes add a clearly marked note to the current version unless publishing a new version; see [release notes](ADD_A_MODULE.md#release-notes-required-for-every-fix-and-module-version).
 2. Follow [the module workflow](MODULE_ADDITION_WORKFLOW.md) and [qualification requirements](MODULE_QUALIFICATION.md). Runtime changes need current source/build evidence and native/browser parity. Exercise multiple instances with different settings across tracks, FX slots and DSP cores where supported; check instance isolation, Part save/reload, project save/load/reload, and a restart retaining only battery RAM and the saved card. Record physical reboot reports separately from emulator evidence. An explicit project load after fresh boot does not prove reboot survival. Failed or untested required acceptance checks need owner review and cannot use automatic publication. Editorial updates may retain approved evidence through the existing checked path.
-3. Regenerate the catalog, rebuild changed Octatrack packages with the reviewed isolated toolchain, and commit the source, documentation, evidence, media and stock-free packages together. For Digitakt/Digitone runtime updates, include each approved OS variant of your own `vendor/elekloader/catalog/*.elemod`, update the matching catalog rows to the new version, and update the catalog hash/revision in `vendor/elekloader/elekloader.lock.json`. Pin each package to the exact module source commit in the already approved GitHub repository (no floating release tag), retain its stock target, author and licence, and verify source/native/browser parity for that released package. Kit code, cores, dependencies and other modules stay fixed. Source or version changes without matching shipped packages cannot publish automatically. Keep firmware, projects/cards and raw dumps local. Run `npm run pr:ready -- --author <your-login>` until it says Ready: it brings in current `main`, regenerates the catalog, runs the checks CI runs and says whether the bot can merge your update ([Make it mergeable](DEVELOPER_WORKFLOW.md#make-it-mergeable)). Keep the PR to your module: a guide edit or a finding for other developers goes in its own docs-only PR ([Share what you learn](DEVELOPER_WORKFLOW.md#share-what-you-learn)), because anything outside your module's folder ends automatic publication.
+3. Regenerate the catalog, rebuild changed Octatrack packages with the reviewed isolated toolchain, and commit the source, documentation, evidence, media and stock-free packages together. For Digitakt/Digitone runtime updates, include each approved OS variant of your own `vendor/elekloader/catalog/*.elemod`, and update its pins in your module's own record, `sdk/imports/elekloader-<id>-<date>.json`, to the new version. The lock and other modules' records stay unchanged. Pin each package to the exact module source commit in the already approved GitHub repository (no floating release tag), retain its stock target, author and licence, and verify source/native/browser parity for that released package. Kit code, cores, dependencies and other modules stay fixed. Source or version changes without matching shipped packages cannot publish automatically. Keep firmware, projects/cards and raw dumps local. Run `npm run pr:ready -- --author <your-login>` until it says Ready: it brings in current `main`, regenerates the catalog, runs the checks CI runs and says whether the bot can merge your update ([Make it mergeable](DEVELOPER_WORKFLOW.md#make-it-mergeable)). Keep the PR to your module: a guide edit or a finding for other developers goes in its own docs-only PR ([Share what you learn](DEVELOPER_WORKFLOW.md#share-what-you-learn)), because anything outside your module's folder ends automatic publication.
 4. Open your PR and check both boxes below only after personally reviewing the actual evidence. You may check them after CI passes: editing an open PR invokes the trusted release verifier against the current source checks, without rebuilding. Editing a closed PR does not rerun source CI or request another release:
 
    - [ ] Automatically merge and publish my module update after checks pass.
    - [ ] I verified the source, licences, UI and qualification reports, including instance isolation and Part/project/reboot persistence where applicable; no required acceptance check is failed or untested.
 
-The workflow verifies your numeric GitHub account ID against `.github/module-authors.json` on the original base commit, then checks authorship or maintainership in the original published manifest. It grants no repository-wide write permission or site administration. A PR cannot change its own ownership, licence metadata or licence text, effect ID, the author registry, common build/runtime code, workflows, other modules or their compiled packages. New import records can describe only the author's modules; published release notes stay intact. Octatrack generated packages must reproduce the fixed compiler's output in CI. Digitakt/Digitone source is compiled twice in isolation; their separately shipped `.elemod` packages retain their approved stock targets and must pass catalog/lock/hash validation and the author's actual parity/qualification verification. Changes outside that scope remain ordinary PRs for owner review.
+The workflow verifies your numeric GitHub account ID against `.github/module-authors.json` on the original base commit, then checks authorship or maintainership in the original published manifest. It grants no repository-wide write permission or site administration. A PR cannot change its own ownership, licence metadata or licence text, effect ID, the author registry, common build/runtime code, workflows, other modules or their compiled packages. New import records can describe only the author's modules; published release notes stay intact. Octatrack generated packages must reproduce the fixed compiler's output in CI. Digitakt/Digitone source is compiled twice in isolation; their separately shipped `.elemod` packages retain their approved stock targets and must pass catalog/hash validation and the author's actual parity/qualification verification. Changes outside that scope remain ordinary PRs for owner review.
 
 `Module PR checks` must succeed on this exact head and base. A trusted CI job name records both hashes, so fork PRs work even when GitHub omits their workflow-run PR association. Required compilation jobs cannot be skipped. A new push, rerun or change to main invalidates the earlier check; rebase and rerun it. GitHub's branch protections remain enforced by the merge API. The automation does not bypass reviews required by a repository ruleset.
 

package.json

@@ -18,7 +18,7 @@
     "test": "vitest run",
     "check": "node scripts/check.mjs",
     "pr:ready": "node scripts/pr-ready.mjs",
-    "prepare": "node scripts/machine-modules.mjs && node scripts/module-changelogs.mjs --write",
+    "prepare": "node scripts/machine-modules.mjs && node scripts/module-changelogs.mjs --write && node scripts/elekloader-vendor.ts --write",
     "predev": "npm run prepare",
     "deploy": "npm run build && wrangler deploy --config wrangler.worker.jsonc",
     "dev:community": "wrangler dev --config wrangler.worker.jsonc --port 8788",

public/licenses/THIRD_PARTY_NOTICES.html

@@ -3708,10 +3708,10 @@
 Digitakt/Digitone mod releases (DIGISLICER, NEIGHBOR, digihealth, digitables, elekloader cores)
 SPDX: GPL-2.0-or-later
 Used in: vendor/elekloader/catalog
-Notice source: vendor/elekloader/catalog/catalog.json
+Notice source: vendor/elekloader/upstream-catalog.json
 Notice source: sdk/digitakt/modules/digislicer/LICENSE
 
-Digitakt/Digitone mod releases built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/catalog/catalog.json:
+Digitakt/Digitone mod releases built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/upstream-catalog.json:
 - DIGISLICER 2.1 by irpina: https://github.com/irpina/digislicer/releases/tag/v2.1
 - NEIGHBOR (digineighbor) 0.6 by irpina: https://github.com/irpina/digineighbor/releases/tag/v0.6
 - digihealth 1.0 (Digitakt) and 1.1 (Digitone) by irpina: https://github.com/irpina/digihealth/releases/tag/v1.1
@@ -4097,7 +4097,7 @@
 Used in: vendor/elekloader/catalog/digichain-1.6.elemod; vendor/elekloader/catalog/digichain-1.6-os1.54.elemod; vendor/elekloader/catalog/digieq-1.0.2-experimental-os1.53.elemod; vendor/elekloader/catalog/digieq-1.0.2-experimental-os1.54.elemod; vendor/elekloader/catalog/digimatrix-1.0b.elemod; vendor/elekloader/catalog/digimatrix-1.0b-os1.54.elemod; vendor/elekloader/catalog/digimono-0.13.3-experimental-os1.53.elemod; vendor/elekloader/catalog/digimono-0.13.3-experimental-os1.54.elemod; vendor/elekloader/catalog/digipoly-2.0.elemod; vendor/elekloader/catalog/digipoly-2.0-os1.54.elemod; vendor/elekloader/catalog/digiutils-1.9a.elemod; vendor/elekloader/catalog/digiutils-1.9a-os1.54.elemod; sdk/imports/packages/digimono/digimono-0.13.2-experimental-os1.53.elemod; sdk/imports/packages/digimono/digimono-0.13.2-experimental-os1.54.elemod; sdk/imports/packages/digimono/digimono-0.13.3-experimental-os1.53.elemod; sdk/imports/packages/digimono/digimono-0.13.3-experimental-os1.54.elemod
 Notice source: sdk/digitakt/modules/digichain/LICENSE
 
-digi1_mods releases by gdeo607, built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/catalog/catalog.json to commit 35bacb3730d108e4dc48a7bd6de4c99ae9b161e6:
+digi1_mods releases by gdeo607, built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/upstream-catalog.json to commit 35bacb3730d108e4dc48a7bd6de4c99ae9b161e6:
 - digichain 1.6, Digi EQ 1.0b, Digi Matrix 1.0b, Digi Mono 0.13b, Digi Poly 2.0 and Digi utilities 1.9a, each for Digitakt OS 1.53 and 1.54: https://github.com/gdeo607/digi1_mods/tree/35bacb3730d108e4dc48a7bd6de4c99ae9b161e6/elemods
 The imported sources and documentation are in sdk/digitakt/modules.
 

public/licenses/THIRD_PARTY_NOTICES.txt

@@ -3707,10 +3707,10 @@ Public License instead of this License.  But first, please read
 Digitakt/Digitone mod releases (DIGISLICER, NEIGHBOR, digihealth, digitables, elekloader cores)
 SPDX: GPL-2.0-or-later
 Used in: vendor/elekloader/catalog
-Notice source: vendor/elekloader/catalog/catalog.json
+Notice source: vendor/elekloader/upstream-catalog.json
 Notice source: sdk/digitakt/modules/digislicer/LICENSE
 
-Digitakt/Digitone mod releases built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/catalog/catalog.json:
+Digitakt/Digitone mod releases built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/upstream-catalog.json:
 - DIGISLICER 2.1 by irpina: https://github.com/irpina/digislicer/releases/tag/v2.1
 - NEIGHBOR (digineighbor) 0.6 by irpina: https://github.com/irpina/digineighbor/releases/tag/v0.6
 - digihealth 1.0 (Digitakt) and 1.1 (Digitone) by irpina: https://github.com/irpina/digihealth/releases/tag/v1.1
@@ -4096,7 +4096,7 @@ SPDX: MIT
 Used in: vendor/elekloader/catalog/digichain-1.6.elemod; vendor/elekloader/catalog/digichain-1.6-os1.54.elemod; vendor/elekloader/catalog/digieq-1.0.2-experimental-os1.53.elemod; vendor/elekloader/catalog/digieq-1.0.2-experimental-os1.54.elemod; vendor/elekloader/catalog/digimatrix-1.0b.elemod; vendor/elekloader/catalog/digimatrix-1.0b-os1.54.elemod; vendor/elekloader/catalog/digimono-0.13.3-experimental-os1.53.elemod; vendor/elekloader/catalog/digimono-0.13.3-experimental-os1.54.elemod; vendor/elekloader/catalog/digipoly-2.0.elemod; vendor/elekloader/catalog/digipoly-2.0-os1.54.elemod; vendor/elekloader/catalog/digiutils-1.9a.elemod; vendor/elekloader/catalog/digiutils-1.9a-os1.54.elemod; sdk/imports/packages/digimono/digimono-0.13.2-experimental-os1.53.elemod; sdk/imports/packages/digimono/digimono-0.13.2-experimental-os1.54.elemod; sdk/imports/packages/digimono/digimono-0.13.3-experimental-os1.53.elemod; sdk/imports/packages/digimono/digimono-0.13.3-experimental-os1.54.elemod
 Notice source: sdk/digitakt/modules/digichain/LICENSE
 
-digi1_mods releases by gdeo607, built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/catalog/catalog.json to commit 35bacb3730d108e4dc48a7bd6de4c99ae9b161e6:
+digi1_mods releases by gdeo607, built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/upstream-catalog.json to commit 35bacb3730d108e4dc48a7bd6de4c99ae9b161e6:
 - digichain 1.6, Digi EQ 1.0b, Digi Matrix 1.0b, Digi Mono 0.13b, Digi Poly 2.0 and Digi utilities 1.9a, each for Digitakt OS 1.53 and 1.54: https://github.com/gdeo607/digi1_mods/tree/35bacb3730d108e4dc48a7bd6de4c99ae9b161e6/elemods
 The imported sources and documentation are in sdk/digitakt/modules.
 

scripts/author-update-git.mjs

@@ -2,6 +2,7 @@
 import { execFileSync } from 'node:child_process'
 import { authorizeAuthorUpdate, parseAuthorRegistry } from '../src/release/author-updates.ts'
 import { readChangeScope } from './change-scope.mjs'
+import { UPSTREAM_CATALOG, assembleCatalog } from './elekloader-vendor.ts'
 
 export function inspectAuthorUpdate(root, base, head, author) {
   if (![base, head].every(sha => /^[a-f0-9]{40}$/.test(sha))) throw new Error('Author updates require exact Git commits.')
@@ -27,10 +28,14 @@ export function inspectAuthorUpdate(root, base, head, author) {
       regular: [old, next].every(entry => !entry || entry.type === 'blob' && ['100644', '100755'].includes(entry.mode)) }
   })
   let vendor
-  if (oldTree.has('vendor/elekloader/catalog/catalog.json')) {
-    const catalog = JSON.parse(text(base, 'vendor/elekloader/catalog/catalog.json'))
+  if (oldTree.has(UPSTREAM_CATALOG)) {
+    // The served catalog (upstream plus every module record, generated and not committed), at the base and the head.
+    const served = (ref, files) => assembleCatalog(path => files.has(path) ? text(ref, path) : undefined, [...files.keys()]).catalog
+    const catalog = served(base, oldTree)
+    let next
+    try { next = served(head, newTree) } catch (error) { throw new Error('The module records do not form a valid elekloader catalog: ' + error.message, { cause: error }) }
     const artifacts = Object.fromEntries(catalog.mods.filter(row => modules.some(module => module.document.id === row.id && module.document.machine + '-mk1' === row.device)).map(row => [row.file, JSON.parse(text(base, 'vendor/elekloader/catalog/' + row.file))]))
-    vendor = { catalog, lock: JSON.parse(text(base, 'vendor/elekloader/elekloader.lock.json')), artifacts }
+    vendor = { catalog, next, lock: JSON.parse(text(base, 'vendor/elekloader/elekloader.lock.json')), artifacts }
   }
   const authorization = authorizeAuthorUpdate(registry, author, modules, changes, vendor)
   const scope = readChangeScope({ root, base, head })