diff --git a/.gitignore b/.gitignore index 9f574fdd0..891491f08 100644 --- a/.gitignore +++ b/.gitignore @@ -54,3 +54,4 @@ docs/HANDOFF-*.md # Committing them made every module PR edit the same lines. src/catalog/machine-modules.json src/community/module-changelogs.json +vendor/elekloader/catalog/catalog.json diff --git a/docs/ADD_A_MODULE.md b/docs/ADD_A_MODULE.md index a17325a85..c9efaf575 100644 --- a/docs/ADD_A_MODULE.md +++ b/docs/ADD_A_MODULE.md @@ -233,7 +233,7 @@ Firmware for these machines is built by elekloader's builder, vendored unchanged **Writing a new mod:** `npm run module:new -- my-mod --machine digitakt --author ` creates the folder (`--machine digitone` for Digitone). - Write `src/`, and subscribe to core events in `build.json` ([events and budgets](../sdk/machines/digitakt/README.md)). - Modwerk compiles the source in its pinned toolchain ([source builds](ELEMOD_SOURCE_BUILDS.md)). -- The browser builds only from the `.elemod` files in elekloader's catalog, so to appear in the configurator a new mod needs an `.elemod` release, added with `npm run elekloader:update` as above. +- The browser builds only from the `.elemod` files in elekloader's catalog, so to appear in the configurator a new mod needs an `.elemod` release pinned in its own record, `sdk/imports/elekloader--.json`, and downloaded with `npm run elekloader:update -- --overlay `. Never edit `vendor/elekloader/catalog/catalog.json`: it is generated from the upstream catalog and the records. - `npm run module:doctor -- ` checks the module's manifest, category guide, catalog pin and module rules. ## Reference diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index edb817d66..40688861d 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -220,7 +220,7 @@ elekloader packaged its builder as a kit that any website can use, with nothing - **Vendored kit.** [`vendor/elekloader/kit`](../vendor/elekloader/README.md) is the kit's release zip, unchanged: the engine, the builder worker, the page's client and the catalog format. - **The catalog.** [`vendor/elekloader/catalog`](../vendor/elekloader/README.md) holds the same cores and mods, in the kit's catalog format, with revision `e4d8ba8`, so backups keep importing. -- **The lock.** `vendor/elekloader/elekloader.lock.json` pins every kit file and the catalog. `npm run elekloader:check` checks against it. +- **The lock.** `vendor/elekloader/elekloader.lock.json` pins every kit file and the upstream catalog (`upstream-catalog.json`). Site-owned modules keep their pins in their own records under `sdk/imports/`, so module PRs never edit the lock; the served `catalog.json` is generated from both. `npm run elekloader:check` checks against it. - **What Modwerk keeps.** `src/engine/elekloader/digi-build.ts` maps machine ids to the kit's device keys and starts the kit's worker. Modwerk's own worker and client are gone. - **Same builds.** It is the same engine, so the same bytes: the browser check gave the parity record's identities ([verification record](VERIFICATION.md)). diff --git a/docs/MODULE_AUTHOR_UPDATES.md b/docs/MODULE_AUTHOR_UPDATES.md index f49c46b24..e39fcc602 100644 --- a/docs/MODULE_AUTHOR_UPDATES.md +++ b/docs/MODULE_AUTHOR_UPDATES.md @@ -55,13 +55,13 @@ Existing followers and eligible prior downloaders receive the normal per-version 1. Rebase your fork onto current `main`. Edit your module under `sdk//modules//` and keep its original authors and licences. Include the fix's changelog update in your module’s `src/community/module-changelogs/.json` in this PR. For automatic publication, increase its semantic version and update its catalog entry and matching changelog entry, including for documentation-only updates. Ordinary documentation/media-only fixes add a clearly marked note to the current version unless publishing a new version; see [release notes](ADD_A_MODULE.md#release-notes-required-for-every-fix-and-module-version). 2. Follow [the module workflow](MODULE_ADDITION_WORKFLOW.md) and [qualification requirements](MODULE_QUALIFICATION.md). Runtime changes need current source/build evidence and native/browser parity. Exercise multiple instances with different settings across tracks, FX slots and DSP cores where supported; check instance isolation, Part save/reload, project save/load/reload, and a restart retaining only battery RAM and the saved card. Record physical reboot reports separately from emulator evidence. An explicit project load after fresh boot does not prove reboot survival. Failed or untested required acceptance checks need owner review and cannot use automatic publication. Editorial updates may retain approved evidence through the existing checked path. -3. Regenerate the catalog, rebuild changed Octatrack packages with the reviewed isolated toolchain, and commit the source, documentation, evidence, media and stock-free packages together. For Digitakt/Digitone runtime updates, include each approved OS variant of your own `vendor/elekloader/catalog/*.elemod`, update the matching catalog rows to the new version, and update the catalog hash/revision in `vendor/elekloader/elekloader.lock.json`. Pin each package to the exact module source commit in the already approved GitHub repository (no floating release tag), retain its stock target, author and licence, and verify source/native/browser parity for that released package. Kit code, cores, dependencies and other modules stay fixed. Source or version changes without matching shipped packages cannot publish automatically. Keep firmware, projects/cards and raw dumps local. Run `npm run pr:ready -- --author ` until it says Ready: it brings in current `main`, regenerates the catalog, runs the checks CI runs and says whether the bot can merge your update ([Make it mergeable](DEVELOPER_WORKFLOW.md#make-it-mergeable)). Keep the PR to your module: a guide edit or a finding for other developers goes in its own docs-only PR ([Share what you learn](DEVELOPER_WORKFLOW.md#share-what-you-learn)), because anything outside your module's folder ends automatic publication. +3. Regenerate the catalog, rebuild changed Octatrack packages with the reviewed isolated toolchain, and commit the source, documentation, evidence, media and stock-free packages together. For Digitakt/Digitone runtime updates, include each approved OS variant of your own `vendor/elekloader/catalog/*.elemod`, and update its pins in your module's own record, `sdk/imports/elekloader--.json`, to the new version. The lock and other modules' records stay unchanged. Pin each package to the exact module source commit in the already approved GitHub repository (no floating release tag), retain its stock target, author and licence, and verify source/native/browser parity for that released package. Kit code, cores, dependencies and other modules stay fixed. Source or version changes without matching shipped packages cannot publish automatically. Keep firmware, projects/cards and raw dumps local. Run `npm run pr:ready -- --author ` until it says Ready: it brings in current `main`, regenerates the catalog, runs the checks CI runs and says whether the bot can merge your update ([Make it mergeable](DEVELOPER_WORKFLOW.md#make-it-mergeable)). Keep the PR to your module: a guide edit or a finding for other developers goes in its own docs-only PR ([Share what you learn](DEVELOPER_WORKFLOW.md#share-what-you-learn)), because anything outside your module's folder ends automatic publication. 4. Open your PR and check both boxes below only after personally reviewing the actual evidence. You may check them after CI passes: editing an open PR invokes the trusted release verifier against the current source checks, without rebuilding. Editing a closed PR does not rerun source CI or request another release: - [ ] Automatically merge and publish my module update after checks pass. - [ ] I verified the source, licences, UI and qualification reports, including instance isolation and Part/project/reboot persistence where applicable; no required acceptance check is failed or untested. -The workflow verifies your numeric GitHub account ID against `.github/module-authors.json` on the original base commit, then checks authorship or maintainership in the original published manifest. It grants no repository-wide write permission or site administration. A PR cannot change its own ownership, licence metadata or licence text, effect ID, the author registry, common build/runtime code, workflows, other modules or their compiled packages. New import records can describe only the author's modules; published release notes stay intact. Octatrack generated packages must reproduce the fixed compiler's output in CI. Digitakt/Digitone source is compiled twice in isolation; their separately shipped `.elemod` packages retain their approved stock targets and must pass catalog/lock/hash validation and the author's actual parity/qualification verification. Changes outside that scope remain ordinary PRs for owner review. +The workflow verifies your numeric GitHub account ID against `.github/module-authors.json` on the original base commit, then checks authorship or maintainership in the original published manifest. It grants no repository-wide write permission or site administration. A PR cannot change its own ownership, licence metadata or licence text, effect ID, the author registry, common build/runtime code, workflows, other modules or their compiled packages. New import records can describe only the author's modules; published release notes stay intact. Octatrack generated packages must reproduce the fixed compiler's output in CI. Digitakt/Digitone source is compiled twice in isolation; their separately shipped `.elemod` packages retain their approved stock targets and must pass catalog/hash validation and the author's actual parity/qualification verification. Changes outside that scope remain ordinary PRs for owner review. `Module PR checks` must succeed on this exact head and base. A trusted CI job name records both hashes, so fork PRs work even when GitHub omits their workflow-run PR association. Required compilation jobs cannot be skipped. A new push, rerun or change to main invalidates the earlier check; rebase and rerun it. GitHub's branch protections remain enforced by the merge API. The automation does not bypass reviews required by a repository ruleset. diff --git a/package.json b/package.json index 7c0ef3700..ba96ec027 100644 --- a/package.json +++ b/package.json @@ -18,7 +18,7 @@ "test": "vitest run", "check": "node scripts/check.mjs", "pr:ready": "node scripts/pr-ready.mjs", - "prepare": "node scripts/machine-modules.mjs && node scripts/module-changelogs.mjs --write", + "prepare": "node scripts/machine-modules.mjs && node scripts/module-changelogs.mjs --write && node scripts/elekloader-vendor.ts --write", "predev": "npm run prepare", "deploy": "npm run build && wrangler deploy --config wrangler.worker.jsonc", "dev:community": "wrangler dev --config wrangler.worker.jsonc --port 8788", diff --git a/public/licenses/THIRD_PARTY_NOTICES.html b/public/licenses/THIRD_PARTY_NOTICES.html index f4539ab51..3ef6ddf5d 100644 --- a/public/licenses/THIRD_PARTY_NOTICES.html +++ b/public/licenses/THIRD_PARTY_NOTICES.html @@ -3708,10 +3708,10 @@ Digitakt/Digitone mod releases (DIGISLICER, NEIGHBOR, digihealth, digitables, elekloader cores) SPDX: GPL-2.0-or-later Used in: vendor/elekloader/catalog -Notice source: vendor/elekloader/catalog/catalog.json +Notice source: vendor/elekloader/upstream-catalog.json Notice source: sdk/digitakt/modules/digislicer/LICENSE -Digitakt/Digitone mod releases built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/catalog/catalog.json: +Digitakt/Digitone mod releases built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/upstream-catalog.json: - DIGISLICER 2.1 by irpina: https://github.com/irpina/digislicer/releases/tag/v2.1 - NEIGHBOR (digineighbor) 0.6 by irpina: https://github.com/irpina/digineighbor/releases/tag/v0.6 - digihealth 1.0 (Digitakt) and 1.1 (Digitone) by irpina: https://github.com/irpina/digihealth/releases/tag/v1.1 @@ -4097,7 +4097,7 @@ Used in: vendor/elekloader/catalog/digichain-1.6.elemod; vendor/elekloader/catalog/digichain-1.6-os1.54.elemod; vendor/elekloader/catalog/digieq-1.0.2-experimental-os1.53.elemod; vendor/elekloader/catalog/digieq-1.0.2-experimental-os1.54.elemod; vendor/elekloader/catalog/digimatrix-1.0b.elemod; vendor/elekloader/catalog/digimatrix-1.0b-os1.54.elemod; vendor/elekloader/catalog/digimono-0.13.3-experimental-os1.53.elemod; vendor/elekloader/catalog/digimono-0.13.3-experimental-os1.54.elemod; vendor/elekloader/catalog/digipoly-2.0.elemod; vendor/elekloader/catalog/digipoly-2.0-os1.54.elemod; vendor/elekloader/catalog/digiutils-1.9a.elemod; vendor/elekloader/catalog/digiutils-1.9a-os1.54.elemod; sdk/imports/packages/digimono/digimono-0.13.2-experimental-os1.53.elemod; sdk/imports/packages/digimono/digimono-0.13.2-experimental-os1.54.elemod; sdk/imports/packages/digimono/digimono-0.13.3-experimental-os1.53.elemod; sdk/imports/packages/digimono/digimono-0.13.3-experimental-os1.54.elemod Notice source: sdk/digitakt/modules/digichain/LICENSE -digi1_mods releases by gdeo607, built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/catalog/catalog.json to commit 35bacb3730d108e4dc48a7bd6de4c99ae9b161e6: +digi1_mods releases by gdeo607, built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/upstream-catalog.json to commit 35bacb3730d108e4dc48a7bd6de4c99ae9b161e6: - digichain 1.6, Digi EQ 1.0b, Digi Matrix 1.0b, Digi Mono 0.13b, Digi Poly 2.0 and Digi utilities 1.9a, each for Digitakt OS 1.53 and 1.54: https://github.com/gdeo607/digi1_mods/tree/35bacb3730d108e4dc48a7bd6de4c99ae9b161e6/elemods The imported sources and documentation are in sdk/digitakt/modules. diff --git a/public/licenses/THIRD_PARTY_NOTICES.txt b/public/licenses/THIRD_PARTY_NOTICES.txt index 8da3e62a6..6b2bcea4e 100644 --- a/public/licenses/THIRD_PARTY_NOTICES.txt +++ b/public/licenses/THIRD_PARTY_NOTICES.txt @@ -3707,10 +3707,10 @@ Public License instead of this License. But first, please read Digitakt/Digitone mod releases (DIGISLICER, NEIGHBOR, digihealth, digitables, elekloader cores) SPDX: GPL-2.0-or-later Used in: vendor/elekloader/catalog -Notice source: vendor/elekloader/catalog/catalog.json +Notice source: vendor/elekloader/upstream-catalog.json Notice source: sdk/digitakt/modules/digislicer/LICENSE -Digitakt/Digitone mod releases built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/catalog/catalog.json: +Digitakt/Digitone mod releases built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/upstream-catalog.json: - DIGISLICER 2.1 by irpina: https://github.com/irpina/digislicer/releases/tag/v2.1 - NEIGHBOR (digineighbor) 0.6 by irpina: https://github.com/irpina/digineighbor/releases/tag/v0.6 - digihealth 1.0 (Digitakt) and 1.1 (Digitone) by irpina: https://github.com/irpina/digihealth/releases/tag/v1.1 @@ -4096,7 +4096,7 @@ SPDX: MIT Used in: vendor/elekloader/catalog/digichain-1.6.elemod; vendor/elekloader/catalog/digichain-1.6-os1.54.elemod; vendor/elekloader/catalog/digieq-1.0.2-experimental-os1.53.elemod; vendor/elekloader/catalog/digieq-1.0.2-experimental-os1.54.elemod; vendor/elekloader/catalog/digimatrix-1.0b.elemod; vendor/elekloader/catalog/digimatrix-1.0b-os1.54.elemod; vendor/elekloader/catalog/digimono-0.13.3-experimental-os1.53.elemod; vendor/elekloader/catalog/digimono-0.13.3-experimental-os1.54.elemod; vendor/elekloader/catalog/digipoly-2.0.elemod; vendor/elekloader/catalog/digipoly-2.0-os1.54.elemod; vendor/elekloader/catalog/digiutils-1.9a.elemod; vendor/elekloader/catalog/digiutils-1.9a-os1.54.elemod; sdk/imports/packages/digimono/digimono-0.13.2-experimental-os1.53.elemod; sdk/imports/packages/digimono/digimono-0.13.2-experimental-os1.54.elemod; sdk/imports/packages/digimono/digimono-0.13.3-experimental-os1.53.elemod; sdk/imports/packages/digimono/digimono-0.13.3-experimental-os1.54.elemod Notice source: sdk/digitakt/modules/digichain/LICENSE -digi1_mods releases by gdeo607, built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/catalog/catalog.json to commit 35bacb3730d108e4dc48a7bd6de4c99ae9b161e6: +digi1_mods releases by gdeo607, built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/upstream-catalog.json to commit 35bacb3730d108e4dc48a7bd6de4c99ae9b161e6: - digichain 1.6, Digi EQ 1.0b, Digi Matrix 1.0b, Digi Mono 0.13b, Digi Poly 2.0 and Digi utilities 1.9a, each for Digitakt OS 1.53 and 1.54: https://github.com/gdeo607/digi1_mods/tree/35bacb3730d108e4dc48a7bd6de4c99ae9b161e6/elemods The imported sources and documentation are in sdk/digitakt/modules. diff --git a/scripts/author-update-git.mjs b/scripts/author-update-git.mjs index 6cc92fdfd..02a5d026c 100644 --- a/scripts/author-update-git.mjs +++ b/scripts/author-update-git.mjs @@ -2,6 +2,7 @@ import { execFileSync } from 'node:child_process' import { authorizeAuthorUpdate, parseAuthorRegistry } from '../src/release/author-updates.ts' import { readChangeScope } from './change-scope.mjs' +import { UPSTREAM_CATALOG, assembleCatalog } from './elekloader-vendor.ts' export function inspectAuthorUpdate(root, base, head, author) { if (![base, head].every(sha => /^[a-f0-9]{40}$/.test(sha))) throw new Error('Author updates require exact Git commits.') @@ -27,10 +28,14 @@ export function inspectAuthorUpdate(root, base, head, author) { regular: [old, next].every(entry => !entry || entry.type === 'blob' && ['100644', '100755'].includes(entry.mode)) } }) let vendor - if (oldTree.has('vendor/elekloader/catalog/catalog.json')) { - const catalog = JSON.parse(text(base, 'vendor/elekloader/catalog/catalog.json')) + if (oldTree.has(UPSTREAM_CATALOG)) { + // The served catalog (upstream plus every module record, generated and not committed), at the base and the head. + const served = (ref, files) => assembleCatalog(path => files.has(path) ? text(ref, path) : undefined, [...files.keys()]).catalog + const catalog = served(base, oldTree) + let next + try { next = served(head, newTree) } catch (error) { throw new Error('The module records do not form a valid elekloader catalog: ' + error.message, { cause: error }) } const artifacts = Object.fromEntries(catalog.mods.filter(row => modules.some(module => module.document.id === row.id && module.document.machine + '-mk1' === row.device)).map(row => [row.file, JSON.parse(text(base, 'vendor/elekloader/catalog/' + row.file))])) - vendor = { catalog, lock: JSON.parse(text(base, 'vendor/elekloader/elekloader.lock.json')), artifacts } + vendor = { catalog, next, lock: JSON.parse(text(base, 'vendor/elekloader/elekloader.lock.json')), artifacts } } const authorization = authorizeAuthorUpdate(registry, author, modules, changes, vendor) const scope = readChangeScope({ root, base, head }) diff --git a/scripts/author-update-git.test.mjs b/scripts/author-update-git.test.mjs index 5abe92ec1..d6775bb11 100644 --- a/scripts/author-update-git.test.mjs +++ b/scripts/author-update-git.test.mjs @@ -57,6 +57,39 @@ describe('trusted Git object inspection', () => { const f = fixture(options) expect(() => inspectAuthorUpdate(f.root, f.base, f.head, author)).toThrow() }) + it('assembles the served elekloader catalog from the upstream catalog and module records at both commits', () => { + const root = mkdtempSync(join(tmpdir(), 'modwerk-author-records-')); roots.push(root) + const git = (...args) => execFileSync('git', args, { cwd: root, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim() + const write = (path, data) => { mkdirSync(dirname(join(root, path)), { recursive: true }); writeFileSync(join(root, path), JSON.stringify(data) + '\n') } + git('init', '-q'); git('config', 'user.name', 'Modwerk test'); git('config', 'user.email', 'modwerk-test@example.test') + const doc = { id: 'demo', machine: 'digitakt', version: '1.0.0', author: { github: author.login }, license: { spdx: 'MIT' }, source: { repository: 'https://github.com/' + author.login + '/demo', revision: 'c'.repeat(40) } } + const source = { repo: author.login + '/demo', commit: 'c'.repeat(40), path: 'demo.elemod' } + const pin = { file: 'demo-1.0.0.elemod', sha256: '1'.repeat(64), id: 'demo', version: '1.0.0', device: 'digitakt-mk1', os: '1.54', license: 'MIT', author: author.login, title: 'Demo', requires: ['core'], conflicts: [], source } + const other = { ...pin, file: 'other.elemod', id: 'other', title: 'Other', author: 'someone' } + const record = mods => ({ schema: 1, kind: 'elekloader-catalog', revision: 'record', cores: [], mods }) + const artifact = { elemod: 2, id: 'demo', version: '1.0.0', license: 'MIT', author: author.login, target: { device: 'digitakt-mk1', os: '1.54' } } + write('.github/module-authors.json', { schemaVersion: 1, accounts: { [author.login]: author.id } }) + write('sdk/catalog.json', { schemaVersion: 1, sourceRevision: 'd'.repeat(40), modules: [] }) + write('sdk/digitakt/modules/demo/modwerk.module.json', doc) + write('vendor/elekloader/upstream-catalog.json', { schema: 1, kind: 'elekloader-catalog', revision: 'upstream', cores: [{ file: 'core.elemod', sha256: '0'.repeat(64), id: 'core', version: '2.1', device: 'digitakt-mk1', os: '1.54', license: 'GPL-3.0', source: { repo: 'irpina/elekloader', tag: 'v1' } }], mods: [] }) + write('vendor/elekloader/elekloader.lock.json', { schema: 1, kind: 'elekloader-kit-lock', kit: { files: {} }, catalog: {} }) + write('vendor/elekloader/catalog/demo-1.0.0.elemod', artifact) + write('sdk/imports/elekloader-demo-2026-10-14.json', record([pin])) + write('sdk/imports/elekloader-other-2026-10-14.json', record([other])) + git('add', '.'); git('commit', '-qm', 'approved base'); const base = git('rev-parse', 'HEAD') + const update = (extra = () => {}) => { + git('checkout', '-q', '--detach', base) + write('sdk/digitakt/modules/demo/modwerk.module.json', { ...doc, version: '1.0.1', source: { ...doc.source, revision: 'e'.repeat(40) } }) + write('vendor/elekloader/catalog/demo-1.0.1.elemod', { ...artifact, version: '1.0.1' }) + rmSync(join(root, 'vendor/elekloader/catalog/demo-1.0.0.elemod')) + write('sdk/imports/elekloader-demo-2026-10-14.json', record([{ ...pin, file: 'demo-1.0.1.elemod', sha256: '2'.repeat(64), version: '1.0.1', source: { ...source, commit: 'e'.repeat(40) } }])) + extra() + git('add', '-A'); git('commit', '-qm', 'author update'); return git('rev-parse', 'HEAD') + } + expect(inspectAuthorUpdate(root, base, update(), author)).toMatchObject({ modules: ['digitakt-demo'], vendorPackages: true }) + expect(() => inspectAuthorUpdate(root, base, update(() => write('sdk/imports/elekloader-other-2026-10-14.json', record([{ ...other, sha256: '3'.repeat(64) }]))), author)).toThrow('Other elekloader modules') + expect(() => inspectAuthorUpdate(root, base, update(() => write('sdk/imports/elekloader-demo-2026-10-14.json', { kind: 'not a catalog' })), author)).toThrow('valid elekloader catalog') + }) }) function github(f, options = {}) { diff --git a/scripts/change-scope.mjs b/scripts/change-scope.mjs index b0bb888eb..b9bab435c 100644 --- a/scripts/change-scope.mjs +++ b/scripts/change-scope.mjs @@ -66,7 +66,7 @@ function nativeChange(change, machine) { if (machine === 'octatrack') return common || /^sdk\/octabam\/(modules|platform|tools|dsp|licenses)\//.test(path) || path === 'sdk/catalog.json' || /^src\/engine\/assets\/(?!elemod\/)/.test(path) || /^scripts\/(build-module-packages\.py|build-utility-packages\.py|build-usb-audio-packages\.py|build-modules-isolated\.sh|import-module-build\.mjs|module-source\.mjs)$/.test(path) - return common || /^sdk\/(digitakt|digitone|elemod|machines)\//.test(path) || /^vendor\/elekloader\/(catalog\/|elekloader\.lock\.json$)/.test(path) + return common || /^sdk\/(digitakt|digitone|elemod|machines)\//.test(path) || /^vendor\/elekloader\/(catalog\/|elekloader\.lock\.json$|upstream-catalog\.json$)/.test(path) || /^sdk\/imports\/elekloader-[^/]+\.json$/.test(path) || /^sdk\/octabam\/(scripts\/vendor\.sh|tools\/patches\/|tools\/harness\/dsp_host)/.test(path) || /^scripts\/(build-elemod[^/]*|elemod-elf\.mjs|verify-elemod-source-parity\.mjs)$/.test(path) || /^src\/engine\/assets\/elemod\//.test(path) diff --git a/scripts/change-scope.test.mjs b/scripts/change-scope.test.mjs index 4b0225e53..f5f398d86 100644 --- a/scripts/change-scope.test.mjs +++ b/scripts/change-scope.test.mjs @@ -61,7 +61,7 @@ describe('documentation and firmware scheduling', () => { expect(actual.elemod).toBe(true) expect(actual.documentation).toBe(false) }) - it.each(['vendor/elekloader/catalog/catalog.json', 'vendor/elekloader/catalog/digihealth.elemod', 'vendor/elekloader/elekloader.lock.json'])('compiles and checks downloaded native package input %s', path => { + it.each(['vendor/elekloader/upstream-catalog.json', 'sdk/imports/elekloader-digihealth-2026-10-14.json', 'vendor/elekloader/catalog/digihealth.elemod', 'vendor/elekloader/elekloader.lock.json'])('compiles and checks downloaded native package input %s', path => { expect(scope(change(path))).toMatchObject({ elemod: true, documentation: false }) }) it('compiles new/removed modules and refuses invalid manifest JSON', () => { diff --git a/scripts/elekloader-update.ts b/scripts/elekloader-update.ts index 767a884ca..99ef0c60a 100644 --- a/scripts/elekloader-update.ts +++ b/scripts/elekloader-update.ts @@ -1,15 +1,16 @@ // SPDX-License-Identifier: GPL-3.0-or-later -// Updates the vendored elekloader kit, upstream catalog, site-owned catalog overlays, or a combination: +// Updates the vendored elekloader kit, upstream catalog, site-owned module records, or a combination: // -// npm run elekloader:update -- [elekloader-kit-.zip [--sha256 ]] [elekloader-catalog.json [--library]] [--overlay site-catalog.json] +// npm run elekloader:update -- [elekloader-kit-.zip [--sha256 ]] [elekloader-catalog.json [--library]] [--overlay sdk/imports/elekloader--.json] // // - The kit: every file in the zip checked against its kit.json, and the files Modwerk vendors put in place of kit/. // A kit of another protocol is refused: src/engine/elekloader/digi-build.ts is written for KIT_PROTOCOL. // - The catalog: its Digitakt and Digitone entries, each file downloaded from its author's release by the kit's own // sync and checked against its pin. Files it no longer names are removed. With --library, only the mods Modwerk's // library lists (sdk//modules//modwerk.module.json) and the mods they require are taken, so elekloader's -// whole catalog can go in as it is; the rest are not downloaded. -// - The optional overlay adds site-owned pins to the upstream catalog without replacing it. +// whole catalog can go in as it is; the rest are not downloaded. It becomes vendor/elekloader/upstream-catalog.json. +// - Site-owned pins live in each module's own record, sdk/imports/elekloader-*.json. Every record joins the served +// catalog; --overlay names a new or changed record so only its packages need syncing. // Then the lock, the elekloader licence entry and the notices, and the vendor check. It prints what changed and what // is left to do by hand. Both new copies and their lock are checked in a temporary folder first. If installing them // or regenerating/checking notices fails, the original files are restored. @@ -17,10 +18,12 @@ import { spawnSync } from 'node:child_process' import { createHash } from 'node:crypto' import { cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, rmdirSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' -import { dirname, join, resolve } from 'node:path' +import { basename, dirname, join, resolve } from 'node:path' import { fileURLToPath } from 'node:url' import { members, read } from '../vendor/elekloader/kit/src/zip.ts' -import { parseCatalog } from '../vendor/elekloader/kit/src/kit/catalog.ts' +import { UPSTREAM_CATALOG, assembleCatalog, isCatalogRecord, type CatalogFile } from './elekloader-vendor.ts' + +export { mergeCatalogOverlay } from './elekloader-vendor.ts' /** The kit protocol src/engine/elekloader/digi-build.ts is written for. */ export const KIT_PROTOCOL = 1 @@ -30,8 +33,7 @@ export const DEVICES: Record = { 'digitakt-mk1': 'digitakt', 'di const VENDORED = (name: string) => name.startsWith('src/') || ['tools/kit.ts', 'LICENSE', 'NOTICE', 'README.md', 'kit.json'].includes(name) export type KitJson = { name: string; version: string; protocol: number; commit: string; files: Record } -type Pin = { file: string; sha256: string; id: string; version: string; device: string; os: string; requires?: string[] } -type CatalogFile = { schema?: number; kind?: string; about?: string; revision: string; cores: Pin[]; mods: Pin[] } +type Pin = CatalogFile['mods'][number] const sha = (data: Uint8Array) => createHash('sha256').update(data).digest('hex') @@ -105,30 +107,6 @@ export function catalogChanges(before: CatalogFile, after: CatalogFile): Change[ return [...groups.values()].sort((x, y) => (x.key < y.key ? -1 : x.key > y.key ? 1 : 0)) } -/** Add pinned site-owned package pins without replacing the upstream catalog. */ -export function mergeCatalogOverlay(base: CatalogFile, overlay: CatalogFile): CatalogFile { - parseCatalog(overlay) - const names = new Map() - for (const pin of [...base.cores, ...base.mods]) names.set(pin.file, pin) - const cores = [...base.cores], mods = [...base.mods] - const append = (target: Pin[], pins: Pin[]) => { - for (const pin of pins) { - const existing = names.get(pin.file) - if (existing) { - if (existing.sha256 !== pin.sha256) throw new Error(`Catalog overlay changes the existing file ${pin.file}.`) - continue - } - names.set(pin.file, pin) - target.push(pin) - } - } - append(cores, overlay.cores) - append(mods, overlay.mods) - const merged = { ...base, cores, mods, revision: '' } - merged.revision = sha(new TextEncoder().encode(JSON.stringify({ schema: merged.schema, kind: merged.kind, cores, mods }))) - parseCatalog(merged) - return merged -} /** A change in a line: replaced, added or removed. */ export function describeChange(c: Change) { @@ -197,7 +175,7 @@ async function main(root: string, argv: string[]) { else paths.push(argv[i]) } const zip = paths.find(p => p.toLowerCase().endsWith('.zip')), from = paths.find(p => p.toLowerCase().endsWith('.json')) - if ((!paths.length && !overlayPath) || paths.length !== [zip, from].filter(Boolean).length || expected !== undefined && (!zip || !/^[0-9a-fA-F]{64}$/.test(expected)) || library && !(from || overlayPath)) throw new Error(usage) + if ((!paths.length && !overlayPath) || paths.length !== [zip, from].filter(Boolean).length || expected !== undefined && (!zip || !/^[0-9a-fA-F]{64}$/.test(expected)) || library && !from) throw new Error(usage) const vendor = resolve(root, 'vendor/elekloader'), kitDir = join(vendor, 'kit'), catalogDir = join(vendor, 'catalog') const scratch = mkdtempSync(join(tmpdir(), 'modwerk-elekloader-')), notes: string[] = [] const kitNext = join(scratch, 'kit'), catalogNext = join(scratch, 'catalog') @@ -214,43 +192,56 @@ async function main(root: string, argv: string[]) { changedKit = [...files.keys()].filter(name => !existsSync(join(kitDir, name)) || sha(readFileSync(join(kitDir, name))) !== sha(files.get(name)!)) after = kit; kitCount = files.size } - const oldCatalog = JSON.parse(readFileSync(join(catalogDir, 'catalog.json'), 'utf8')) as CatalogFile - let newCatalog = oldCatalog, required: ReadonlySet = new Set() - if (from || overlayPath) { - let source = from ? resolve(from) : undefined - if (overlayPath) { - const base = source ? JSON.parse(readFileSync(source, 'utf8')) as CatalogFile : oldCatalog - const overlay = JSON.parse(readFileSync(resolve(overlayPath), 'utf8')) as CatalogFile - source = join(scratch, 'catalog-overlay.json') - writeFileSync(source, JSON.stringify(mergeCatalogOverlay(base, overlay), null, 1) + '\n') - } + // Upstream pins are locked on their own; each module's record in sdk/imports/ adds its pins, so the served catalog + // is upstream-catalog.json plus every record. + if (overlayPath && !(dirname(resolve(overlayPath)) === resolve(root, 'sdk/imports') && isCatalogRecord('sdk/imports/' + basename(overlayPath)) && existsSync(resolve(overlayPath)))) + throw new Error('--overlay takes a module record in sdk/imports/ (elekloader--.json); the catalog includes every record there.') + const repo = (path: string) => existsSync(resolve(root, path)) ? readFileSync(resolve(root, path), 'utf8') : undefined + const records = readdirSync(resolve(root, 'sdk/imports')).map(name => 'sdk/imports/' + name) + const recordPins = records.filter(isCatalogRecord).flatMap(path => { const record = JSON.parse(repo(path)!) as CatalogFile; return [...record.cores, ...record.mods] }) + const notRecord = (pins: Pin[]) => pins.filter(pin => !recordPins.some(own => own.file === pin.file)) + const oldCatalog = assembleCatalog(repo, records).catalog + let upstream = JSON.parse(repo(UPSTREAM_CATALOG)!) as CatalogFile, required: ReadonlySet = new Set() + const nextKit = zip ? kitNext : kitDir, tool = join(nextKit, 'tools/kit.ts'), devices = Object.keys(DEVICES).flatMap(key => ['--device', key]) + // the kit's own sync, the new kit's if there is one, into a copy of the current folder: files already there + // with the pinned sha256 are not downloaded again + cpSync(catalogDir, catalogNext, { recursive: true }) + if (from) { + // A full catalog can hold module-record pins too; they stay in their records. + const source = JSON.parse(readFileSync(resolve(from), 'utf8')) as CatalogFile + let next: CatalogFile = { ...source, cores: notRecord(source.cores), mods: notRecord(source.mods) } if (library) { - // only what Modwerk's library lists, and what that requires: the rest is not downloaded. Site-owned overlay - // packages have no library record yet, so the cut must not drop them. - const owned = new Set(overlayPath ? (JSON.parse(readFileSync(resolve(overlayPath), 'utf8')) as CatalogFile).mods.map(m => `${DEVICES[m.device]}/${m.id}`) : []) - const cut = libraryCatalog(JSON.parse(readFileSync(source!, 'utf8')), (machine, id) => owned.has(`${machine}/${id}`) || existsSync(resolve(root, `sdk/${machine}/modules/${id}/modwerk.module.json`))) - source = join(scratch, 'catalog-library.json') - writeFileSync(source, JSON.stringify(cut.catalog, null, 1) + '\n') + // only what Modwerk's library lists, what that requires, and what the module records require: the rest is not downloaded. + const owned = new Set(recordPins.map(m => `${DEVICES[m.device]}/${m.id}`)) + const withRecords = assembleCatalog(path => path === UPSTREAM_CATALOG ? JSON.stringify(next) : repo(path), records).catalog + const cut = libraryCatalog(withRecords, (machine, id) => owned.has(`${machine}/${id}`) || existsSync(resolve(root, `sdk/${machine}/modules/${id}/modwerk.module.json`))) + next = { ...next, cores: notRecord(cut.catalog.cores), mods: notRecord(cut.catalog.mods) } required = cut.required if (cut.leftOut.length) console.log(`--library: left out ${cut.leftOut.length} mods Modwerk's library does not list: ${cut.leftOut.join(', ')}`) if (required.size) console.log(`--library: kept as requirements of library mods: ${[...required].join(', ')}`) } - // the kit's own sync, the new kit's if there is one, into a copy of the current folder: files already there - // with the pinned sha256 are not downloaded again - cpSync(catalogDir, catalogNext, { recursive: true }) - const tool = join(zip ? kitNext : kitDir, 'tools/kit.ts') - run([tool, 'sync', source!, catalogNext, ...Object.keys(DEVICES).flatMap(key => ['--device', key])], { cwd: root }) - newCatalog = JSON.parse(readFileSync(join(catalogNext, 'catalog.json'), 'utf8')) as CatalogFile - const named = new Set([...newCatalog.cores, ...newCatalog.mods].map(p => p.file)) - for (const name of readdirSync(catalogNext)) if (name !== 'catalog.json' && !named.has(name)) rmSync(join(catalogNext, name)) + writeFileSync(join(scratch, 'upstream.json'), JSON.stringify(next, null, 1) + '\n') + run([tool, 'sync', join(scratch, 'upstream.json'), catalogNext, ...devices], { cwd: root }) + upstream = JSON.parse(readFileSync(join(catalogNext, 'catalog.json'), 'utf8')) as CatalogFile } + const upstreamText = JSON.stringify(upstream, null, 1) + '\n' + const site = assembleCatalog(path => path === UPSTREAM_CATALOG ? upstreamText : repo(path), records), newCatalog = site.catalog + writeFileSync(join(scratch, 'site.json'), site.text) + run([tool, 'sync', join(scratch, 'site.json'), catalogNext, ...devices], { cwd: root }) + writeFileSync(join(catalogNext, 'catalog.json'), site.text) + const named = new Set([...newCatalog.cores, ...newCatalog.mods].map(p => p.file)) + for (const name of readdirSync(catalogNext)) if (name !== 'catalog.json' && !named.has(name)) rmSync(join(catalogNext, name)) // The incoming kit must read the catalog even for a kit-only update. Generate and verify its lock before - // replacing any installed file: the worker protocol and the catalog schema can change independently. - const nextKit = zip ? kitNext : kitDir, nextCatalog = from || overlayPath ? catalogNext : catalogDir - const tool = join(nextKit, 'tools/kit.ts'), lockPath = join(scratch, 'elekloader.lock.json') - const lock = run([tool, 'lock', '--kit', nextKit, '--catalog', nextCatalog], { capture: true, cwd: root }) + // replacing any installed file: the worker protocol and the catalog schema can change independently. The lock + // pins the kit and the upstream catalog alone; the served folder is verified whole. + const upstreamDir = join(scratch, 'upstream'), lockPath = join(scratch, 'elekloader.lock.json') + mkdirSync(upstreamDir) + writeFileSync(join(upstreamDir, 'catalog.json'), upstreamText) + for (const pin of [...upstream.cores, ...upstream.mods]) cpSync(join(catalogNext, pin.file), join(upstreamDir, pin.file)) + const lock = run([tool, 'lock', '--kit', nextKit, '--catalog', upstreamDir], { capture: true, cwd: root }) writeFileSync(lockPath, lock) - run([tool, 'verify', nextCatalog, '--lock', lockPath, '--kit', nextKit], { cwd: root }) + run([tool, 'verify', upstreamDir, '--lock', lockPath, '--kit', nextKit], { cwd: root }) + run([tool, 'verify', catalogNext], { cwd: root }) const manifestPath = resolve(root, 'vendor/licenses/manifest.json') let manifest = readFileSync(manifestPath, 'utf8') @@ -263,8 +254,8 @@ async function main(root: string, argv: string[]) { // Keep every affected file, including the generated distribution notices. A later generation/verification // error must not leave a new kit beside an old lock, or a partially regenerated set of licence notices. - const folders = [...(zip ? ['vendor/elekloader/kit'] : []), ...(from || overlayPath ? ['vendor/elekloader/catalog'] : [])] - const files = ['vendor/elekloader/elekloader.lock.json', 'vendor/licenses/manifest.json', 'vendor/licenses/elekloader.txt', + const folders = [...(zip ? ['vendor/elekloader/kit'] : []), 'vendor/elekloader/catalog'] + const files = ['vendor/elekloader/elekloader.lock.json', UPSTREAM_CATALOG, 'vendor/licenses/manifest.json', 'vendor/licenses/elekloader.txt', 'sdk/octabam/licenses/THIRD_PARTY_NOTICES.txt', 'public/licenses/THIRD_PARTY_NOTICES.txt', 'public/licenses/THIRD_PARTY_NOTICES.html'] const originals = new Map(files.map(path => [path, existsSync(resolve(root, path)) ? readFileSync(resolve(root, path)) : undefined])) for (const path of folders) cpSync(resolve(root, path), join(scratch, 'original', path), { recursive: true }) @@ -274,7 +265,8 @@ async function main(root: string, argv: string[]) { } try { if (zip) mirror(kitNext, kitDir) - if (from || overlayPath) mirror(catalogNext, catalogDir) + mirror(catalogNext, catalogDir) + writeFileSync(resolve(root, UPSTREAM_CATALOG), upstreamText) writeFileSync(join(vendor, 'elekloader.lock.json'), lock) writeFileSync(manifestPath, manifest) writeFileSync(resolve(root, 'vendor/licenses/elekloader.txt'), notice) diff --git a/scripts/elekloader-vendor.ts b/scripts/elekloader-vendor.ts index 126d1a294..8274a0fb0 100644 --- a/scripts/elekloader-vendor.ts +++ b/scripts/elekloader-vendor.ts @@ -1,8 +1,11 @@ // SPDX-License-Identifier: GPL-3.0-or-later -// The vendored elekloader kit (vendor/elekloader/kit) and the catalog Modwerk offers with it (vendor/elekloader/catalog), -// both pinned by vendor/elekloader/elekloader.lock.json, as the kit's own `tools/kit.ts verify --lock` checks them. -// Verifies every pinned file offline, and serves/emits the catalog and its files under elekloader/, where the kit's -// worker loads them. The kit's sources are bundled into the page and the worker through their imports. +// The vendored elekloader kit (vendor/elekloader/kit) and the catalog Modwerk offers with it (vendor/elekloader/catalog). +// The lock (vendor/elekloader/elekloader.lock.json) pins the kit and the upstream catalog, upstream-catalog.json. Each +// Modwerk-built mod's pins live in its own import record, sdk/imports/elekloader-*.json, so two module PRs never edit +// one shared file. The served catalog/catalog.json is both together: generated by npm ci (prepare) and npm run check, +// never committed. Verifies every pinned file offline, and serves/emits the catalog and its files under elekloader/, +// where the kit's worker loads them. The kit's sources are bundled into the page and the worker through their imports. +import { existsSync, readFileSync, readdirSync, writeFileSync } from 'node:fs' import { readFile, readdir } from 'node:fs/promises' import { createHash } from 'node:crypto' import { dirname, relative, resolve, sep } from 'node:path' @@ -11,8 +14,59 @@ import type { Plugin } from 'vite' import { parseCatalog, type Catalog } from '../vendor/elekloader/kit/src/kit/catalog.ts' export const ELEKLOADER_SITE = 'elekloader/' +export const UPSTREAM_CATALOG = 'vendor/elekloader/upstream-catalog.json', SITE_CATALOG = 'vendor/elekloader/catalog/catalog.json' +/** A module's own catalog pins (a repository path), as `npm run elekloader:update -- --overlay` takes them. */ +export const isCatalogRecord = (path: string) => /^sdk\/imports\/elekloader-[a-z0-9][a-z0-9.-]*\.json$/.test(path) type Lock = { schema: number; kind: string; kit: { version: string; protocol: number; files: Record }; catalog: { revision: string; sha256: string } } -const sha = (bytes: Buffer) => createHash('sha256').update(bytes).digest('hex') +type Pin = { file: string; sha256: string; id: string; version: string; device: string; os: string; requires?: string[] } +export type CatalogFile = { schema?: number; kind?: string; about?: string; revision: string; cores: Pin[]; mods: Pin[] } +const sha = (bytes: Uint8Array) => createHash('sha256').update(bytes).digest('hex') + +/** Add pinned site-owned package pins without replacing the upstream catalog. */ +export function mergeCatalogOverlay(base: CatalogFile, overlay: CatalogFile): CatalogFile { + parseCatalog(overlay) + const names = new Map() + for (const pin of [...base.cores, ...base.mods]) names.set(pin.file, pin) + const cores = [...base.cores], mods = [...base.mods] + const append = (target: Pin[], pins: Pin[]) => { + for (const pin of pins) { + const existing = names.get(pin.file) + if (existing) { + if (existing.sha256 !== pin.sha256) throw new Error(`Catalog overlay changes the existing file ${pin.file}.`) + continue + } + names.set(pin.file, pin) + target.push(pin) + } + } + append(cores, overlay.cores) + append(mods, overlay.mods) + const merged = { ...base, cores, mods, revision: '' } + merged.revision = sha(new TextEncoder().encode(JSON.stringify({ schema: merged.schema, kind: merged.kind, cores, mods }))) + parseCatalog(merged) + return merged +} + +/** The served catalog: the upstream catalog plus every module record, in file-name order. `read(path)` gives a + * repository file's text, or undefined when it is absent, so trusted code can assemble it from Git objects too. */ +export function assembleCatalog(read: (path: string) => string | undefined, records: readonly string[]) { + const upstream = read(UPSTREAM_CATALOG) + if (upstream === undefined) throw new Error(UPSTREAM_CATALOG + ' is missing') + const catalog = records.filter(isCatalogRecord).sort() + .reduce((merged, path) => mergeCatalogOverlay(merged, JSON.parse(read(path)!) as CatalogFile), JSON.parse(upstream) as CatalogFile) + parseCatalog(catalog) + return { upstream, catalog, text: JSON.stringify(catalog, null, 1) + '\n' } +} +function siteCatalog(root: string) { + const imports = resolve(root, 'sdk/imports') + const records = existsSync(imports) ? readdirSync(imports).map(name => 'sdk/imports/' + name) : [] + return assembleCatalog(path => existsSync(resolve(root, path)) ? readFileSync(resolve(root, path), 'utf8') : undefined, records) +} +/** Writes catalog/catalog.json when it changed. */ +export function writeSiteCatalog(root: string) { + const { text } = siteCatalog(root), target = resolve(root, SITE_CATALOG) + if (!existsSync(target) || readFileSync(target, 'utf8') !== text) writeFileSync(target, text) +} async function walk(folder: string): Promise { const out: string[] = [] @@ -32,14 +86,15 @@ export async function readVendoredElekloader(root: string): Promise<{ lock: Lock const lock = JSON.parse(await readFile(resolve(folder, 'elekloader.lock.json'), 'utf8')) as Lock if (lock.schema !== 1 || lock.kind !== 'elekloader-kit-lock' || !lock.kit?.files || !lock.catalog?.sha256) throw new Error('Invalid elekloader lock file') const top = (await readdir(folder)).sort().join(',') - if (top !== 'README.md,catalog,elekloader.lock.json,kit') throw new Error('vendor/elekloader holds only README.md, catalog/, elekloader.lock.json and kit/: ' + top) + if (top !== 'README.md,catalog,elekloader.lock.json,kit,upstream-catalog.json') throw new Error('vendor/elekloader holds only README.md, catalog/, elekloader.lock.json, kit/ and upstream-catalog.json: ' + top) const bad: string[] = [] const kitFiles = await names(kit) for (const path of kitFiles) if (lock.kit.files[path] !== sha(await readFile(resolve(kit, path)))) bad.push(path in lock.kit.files ? 'kit file changed: ' + path : 'kit file not in the lock: ' + path) for (const path of Object.keys(lock.kit.files)) if (!kitFiles.includes(path)) bad.push('kit file missing: ' + path) - const raw = await readFile(resolve(dir, 'catalog.json')) - if (sha(raw) !== lock.catalog.sha256) bad.push('catalog.json is not the one the lock pins') - const catalog = parseCatalog(JSON.parse(raw.toString('utf8'))) + const site = siteCatalog(root), raw = Buffer.from(site.text) + if (sha(Buffer.from(site.upstream)) !== lock.catalog.sha256) bad.push('upstream-catalog.json is not the one the lock pins') + if (!(await readFile(resolve(dir, 'catalog.json')).catch(() => Buffer.alloc(0))).equals(raw)) bad.push('catalog/catalog.json is not the upstream catalog plus the module records: run npm run prepare') + const catalog = parseCatalog(site.catalog) const pins = [...catalog.cores, ...catalog.mods], assets: [string, Buffer][] = [[ELEKLOADER_SITE + 'catalog.json', raw]] for (const pin of pins) { const data = await readFile(resolve(dir, pin.file)).catch(() => undefined) @@ -76,7 +131,8 @@ export function elekloaderSite(root: string): Plugin { } } -if (process.argv[1] === fileURLToPath(import.meta.url)) { +if (process.argv[1] === fileURLToPath(import.meta.url) && process.argv.includes('--write')) writeSiteCatalog(resolve(dirname(fileURLToPath(import.meta.url)), '..')) +else if (process.argv[1] === fileURLToPath(import.meta.url)) { const { lock, catalog, assets } = await readVendoredElekloader(resolve(dirname(fileURLToPath(import.meta.url)), '..')) console.log('Vendored elekloader kit ' + lock.kit.version + ' (protocol ' + lock.kit.protocol + ', ' + Object.keys(lock.kit.files).length + ' files) and catalog ' + catalog.revision.slice(0, 7) + ' (' + catalog.cores.length + ' cores, ' + catalog.mods.length + ' mods): ' + assets.length + ' site assets, as the lock pins.') } diff --git a/src/engine/elekloader/digi-build.test.ts b/src/engine/elekloader/digi-build.test.ts index a5d77a213..2939aba96 100644 --- a/src/engine/elekloader/digi-build.test.ts +++ b/src/engine/elekloader/digi-build.test.ts @@ -1,7 +1,6 @@ import { describe, expect, it, vi } from 'vitest' import { DIGI_MODS } from '../../devices/digi-mods' import MACHINES from '../../devices/machines.generated.json' -import LOCK from '../../../vendor/elekloader/elekloader.lock.json' import { BUILDER_CATALOG, BUILDER_SOURCE, buildLogText, buildStep, builderReleases, planBuild, prepareBuild, type Builder } from './digi-build' import type { Catalog } from '../../../vendor/elekloader/kit/src/kit/catalog' import { DIGI_DOWNLOADS_ENABLED, digiDownloadsEnabled } from './protocol' @@ -16,7 +15,8 @@ describe('the vendored elekloader catalog', () => { it('has the pinned release file for every library module on each release it lists', () => { for (const mod of DIGI_MODS) expect(builderReleases(mod.device, mod.id).sort(), mod.device + ' ' + mod.id).toEqual([...mod.releases].sort()) expect(BUILDER_CATALOG.mods.every(item => /^[a-f0-9]{64}$/.test(item.sha256) && /^[\w.-]+\.elemod$/.test(item.file))).toBe(true) - expect(BUILDER_CATALOG.revision).toBe(LOCK.catalog.revision) // saved backups name it + // Saved backups name it; npm run elekloader:check verifies the catalog is the upstream catalog plus the module records. + expect(BUILDER_CATALOG.revision).toMatch(/^[0-9a-f]{64}$/) }) it('plans the released Digitakt II mods only for OS 1.17 with their DTII core', () => { expect(planBuild('digitakt-ii', '1.17', ['perform-direct', 'perform-levels'])).toMatchObject({ diff --git a/src/release/author-updates.ts b/src/release/author-updates.ts index cec2eebdc..6e6ed5b8c 100644 --- a/src/release/author-updates.ts +++ b/src/release/author-updates.ts @@ -6,7 +6,9 @@ export const ACTIONS_BOT_ID = 41898282 export type AuthorRegistry = { schemaVersion: 1; accounts: Record } export type AuthorModule = { folder: string; document: Record } export type AuthorChange = { path: string; before: string | null; after: string | null; regular: boolean } -export type AuthorVendor = { catalog: Record; lock: Record; artifacts: Record> } +/** The served elekloader catalog (upstream plus every module record) at the base and at the head, assembled by trusted code. */ +export type AuthorVendor = { catalog: Record; next: Record; lock: Record; artifacts: Record> } +const CATALOG_RECORD = /^sdk\/imports\/elekloader-[a-z0-9][a-z0-9.-]*\.json$/ const object = (value: unknown): Record => value && typeof value === 'object' && !Array.isArray(value) ? value as Record : {} const same = (a: unknown, b: unknown): boolean => JSON.stringify(canonical(a)) === JSON.stringify(canonical(b)) function canonical(value: unknown): unknown { @@ -60,21 +62,23 @@ function packageProjection(value: unknown, ids: Set, changedPackages: Se })) } function authorizeVendor(vendor: AuthorVendor | undefined, changes: readonly AuthorChange[], ids: Set, next: Map>) { - const paths = changes.filter(change => change.path.startsWith('vendor/elekloader/')) + const paths = changes.filter(change => change.path.startsWith('vendor/elekloader/') || CATALOG_RECORD.test(change.path)) if (!paths.length) return new Set() - if (!vendor) throw new Error('An author package needs a previously approved elekloader catalog.') - const catalogPath = 'vendor/elekloader/catalog/catalog.json', lockPath = 'vendor/elekloader/elekloader.lock.json' - const edit = paths.find(change => change.path === catalogPath), lockEdit = paths.find(change => change.path === lockPath) - if (!edit?.after || !lockEdit?.after || !edit.before || !lockEdit.before) throw new Error('Update both the elekloader catalog and lock with author packages.') - const catalog = object(JSON.parse(edit.after)), lock = object(JSON.parse(lockEdit.after)) + if (!vendor?.next) throw new Error('An author package needs a previously approved elekloader catalog.') + const catalog = object(vendor.next) const oldRows = Array.isArray(vendor.catalog.mods) ? vendor.catalog.mods.map(object) : [], rows = Array.isArray(catalog.mods) ? catalog.mods.map(object) : [] const id = (row: Record) => String(row.device).replace(/-mk1$/, '') + '-' + row.id const identity = (row: Record) => [row.id, row.device, row.os, row.license, row.author, row.requires, row.needs_core] const project = (value: Record) => ({ ...value, revision: null, mods: (value.mods as unknown[]).map(object).map(row => ids.has(id(row)) ? identity(row) : row) }) if (!same(project(vendor.catalog), project(catalog))) throw new Error('Other elekloader modules, cores, OS targets and licences require owner review.') - const projectLock = (value: Record) => ({ ...value, catalog: { ...object(value.catalog), revision: null, sha256: null } }) - if (!same(projectLock(vendor.lock), projectLock(lock)) || object(lock.catalog).revision !== catalog.revision) throw new Error('The elekloader kit and unrelated catalog pins require owner review.') - const permitted = new Set([catalogPath, lockPath]) + // The lock pins the kit and the upstream catalog, so an author update never changes it. A module record may hold + // only the author's own pins, before and after. + const permitted = new Set() + for (const record of paths.filter(change => CATALOG_RECORD.test(change.path))) { + const pins = [record.before, record.after].flatMap(text => text === null ? [] : Object.values(object(JSON.parse(text))).filter(Array.isArray).flat().map(object)) + if (!pins.length || pins.some(row => !ids.has(id(row)))) throw new Error('Elekloader records of other modules or cores require owner review.') + permitted.add(record.path) + } for (const row of rows.filter(row => ids.has(id(row)))) { const old = oldRows.find(item => same(identity(item), identity(row))) if (!old || typeof row.file !== 'string' || typeof old.file !== 'string' || !/^[A-Za-z0-9_.-]+\.elemod$/.test(row.file) || !next.has(id(row))) throw new Error('Invalid author elekloader package identity.') @@ -95,7 +99,7 @@ function authorizeVendor(vendor: AuthorVendor | undefined, changes: readonly Aut } if (paths.some(change => !permitted.has(change.path))) throw new Error('Elekloader changes outside the author packages require owner review.') for (const moduleId of ids) if (/^(digitakt|digitone)-/.test(moduleId) && !rows.some(row => id(row) === moduleId)) throw new Error('Every released Digi module needs matching approved native packages.') - if (permitted.size <= 2) throw new Error('An author package update must change a pinned native package.') + if (![...permitted].some(path => path.endsWith('.elemod'))) throw new Error('An author package update must change a pinned native package.') return permitted } diff --git a/src/release/vendor-author-updates.test.ts b/src/release/vendor-author-updates.test.ts index 171ce0600..f077ac057 100644 --- a/src/release/vendor-author-updates.test.ts +++ b/src/release/vendor-author-updates.test.ts @@ -4,26 +4,26 @@ const registry = { schemaVersion: 1 as const, accounts: { irpina: 264014615 } }, const folder = 'sdk/digitakt/modules/digihealth', document = { id: 'digihealth', machine: 'digitakt', version: '1.1.0-experimental', author: { github: 'irpina' }, maintainers: ['irpina'], license: { spdx: 'MIT' }, source: { repository: 'https://github.com/irpina/digihealth', revision: 'c'.repeat(40) } }, next = { ...document, version: '1.1.1-experimental' } const modules = [{ folder, document }] const pin = { id: 'digihealth', device: 'digitakt-mk1', os: '1.54', version: document.version, file: 'digihealth.elemod', sha256: 'old', license: 'MIT', author: 'irpina', requires: ['core'], source: { repo: 'irpina/digihealth', commit: document.source.revision, path: 'digihealth.elemod' } } -const other = { ...pin, id: 'digislicer', file: 'digislicer.elemod' } +const other = { ...pin, id: 'digislicer', file: 'digislicer.elemod' }, updated = { ...pin, version: next.version, sha256: 'new' } +// The served catalog: the upstream catalog plus every module record, as trusted code assembles it at the base and head. const catalog = { schema: 1, kind: 'elekloader-catalog', revision: 'a'.repeat(40), cores: [{ id: 'core', file: 'core.elemod' }], mods: [pin, other] } -const lock = { schema: 1, kind: 'elekloader-kit-lock', kit: { files: { 'index.ts': 'fixed' } }, catalog: { revision: catalog.revision, sha256: 'old', cores: 1, mods: 2 } } +const served = (mods: unknown[]) => ({ ...catalog, revision: 'b'.repeat(40), mods }) +const lock = { schema: 1, kind: 'elekloader-kit-lock', kit: { files: { 'index.ts': 'fixed' } }, catalog: { revision: 'upstream', sha256: 'upstream', cores: 1, mods: 1 } } const artifact = { elemod: 2, id: 'digihealth', version: pin.version, license: 'MIT', author: 'irpina', target: { device: pin.device, os: pin.os, section3_sha256: 'stock' }, sections: { '.run': { parts: [['hex', 'old']] } } } -const vendor = { catalog, lock, artifacts: { 'digihealth.elemod': artifact } } -const change = (path: string, before: unknown, after: unknown): AuthorChange => ({ path, before: JSON.stringify(before), after: JSON.stringify(after), regular: true }) +const vendor = { catalog, next: served([updated, other]), lock, artifacts: { 'digihealth.elemod': artifact } } +const record = 'sdk/imports/elekloader-digihealth-2026-10-14.json', recordOf = (mods: unknown[]) => ({ schema: 1, kind: 'elekloader-catalog', revision: 'digihealth', cores: [], mods }) +const change = (path: string, before: unknown, after: unknown): AuthorChange => ({ path, before: before === null ? null : JSON.stringify(before), after: after === null ? null : JSON.stringify(after), regular: true }) function edits() { return [change(folder + '/modwerk.module.json', document, next), change(folder + '/source.c', 'old', 'new'), - change('vendor/elekloader/catalog/catalog.json', catalog, { ...catalog, revision: 'b'.repeat(40), mods: [{ ...pin, version: next.version, sha256: 'new' }, other] }), - change('vendor/elekloader/elekloader.lock.json', lock, { ...lock, catalog: { ...lock.catalog, revision: 'b'.repeat(40), sha256: 'new' } }), + change(record, recordOf([pin]), recordOf([updated])), change('vendor/elekloader/catalog/digihealth.elemod', artifact, { ...artifact, version: next.version, sections: { '.run': { parts: [['hex', 'new']] } } })] } describe('authored Digi packages', () => { - it('permits the author package and matching catalog/lock, with common kit and other modules frozen', () => expect(authorizeAuthorUpdate(registry, author, modules, edits(), vendor)).toMatchObject({ vendorPackages: true, modules: ['digitakt-digihealth'] })) + it('permits the author package and its own record, with the kit, upstream catalog and other modules frozen', () => expect(authorizeAuthorUpdate(registry, author, modules, edits(), vendor)).toMatchObject({ vendorPackages: true, modules: ['digitakt-digihealth'] })) it('rejects relabelling a version without shipping its package', () => expect(() => authorizeAuthorUpdate(registry, author, modules, edits().slice(0, -1), vendor)).toThrow('updated elekloader package')) it('requires every approved OS variant rather than updating one while serving another old version', () => { const variant = { ...pin, os: '1.53', file: 'digihealth-153.elemod' } - const original = { ...catalog, mods: [pin, variant, other] } - const changed = edits().map(edit => edit.path.endsWith('/catalog.json') ? change(edit.path, original, { ...original, revision: 'b'.repeat(40), mods: [{ ...pin, version: next.version, sha256: 'new' }, variant, other] }) : edit) - expect(() => authorizeAuthorUpdate(registry, author, modules, changed, { ...vendor, catalog: original })).toThrow('every approved OS package') + expect(() => authorizeAuthorUpdate(registry, author, modules, edits(), { ...vendor, catalog: { ...catalog, mods: [pin, variant, other] }, next: served([updated, variant, other]) })).toThrow('every approved OS package') }) it('does not let a package for one module cover another touched module', () => { const second = { folder: 'sdk/digitakt/modules/another', document: { ...document, id: 'another' } } @@ -32,16 +32,22 @@ describe('authored Digi packages', () => { }) it('rejects floating or mismatched package source pins', () => { for (const source of [{ repo: pin.source.repo, tag: 'v1.1' }, { ...pin.source, commit: 'd'.repeat(40) }, { ...pin.source, repo: 'attacker/fork' }]) { - const changed = edits().map(edit => edit.path.endsWith('/catalog.json') ? change(edit.path, catalog, { ...catalog, revision: 'b'.repeat(40), mods: [{ ...pin, version: next.version, source }, other] }) : edit) - expect(() => authorizeAuthorUpdate(registry, author, modules, changed, vendor)).toThrow('exact updated module source commit') + expect(() => authorizeAuthorUpdate(registry, author, modules, edits(), { ...vendor, next: served([{ ...updated, source }, other]) })).toThrow('exact updated module source commit') } }) - it('rejects a changed core, kit, other module, stock target or package owner', () => { - for (const [path, after] of [ - ['vendor/elekloader/catalog/catalog.json', { ...catalog, mods: [{ ...pin, version: next.version }, { ...other, sha256: 'tampered' }] }], - ['vendor/elekloader/elekloader.lock.json', { ...lock, kit: { files: { 'index.ts': 'tampered' } } }], - ['vendor/elekloader/catalog/digihealth.elemod', { ...artifact, version: next.version, target: { ...artifact.target, os: 'unknown' } }], - ['vendor/elekloader/catalog/digihealth.elemod', { ...artifact, version: next.version, author: 'attacker' }], - ] as const) expect(() => authorizeAuthorUpdate(registry, author, modules, edits().map(edit => edit.path === path ? { ...edit, after: JSON.stringify(after) } : edit), vendor)).toThrow() + it('rejects a changed core, other module, stock target or package owner', () => { + expect(() => authorizeAuthorUpdate(registry, author, modules, edits(), { ...vendor, next: served([updated, { ...other, sha256: 'tampered' }]) })).toThrow('Other elekloader modules') + expect(() => authorizeAuthorUpdate(registry, author, modules, edits(), { ...vendor, next: { ...served([updated, other]), cores: [{ id: 'core', file: 'tampered.elemod' }] } })).toThrow('Other elekloader modules') + for (const after of [{ ...artifact, version: next.version, target: { ...artifact.target, os: 'unknown' } }, { ...artifact, version: next.version, author: 'attacker' }]) { + expect(() => authorizeAuthorUpdate(registry, author, modules, edits().map(edit => edit.path.endsWith('.elemod') ? change(edit.path, artifact, after) : edit), vendor)).toThrow('stock target') + } + }) + it('leaves the lock, the upstream catalog and other modules’ records to the owner', () => { + for (const extra of [change('vendor/elekloader/elekloader.lock.json', lock, { ...lock, kit: { files: { 'index.ts': 'tampered' } } }), + change('vendor/elekloader/upstream-catalog.json', catalog, { ...catalog, about: 'changed' })]) { + expect(() => authorizeAuthorUpdate(registry, author, modules, [...edits(), extra], vendor)).toThrow('outside the author packages') + } + expect(() => authorizeAuthorUpdate(registry, author, modules, [...edits(), change('sdk/imports/elekloader-digislicer-2026-10-14.json', recordOf([other]), recordOf([{ ...other, sha256: 'x' }]))], vendor)).toThrow('records of other modules') + expect(() => authorizeAuthorUpdate(registry, author, modules, edits().map(edit => edit.path === record ? change(record, recordOf([pin]), { ...recordOf([updated]), cores: [{ id: 'core', device: 'digitakt-mk1', file: 'core.elemod' }] }) : edit), vendor)).toThrow('or cores') }) }) diff --git a/src/tooling/elekloader-update.test.ts b/src/tooling/elekloader-update.test.ts index ecc8a3717..9cc79ebe7 100644 --- a/src/tooling/elekloader-update.test.ts +++ b/src/tooling/elekloader-update.test.ts @@ -171,6 +171,8 @@ function fixture() { cpSync(join(ROOT, path), join(root, path), { recursive: true }) } for (const path of [...noticePaths, 'LICENSE', 'LICENSES', 'docs/USB_AUDIO_ATTRIBUTION.txt', 'scripts/elekloader-update.ts', 'scripts/elekloader-vendor.ts', 'scripts/licenses.mjs', 'scripts/license-notices.mjs']) copy(path) + // The module records that add Modwerk-built pins to the upstream catalog. + for (const name of readdirSync(join(ROOT, 'sdk/imports')).filter(name => /^elekloader-.*\.json$/.test(name))) copy('sdk/imports/' + name) const manifest = JSON.parse(readFileSync(join(root, 'sdk/octabam/licenses/manifest.json'), 'utf8')) as { moduleComponents: Record } for (const id of Object.keys(manifest.moduleComponents)) { const folder = `sdk/octabam/modules/${id}` @@ -248,7 +250,8 @@ describe('elekloader update CLI', () => { const absent = join(root, 'public/licenses/THIRD_PARTY_NOTICES.html') rmSync(absent) if (failure === 'notice generation') writeFileSync(script, `import { writeFileSync } from 'node:fs'; writeFileSync(${JSON.stringify(absent)}, 'partial notices'); throw new Error('fixture generation failure')`) - else writeFileSync(script, "throw new Error('fixture validation failure')") + // The update imports this module, so only its run as the final vendor check fails. + else writeFileSync(script, readFileSync(script, 'utf8') + "\nif (process.argv[1]?.endsWith('elekloader-vendor.ts')) throw new Error('fixture validation failure')\n") const before = identities(root) const result = update(root, [...args, catalogPath]) expect(result.status).toBe(1) diff --git a/vendor/elekloader/README.md b/vendor/elekloader/README.md index f4dc3ba2f..8426f6c66 100644 --- a/vendor/elekloader/README.md +++ b/vendor/elekloader/README.md @@ -5,12 +5,13 @@ Modwerk builds Digitakt mk1 and Digitone mk1/Keys firmware with [elekloader](htt | Path | What | From | | --- | --- | --- | | `kit/` | the kit, unchanged: `src/` (the engine and `src/kit/`), `tools/kit.ts`, `LICENSE` (GPL-3.0-or-later), `NOTICE`, `README.md` and `kit.json` | `elekloader-kit-.zip`, the commit in `kit.json` | -| `catalog/` | `catalog.json` and its pinned packages for the mk1 library plus the Digitakt II 1.17 preview | upstream cores/mods and the site-owned `sdk/imports/elekloader-digitakt2-perform-v1.0.json` overlay | -| `elekloader.lock.json` | the lock: every kit file and the catalog by SHA-256 | written by `kit/tools/kit.ts lock` | +| `upstream-catalog.json` | elekloader's cores and mods that Modwerk offers | `elekloader-catalog.json` from an elekloader release, cut to the library | +| `catalog/` | the pinned packages for the mk1 library plus the Digitakt II 1.17 preview, and `catalog.json`: `upstream-catalog.json` plus every module record, generated by `npm ci` (`prepare`) and `npm run check`, never committed | upstream packages, and each site-owned module's record `sdk/imports/elekloader-*.json` | +| `elekloader.lock.json` | the lock: every kit file and `upstream-catalog.json` by SHA-256 | written by `kit/tools/kit.ts lock` | -`npm run elekloader:check` (part of `npm run check`) checks everything against the lock, as the kit's own `verify --lock` does. It refuses any changed, missing or extra file in `kit/` or `catalog/`. The site serves `catalog/` under `elekloader/`. +`npm run elekloader:check` (part of `npm run check`) checks the kit and `upstream-catalog.json` against the lock, as the kit's own `verify --lock` does, and that `catalog/catalog.json` is the upstream catalog plus every module record. It refuses any changed, missing or extra file in `kit/` or `catalog/`. Each site-owned module keeps its pins in its own record, so two module PRs never edit one shared catalog or the lock. The site serves `catalog/` under `elekloader/`. -The catalog's `revision` identifies its set of cores and mods. For upstream-only updates it is elekloader's catalog revision; when a site-owned overlay is merged, it is a deterministic SHA-256 fingerprint of the merged pins. Configuration backups record it, so it changes with the package set, not with the kit. A backup that names an older revision still imports: its modules are looked up again in the library, and the page says which were left out or changed version. +The catalog's `revision` identifies its set of cores and mods. For upstream-only updates it is elekloader's catalog revision; with module records merged, it is a deterministic SHA-256 fingerprint of all the pins. Configuration backups record it, so it changes with the package set, not with the kit. A backup that names an older revision still imports: its modules are looked up again in the library, and the page says which were left out or changed version. `src/engine/elekloader/digi-build.ts` is Modwerk's side: @@ -33,11 +34,9 @@ Update by pull request, owner reviewed: It refuses a zip whose files are not the ones its `kit.json` names, and a kit of another protocol than `digi-build.ts` is written for. The catalog's Digitakt and Digitone files are downloaded from their authors' releases by the kit's `sync` and checked against their pins, and files it no longer names are removed. Both are staged first, and the new kit generates and verifies their lock before any installed file changes. Then it installs the files and lock, points the elekloader licence entry at the new commit, rebuilds its notice from the kit's full `NOTICE` and `LICENSE`, regenerates the distribution notices and runs the vendor check. A failure during installation, notice generation or validation restores the previous kit, catalog, lock, licence entry and notices. If a filesystem error prevents restoration, the command reports where it kept the original files for recovery. - Site-owned packages use a stock-free catalog pin under `sdk/imports/` and the overlay option: + Site-owned packages are pinned in the module's own stock-free record, `sdk/imports/elekloader--.json`. Every record joins the catalog, and a `--library` update keeps their packages. After adding or changing a record, download its packages with: `npm run elekloader:update -- --overlay sdk/imports/elekloader-digitakt2-perform-v1.0.json` - Repeat `--overlay` on every `--library` update: the library cut keeps overlay packages only when the overlay is named, and drops (and deletes) them otherwise. - - The kit sync fetches each package from its pinned author release, verifies SHA-256, and preserves existing upstream packages. The updater regenerates the lock and validates the complete catalog transactionally. + The kit sync fetches each package from its pinned author release, verifies SHA-256, and preserves existing upstream packages. The updater regenerates the lock for the upstream catalog and validates the complete catalog transactionally. 3. **Do what it lists.** It prints what changed and what is left by hand: Modwerk's own module files (`sdk//modules//modwerk.module.json`) for each mod that changed, came or left, and licence entries naming a file that is gone. A new catalog `revision` needs nothing more: older configuration backups still import, with their modules checked again against the library. 4. **Check and record:** run `npm run check`. Build every module subset locally against the previous builder or elekloader's command line, and record the result in `docs/VERIFICATION.md`. diff --git a/vendor/elekloader/elekloader.lock.json b/vendor/elekloader/elekloader.lock.json index 70e10b278..5e7b65565 100644 --- a/vendor/elekloader/elekloader.lock.json +++ b/vendor/elekloader/elekloader.lock.json @@ -40,9 +40,9 @@ } }, "catalog": { - "revision": "10417666b79acbd37bfdad4e1b2319b0baa082393380bcda26ddaad5ad395b57", - "sha256": "ae1d779791afb6d3d3278c17abfbf0b8a6476ebb687db045fdbb34703d5fb0b0", - "cores": 6, - "mods": 37 + "revision": "0da420d6532225851c0fb0ae7da58638054406c134bd34889a00c624e4a17687", + "sha256": "551e50618333e06875df838b0981b591eedbfa87cc7e500cd37985ba53665881", + "cores": 5, + "mods": 21 } } diff --git a/vendor/elekloader/catalog/catalog.json b/vendor/elekloader/upstream-catalog.json similarity index 62% rename from vendor/elekloader/catalog/catalog.json rename to vendor/elekloader/upstream-catalog.json index 12a7643cd..f07228ff0 100644 --- a/vendor/elekloader/catalog/catalog.json +++ b/vendor/elekloader/upstream-catalog.json @@ -1,7 +1,7 @@ { "schema": 1, "kind": "elekloader-catalog", - "revision": "10417666b79acbd37bfdad4e1b2319b0baa082393380bcda26ddaad5ad395b57", + "revision": "0da420d6532225851c0fb0ae7da58638054406c134bd34889a00c624e4a17687", "about": "elekloader's curated catalog: the cores and mods a site may offer, each pinned by sha256 to its author's release or commit. docs/INTEGRATING.md.", "cores": [ { @@ -68,19 +68,6 @@ "repo": "irpina/digitables", "tag": "v1.3" } - }, - { - "file": "core-1.0.elemod", - "sha256": "95303655a9358f581b485acb68b884b394a577f5d0b373f0a3e5957c8d0ca72c", - "id": "core", - "version": "1.0", - "device": "digitakt-mk2", - "os": "1.17", - "license": "GPL-2.0-or-later", - "source": { - "repo": "toonst/digitakt2-perform", - "tag": "v1.0" - } } ], "mods": [ @@ -597,412 +584,6 @@ "author": "gdeo607", "needs_core": "2.1", "on_unit": "Its author reports the stand-alone 1.5d tested on a Digitakt mk1, and 1.9a not yet; it passes their emulator tests on OS 1.53 and 1.54." - }, - { - "file": "perform-direct-1.0.elemod", - "sha256": "601b187f5af3025234c83e8c077b1e8f4ac755a8f1a8c929ab03aff331f30732", - "id": "perform-direct", - "version": "1.0", - "device": "digitakt-mk2", - "os": "1.17", - "license": "GPL-2.0-or-later", - "source": { - "repo": "toonst/digitakt2-perform", - "tag": "v1.0" - }, - "title": "Direct perform", - "requires": [ - "core" - ], - "conflicts": [], - "summary": "PRESET alone toggles PERFORM KIT; FUNC + PRESET opens the PRESET/KIT menu.", - "category": "Performance", - "author": "toonst", - "needs_core": "1.0", - "on_unit": "The author reports the key swap working on Digitakt II OS 1.17 in an earlier version that also had a PERSONALIZE on/off row. The current pair with perform-levels is not reported tested together on a unit." - }, - { - "file": "perform-levels-1.0.elemod", - "sha256": "bb100389f2800401495ca3fa3a52d5974bb00434a8a70fd6f89978ec6b25d9e6", - "id": "perform-levels", - "version": "1.0", - "device": "digitakt-mk2", - "os": "1.17", - "license": "GPL-2.0-or-later", - "source": { - "repo": "toonst/digitakt2-perform", - "tag": "v1.0" - }, - "title": "Perform keeps levels", - "requires": [ - "core" - ], - "conflicts": [], - "summary": "Track levels set in PERFORM KIT mode stay when you leave it; every other parameter returns to the pattern's kit.", - "category": "Performance", - "author": "toonst", - "needs_core": "1.0", - "on_unit": "The author reports perform-levels working on Digitakt II OS 1.17. The pair with perform-direct is not reported tested together on a unit." - }, - { - "file": "machine-pages-1.2-os1.53.elemod", - "sha256": "93092ba38bf65583a3d436e3d8c560b0df0480477f81127fe9de7a3264ba38c6", - "id": "machine-pages", - "version": "1.2", - "device": "digitakt-mk1", - "os": "1.53", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "a74bd736c26b235946e3d2e5c4466746de968cd9", - "path": "sdk/imports/packages/machine-pages/machine-pages-1.2-os1.53.elemod" - }, - "title": "machine pages", - "requires": [ - "core" - ], - "conflicts": [ - "digichain" - ], - "summary": "Draws the SRC and LFO pages of the SRC machines that describe their own, and gives the render an event for machines that make or take a track's sound, so several machine mods combine. irpina's Machine Pages 1.1, adapted by sumedho to run on Core 2.1. On its own, and for machines that do not describe a page, it changes nothing the unit does.", - "description": "Draws the SRC and LFO pages of the SRC machines that describe their own, and gives the render an event for machines that make or take a track's sound, so several machine mods combine. irpina's Machine Pages 1.1, adapted by sumedho to run on Core 2.1. On its own, and for machines that do not describe a page, it changes nothing the unit does.", - "category": "System", - "author": "irpina", - "needs_core": "2.1", - "on_unit": "Shared dependency of sumedho’s Digitakt mods; included automatically." - }, - { - "file": "machine-pages-1.2-os1.54.elemod", - "sha256": "1ab853a2ff628dd60b8fa0c5693f3babac64b95c456b331162797aac1526b1a0", - "id": "machine-pages", - "version": "1.2", - "device": "digitakt-mk1", - "os": "1.54", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "a74bd736c26b235946e3d2e5c4466746de968cd9", - "path": "sdk/imports/packages/machine-pages/machine-pages-1.2-os1.54.elemod" - }, - "title": "machine pages", - "requires": [ - "core" - ], - "conflicts": [ - "digichain" - ], - "summary": "Draws the SRC and LFO pages of the SRC machines that describe their own, and gives the render an event for machines that make or take a track's sound, so several machine mods combine. irpina's Machine Pages 1.1, adapted by sumedho to run on Core 2.1. On its own, and for machines that do not describe a page, it changes nothing the unit does.", - "description": "Draws the SRC and LFO pages of the SRC machines that describe their own, and gives the render an event for machines that make or take a track's sound, so several machine mods combine. irpina's Machine Pages 1.1, adapted by sumedho to run on Core 2.1. On its own, and for machines that do not describe a page, it changes nothing the unit does.", - "category": "System", - "author": "irpina", - "needs_core": "2.1", - "on_unit": "Shared dependency of sumedho’s Digitakt mods; included automatically." - }, - { - "file": "dt-support-1.0.1-os1.53.elemod", - "sha256": "2164a3bf1d89743189616127a5bc7e93743f2a691502d4750f517915fbd0180c", - "id": "dt-support", - "version": "1.0.1", - "device": "digitakt-mk1", - "os": "1.53", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "a74bd736c26b235946e3d2e5c4466746de968cd9", - "path": "sdk/imports/packages/dt-support/dt-support-1.0.1-os1.53.elemod" - }, - "title": "Digitakt feature support", - "requires": [ - "core" - ], - "conflicts": [], - "summary": "Shared allocation-lifecycle hook for CC Names and Euclidean. Does not add controls.", - "description": "Shared allocation-lifecycle hook for CC Names and Euclidean. Does not add controls.", - "category": "System", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "Shared dependency of sumedho’s Digitakt mods; included automatically." - }, - { - "file": "dt-support-1.0.1-os1.54.elemod", - "sha256": "7378c51c4970df68a47e51c9a85402b91577a1495931f2ef4ec566a572dd0b88", - "id": "dt-support", - "version": "1.0.1", - "device": "digitakt-mk1", - "os": "1.54", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "a74bd736c26b235946e3d2e5c4466746de968cd9", - "path": "sdk/imports/packages/dt-support/dt-support-1.0.1-os1.54.elemod" - }, - "title": "Digitakt feature support", - "requires": [ - "core" - ], - "conflicts": [], - "summary": "Shared allocation-lifecycle hook for CC Names and Euclidean. Does not add controls.", - "description": "Shared allocation-lifecycle hook for CC Names and Euclidean. Does not add controls.", - "category": "System", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "Shared dependency of sumedho’s Digitakt mods; included automatically." - }, - { - "file": "midi-retrig-2.0.1-os1.53.elemod", - "sha256": "ffe2e60aba95ae09df36ecc3d9e5d185c64e97f52fa02c1ff4ca84e9709486b9", - "id": "midi-retrig", - "version": "2.0.1", - "device": "digitakt-mk1", - "os": "1.53", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "d2f91e3d710acfd1f438bfa996b51b67a77a1534", - "path": "sdk/imports/packages/midi-retrig/midi-retrig-2.0.1-os1.53.elemod" - }, - "title": "MIDI Retrig", - "requires": [ - "core" - ], - "conflicts": [], - "summary": "Use the native retrig menu on MIDI tracks A–H. Repeated notes follow the Digitakt sequencer, including chords, retrig rate, finite length and velocity changes.", - "description": "Use the native retrig menu on MIDI tracks A–H. Repeated notes follow the Digitakt sequencer, including chords, retrig rate, finite length and velocity changes.", - "category": "Sequencer", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "The author reports this build fully tested and working with all six mods on a Digitakt 1, OS 1.53 and 1.54, for about one to two hours including project save/reload and a power cycle (2026-10-14)." - }, - { - "file": "midi-retrig-2.0.1-os1.54.elemod", - "sha256": "29d1f2f99bf47c4688808253a19cfad0627845370170949b19159640f2116d3d", - "id": "midi-retrig", - "version": "2.0.1", - "device": "digitakt-mk1", - "os": "1.54", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "d2f91e3d710acfd1f438bfa996b51b67a77a1534", - "path": "sdk/imports/packages/midi-retrig/midi-retrig-2.0.1-os1.54.elemod" - }, - "title": "MIDI Retrig", - "requires": [ - "core" - ], - "conflicts": [], - "summary": "Use the native retrig menu on MIDI tracks A–H. Repeated notes follow the Digitakt sequencer, including chords, retrig rate, finite length and velocity changes.", - "description": "Use the native retrig menu on MIDI tracks A–H. Repeated notes follow the Digitakt sequencer, including chords, retrig rate, finite length and velocity changes.", - "category": "Sequencer", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "The author reports this build fully tested and working with all six mods on a Digitakt 1, OS 1.53 and 1.54, for about one to two hours including project save/reload and a power cycle (2026-10-14)." - }, - { - "file": "euclidean-1.1.4-os1.53.elemod", - "sha256": "da551e67332ffa070bd9d51879319fb4e4a5807eea684555860c0179cb65db20", - "id": "euclidean", - "version": "1.1.4", - "device": "digitakt-mk1", - "os": "1.53", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "6534fd648f5b2911a6ec7e7284c9dabc6afa021f", - "path": "sdk/imports/packages/euclidean/euclidean-1.1.4-os1.53.elemod" - }, - "title": "Euclidean Sequencing", - "requires": [ - "core", - "dt-support" - ], - "conflicts": [], - "summary": "Generate rhythms from two pulse generators on each audio or MIDI track. Rotate either generator, rotate the whole result and combine them with OR, XOR, AND or SUB.", - "description": "Generate rhythms from two pulse generators on each audio or MIDI track. Rotate either generator, rotate the whole result and combine them with OR, XOR, AND or SUB.", - "category": "Sequencer", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "The author reports this build fully tested and working with all six mods on a Digitakt 1, OS 1.53 and 1.54, for about one to two hours including project save/reload and a power cycle (2026-10-14)." - }, - { - "file": "euclidean-1.1.4-os1.54.elemod", - "sha256": "7093ef6573081f1bb5180e8576bdcba92359ddf8e91aec0ec666f306464a068d", - "id": "euclidean", - "version": "1.1.4", - "device": "digitakt-mk1", - "os": "1.54", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "6534fd648f5b2911a6ec7e7284c9dabc6afa021f", - "path": "sdk/imports/packages/euclidean/euclidean-1.1.4-os1.54.elemod" - }, - "title": "Euclidean Sequencing", - "requires": [ - "core", - "dt-support" - ], - "conflicts": [], - "summary": "Generate rhythms from two pulse generators on each audio or MIDI track. Rotate either generator, rotate the whole result and combine them with OR, XOR, AND or SUB.", - "description": "Generate rhythms from two pulse generators on each audio or MIDI track. Rotate either generator, rotate the whole result and combine them with OR, XOR, AND or SUB.", - "category": "Sequencer", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "The author reports this build fully tested and working with all six mods on a Digitakt 1, OS 1.53 and 1.54, for about one to two hours including project save/reload and a power cycle (2026-10-14)." - }, - { - "file": "ram-machines-1.3.1-os1.53.elemod", - "sha256": "75cad6c7148709ea1e9188def89ebae200497ad78b295d9762a4af83dca15a6d", - "id": "ram-machines", - "version": "1.3.1", - "device": "digitakt-mk1", - "os": "1.53", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "17bcb8f756f8f1c1d21fe84750b67d032d3d9bc0", - "path": "sdk/imports/packages/ram-machines/ram-machines-1.3.1-os1.53.elemod" - }, - "title": "RAM Record and Playback", - "requires": [ - "core", - "machine-pages" - ], - "conflicts": [], - "summary": "RAM-R1 records a temporary mono take from the master mix or inputs when triggered. RAM-P1 plays that shared take using the native sample controls.", - "description": "RAM-R1 records a temporary mono take from the master mix or inputs when triggered. RAM-P1 plays that shared take using the native sample controls.", - "category": "Sampling", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "The author reports this build fully tested and working with all six mods on a Digitakt 1, OS 1.53 and 1.54, for about one to two hours including project save/reload and a power cycle (2026-10-14)." - }, - { - "file": "ram-machines-1.3.1-os1.54.elemod", - "sha256": "0e6d4ca589b84da505b29e3b245b326635c1148730a86d87a93c8ff0dad221eb", - "id": "ram-machines", - "version": "1.3.1", - "device": "digitakt-mk1", - "os": "1.54", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "17bcb8f756f8f1c1d21fe84750b67d032d3d9bc0", - "path": "sdk/imports/packages/ram-machines/ram-machines-1.3.1-os1.54.elemod" - }, - "title": "RAM Record and Playback", - "requires": [ - "core", - "machine-pages" - ], - "conflicts": [], - "summary": "RAM-R1 records a temporary mono take from the master mix or inputs when triggered. RAM-P1 plays that shared take using the native sample controls.", - "description": "RAM-R1 records a temporary mono take from the master mix or inputs when triggered. RAM-P1 plays that shared take using the native sample controls.", - "category": "Sampling", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "The author reports this build fully tested and working with all six mods on a Digitakt 1, OS 1.53 and 1.54, for about one to two hours including project save/reload and a power cycle (2026-10-14)." - }, - { - "file": "note-accumulator-1.1.1-os1.53.elemod", - "sha256": "5e52d94a0991469c2efc83531d96fb42ffacc6de11aa287de6df54929f500f23", - "id": "note-accumulator", - "version": "1.1.1", - "device": "digitakt-mk1", - "os": "1.53", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "f61a8ae5328e4ef816921c2e686bac477131459e", - "path": "sdk/imports/packages/note-accumulator/note-accumulator-1.1.1-os1.53.elemod" - }, - "title": "Per-step Note Accumulator", - "requires": [ - "core" - ], - "conflicts": [], - "summary": "Each enabled audio step remembers its own pitch increment between visits. Positive values climb, negative values descend, and the pitch wraps to the original note at the chosen range.", - "description": "Each enabled audio step remembers its own pitch increment between visits. Positive values climb, negative values descend, and the pitch wraps to the original note at the chosen range.", - "category": "Sequencer", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "The author reports this build fully tested and working with all six mods on a Digitakt 1, OS 1.53 and 1.54, for about one to two hours including project save/reload and a power cycle (2026-10-14)." - }, - { - "file": "note-accumulator-1.1.1-os1.54.elemod", - "sha256": "1f9004c33e91eb8f7eba0ba44c8c0f0dcd2c3731116218f510ab04b7fbdd471e", - "id": "note-accumulator", - "version": "1.1.1", - "device": "digitakt-mk1", - "os": "1.54", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "f61a8ae5328e4ef816921c2e686bac477131459e", - "path": "sdk/imports/packages/note-accumulator/note-accumulator-1.1.1-os1.54.elemod" - }, - "title": "Per-step Note Accumulator", - "requires": [ - "core" - ], - "conflicts": [], - "summary": "Each enabled audio step remembers its own pitch increment between visits. Positive values climb, negative values descend, and the pitch wraps to the original note at the chosen range.", - "description": "Each enabled audio step remembers its own pitch increment between visits. Positive values climb, negative values descend, and the pitch wraps to the original note at the chosen range.", - "category": "Sequencer", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "The author reports this build fully tested and working with all six mods on a Digitakt 1, OS 1.53 and 1.54, for about one to two hours including project save/reload and a power cycle (2026-10-14)." - }, - { - "file": "cc-names-2.1.1-os1.53.elemod", - "sha256": "24aff56846b356b6f0c78459ee38854980804d39949d149076e53f24fe0755ac", - "id": "cc-names", - "version": "2.1.1", - "device": "digitakt-mk1", - "os": "1.53", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "fd9dafa79451497fa0106b0a5ca19f81bfe31370", - "path": "sdk/imports/packages/cc-names/cc-names-2.1.1-os1.53.elemod" - }, - "title": "CC Assignment Names", - "requires": [ - "core", - "dt-support" - ], - "conflicts": [], - "summary": "Give each MIDI CC assignment a label of up to five characters. Labels appear on CC VALUE and follow native pattern and project operations.", - "description": "Give each MIDI CC assignment a label of up to five characters. Labels appear on CC VALUE and follow native pattern and project operations.", - "category": "MIDI", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "The author reports this build fully tested and working with all six mods on a Digitakt 1, OS 1.53 and 1.54, for about one to two hours including project save/reload and a power cycle (2026-10-14)." - }, - { - "file": "cc-names-2.1.1-os1.54.elemod", - "sha256": "78de45bbe8a206422f14c227b5a1b4db7214b388ec06410c7513385287627459", - "id": "cc-names", - "version": "2.1.1", - "device": "digitakt-mk1", - "os": "1.54", - "license": "GPL-2.0-or-later", - "source": { - "repo": "repeat98/modwerk", - "commit": "fd9dafa79451497fa0106b0a5ca19f81bfe31370", - "path": "sdk/imports/packages/cc-names/cc-names-2.1.1-os1.54.elemod" - }, - "title": "CC Assignment Names", - "requires": [ - "core", - "dt-support" - ], - "conflicts": [], - "summary": "Give each MIDI CC assignment a label of up to five characters. Labels appear on CC VALUE and follow native pattern and project operations.", - "description": "Give each MIDI CC assignment a label of up to five characters. Labels appear on CC VALUE and follow native pattern and project operations.", - "category": "MIDI", - "author": "sumedho", - "needs_core": "2.1", - "on_unit": "The author reports this build fully tested and working with all six mods on a Digitakt 1, OS 1.53 and 1.54, for about one to two hours including project save/reload and a power cycle (2026-10-14)." } ] } diff --git a/vendor/licenses/digi-mods.txt b/vendor/licenses/digi-mods.txt index 4395ca469..15fe1c23b 100644 --- a/vendor/licenses/digi-mods.txt +++ b/vendor/licenses/digi-mods.txt @@ -1,4 +1,4 @@ -Digitakt/Digitone mod releases built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/catalog/catalog.json: +Digitakt/Digitone mod releases built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/upstream-catalog.json: - DIGISLICER 2.1 by irpina: https://github.com/irpina/digislicer/releases/tag/v2.1 - NEIGHBOR (digineighbor) 0.6 by irpina: https://github.com/irpina/digineighbor/releases/tag/v0.6 - digihealth 1.0 (Digitakt) and 1.1 (Digitone) by irpina: https://github.com/irpina/digihealth/releases/tag/v1.1 diff --git a/vendor/licenses/digi1-mods.txt b/vendor/licenses/digi1-mods.txt index 9754de5c9..d35fe5abb 100644 --- a/vendor/licenses/digi1-mods.txt +++ b/vendor/licenses/digi1-mods.txt @@ -1,4 +1,4 @@ -digi1_mods releases by gdeo607, built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/catalog/catalog.json to commit 35bacb3730d108e4dc48a7bd6de4c99ae9b161e6: +digi1_mods releases by gdeo607, built by the vendored elekloader builder, unmodified and pinned by SHA-256 in vendor/elekloader/upstream-catalog.json to commit 35bacb3730d108e4dc48a7bd6de4c99ae9b161e6: - digichain 1.6, Digi EQ 1.0b, Digi Matrix 1.0b, Digi Mono 0.13b, Digi Poly 2.0 and Digi utilities 1.9a, each for Digitakt OS 1.53 and 1.54: https://github.com/gdeo607/digi1_mods/tree/35bacb3730d108e4dc48a7bd6de4c99ae9b161e6/elemods The imported sources and documentation are in sdk/digitakt/modules. diff --git a/vendor/licenses/manifest.json b/vendor/licenses/manifest.json index 6179b1e34..d4a816fb6 100644 --- a/vendor/licenses/manifest.json +++ b/vendor/licenses/manifest.json @@ -30,7 +30,7 @@ "noticeFile": "digi-mods.txt", "sources": [ { - "path": "vendor/elekloader/catalog/catalog.json" + "path": "vendor/elekloader/upstream-catalog.json" }, { "path": "sdk/digitakt/modules/digislicer/LICENSE"