Skip to content

Octatrack base: DSP hook chains, so several packaged DSP machines share a hook site - #1418

Open
Zac-Kyoti wants to merge 12 commits into
repeat98:mainfrom
Zac-Kyoti:dsp-hook-chains
Open

Zac-Kyoti wants to merge 12 commits into
repeat98:mainfrom
Zac-Kyoti:dsp-hook-chains

Conversation

@Zac-Kyoti

Copy link
Copy Markdown
Contributor

@repeat98 The hook-slot PR you cleared on #1098: Phase 1's generic DSP hook slots, built as the per-feature chain you approved there ("the chain is what I meant by hook slots"), with the ledger rule as it is.

  • What was wrong or hard: the beta base has one packaged DSP engine slot at site 5 (the per-track sound source), and Analog BD holds it. A second packaged DSP machine (BlackPyramids is next, BlackPyramids_303 on Modwerk, beside Sidechain Compressor: a proposal and some questions #1064) could only install by taking the site from Analog BD or by a base built for it.
  • What this changes:
    • Chain sites. dsp-sites.json gives site 5 "kind": "chain" with its "features" in call order. The resident stub calls each enabled member's engine in turn (one long bsr each; a dormant member is a lone rts), then replays stock.
    • Leaving the chain. A member that renders the track leaves through its exit, which the receiver aims at the chain's trampoline: it drops the site's return, then takes the stock exit (mw_engine_exit_of(site, core), generated). A member that doesn't render returns, and the track passes on.
    • Regions and machines. Regions stay ledger-declared per feature ("neither the host nor a module can name a DSP address"), so a new DSP machine is a new ledger feature plus its trust row, in its own module PR.
    • Engine arrays follow the ledger (MW_ENGINE_FEATURES = MW_DSP_COUNT; the dev view stays 4).
    • Machine controllers are named by their port's "lease" in machine-ports.json: machine_lease_policy.py replaces the Analog BD-named policy, and build.py and base_bound.py read it. analog_bd_write_lease.py is untouched (Analog BD's native observations pin it).
    • A latent receiver bug fixed on the way: dsp_asm substitutes labels as text, so a label that prefixes another breaks (hwchk1 and hwchk16). Receiver labels are padded to three digits, and the reset cursors are generated by the receiver.
    • Regenerated:
      • The beta base. Package 1677e881…; the browser .bin equals build_core.py's, 396582db….
      • The base-bound packages (Analog BD 0.1.6, Output Matrix 0.1.1, Sidechain Compressor 0.2.1): only their calls into the base move. For Analog BD that's 3 bytes, its two lease calls.
      • Analog BD's machine recipe: imageSha256 6d3a27ab → 44ac9e07, the same recapture as the prototype.
      • build.py / machine-ports.json pins in the other machine ports (images unchanged), the standalone and candidate recipes, and Pattern Sample Locks' host recipe. Play Modes' recipe takes only build.py's new pin, as its declared lag needs.
  • How I checked it:
    • test_resident_* in the toolchain container: 44 tests, with the real resident engine in the DSP56300 simulator. test_two_packaged_engines_share_a_chain_site puts a second, made-up member (feature 16) beside Analog BD's: each renders only its own tracks, the stack is balanced at the site (stock SP = 0 at site 5 on both cores, ot_emu --dsp-pcwatch), and a mutated trampoline that skips the pop is caught.
    • The base build's own checks pass: the browser .bin equals the native one, every change sits inside a declared site, and the bootloader's Startup Menu recovery stays available.
    • The package scans pass, and octatrack:module-packages -- --check passes.
    • npm run pr:ready: Ready.
    • Not run: the new base on a unit. The board-level test of a second real machine comes with the BlackPyramids module PR. Would you run this base on your MKII as you did for Octatrack base: the key bus to Y $7E00, Sidechain's MON words to Y $FF0 (#1064) #1098 (Analog BD keyed, Keep/Undo, removal)? My MKI has no beta access yet (asked on Mute Modes 0.2.2-experimental: SHADOW out of the stock timestretch task's SRAM (owner review) #1007).
  • What the guides lacked or got wrong, and where I fixed it: docs/OCTATRACK_PACKAGE_DSP_HOOKS.md described one packaged engine per site and a status from 20 October. It now has the chain and the current status. Changelog: changelog/2026-11-03-octatrack-dsp-hook-chains.md.

Related to #1098 and #1064.

🤖 Generated with Claude Code

Zac-Kyoti and others added 4 commits November 10, 2026 16:25
…re a hook site (repeat98#1098)

dsp-sites.json gives site 5 the kind "chain" with its features in call order. The resident stub
calls each packaged member in turn (a dormant member is a lone rts), then replays stock; a packaged
engine leaves through the chain's trampoline. Engine arrays follow the ledger (MW_ENGINE_FEATURES =
MW_DSP_COUNT), and machine controllers are named by their port's "lease" in machine-ports.json
(machine_lease_policy.py replaces the Analog BD-named policy). Regions stay ledger-declared per
machine. Analog BD's package code is unchanged. Two-member chain test in test_resident_engines.

Replayed from the reviewed prototype (3e70e47d..32bbd838) onto current main; generated files are
main's here and regenerated in the following commits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ked (Analog BD, Output Matrix, Sidechain Compressor); Analog BD's recipe recaptured; build.py pins refreshed in the standalone and candidate recipes

Analog BD's machine recipe: imageSha256 6d3a27ab -> 44ac9e07 (the base's link moved) and its build.py
and machine-ports.json pins. Play Modes' standalone recipe takes only build.py's new pin (its other
pins stay 0.1.1's, as its declared lag needs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…orts' recipes (POLY8, Quantizer, Synth, VECTOR: images unchanged) and Pattern Sample Locks' host recipe

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@repeat98

Copy link
Copy Markdown
Owner

Thanks Zac, this is the chain I had in mind, and the code reads well: no hook index moves, no module source changes, and the dormant path costs what it does today. Three things before it can go in:

  1. Main now has Analog BD 0.1.7 (Analog BD 0.1.7: the recorded-909 engine at half the DSP load #1413), so the branch conflicts in the 0.1.6 package and octatrack-module-packages.json. Please merge main (no rebase), rebuild the base, relink 0.1.7 with its recipe and policy pins, and run pr:ready until it says Ready.
  2. In sdk/tests/test_resident_engines.py the new chain test has its skipUnless line twice, and test_disabling_one_engine_leaves_another_engines_base_word_alone just below lost its own, so it errors instead of skipping where the simulator build is missing.
  3. The new test leaves Analog BD out on core A ("voice faults here (LIMIT)"). What faults there, the harness or the engine? I'd like that case covered, or a line saying why it can't be.

Once it's refreshed it's ready for a run on a unit before it merges.

Zac-Kyoti and others added 4 commits November 11, 2026 15:07
…skipUnless per simulator test

Review on repeat98#1418. The core-A fault was the harness's: Analog BD's glue takes its voice block from
VBASE + 2 * X:$418 (the voice stock is rendering), which the test never set, so the filled random
word sent core A's block outside the interpreter's memory. With X:$418 = 0 core A's working path runs
to the stock exit (210 instructions, bit-exact placed vs resident), the 160-instruction LIMIT is gone,
and the chain test covers Analog BD on both cores. The chain test had its skipUnless twice and the
next test none, so that one errored without the simulator; each now has one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	public/octatrack-modules/analog-bassdrum/0.1.7-experimental.mwrm
#	sdk/machines/octatrack/base-binding.json
#	src/engine/assets/octatrack-base-variants.json
#	src/engine/assets/octatrack-base.elemod
#	src/engine/assets/octatrack-module-packages.json
…; 0.1.7 relinked to it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
….2 (the same code as 0.2.1)

The base moves, so the base-bound package has to be rebuilt; a declared lag keeps a package as it is
(octatrack-module-packages.py), so the lag goes and the USB package is 0.2.2: 0.2.1's code, relinked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Zac-Kyoti

Copy link
Copy Markdown
Contributor Author

@repeat98 Thanks. All three are pushed (93346531d):

  1. Merged main (no rebase) with Analog BD 0.1.7 and Sidechain Compressor 0.2.2: ColdFire units join the DRAM runtime when the set has one (#1175) #1195. I rebuilt the base (package 264c7998…; browser .bin = build_core.py's, 4cc20989…) and relinked 0.1.7 to it. Its recipe didn't move: 0.1.7 reconstructs to the same 44ac9e07… against this base. pr:ready says Ready.
    One decision for you: Sidechain Compressor 0.2.2: ColdFire units join the DRAM runtime when the set has one (#1175) #1195 had just landed with Sidechain's USB package lagging at 0.2.1. The base moves here, so that base-bound package has to be relinked, but octatrack-module-packages.py keeps a declared lag as published ("remove both keys to rebuild it"). So I removed the lag, and the USB package is now 0.2.2: 0.2.1's code, relinked to the chain base (0.2.2 changed only the card placement). If you'd rather keep a 0.2.1 label over USB, say so and I'll find another way.
  2. skipUnless: one per simulator test now. test_disabling_one_engine_leaves_another_engines_base_word_alone skips again without the simulator, and both skip on a machine without it.
  3. The core-A case: the harness, not the engine, and now covered. Analog BD's glue takes its voice block from VBASE + 2 * X:$418 (the voice stock is rendering). The tests never set X:$418, so the filled pseudo-random word sent core A's block to X:$f598xx, outside the interpreter's memory. Core B's random value happened to land inside it.
    • With X:$418 = 0 in Analog BD's active record, core A's working path runs to the stock exit: 210 instructions, bit-exact placed against resident.
    • The old 160-instruction LIMIT in test_placed_engine_runs_bit_exactly_as_the_resident_one is gone.
    • The chain test now runs Analog BD, the member and stock on both cores.
    • test_resident_* in the toolchain image: 44 OK.

Ready for a run on a unit whenever you have your MKII to hand.

@repeat98

Copy link
Copy Markdown
Owner

Thanks Zac, all three are in: one skipUnless per simulator test, Analog BD runs to its exit on core A with X:$418 set, and 0.1.7 is relinked to the rebuilt base.

Sidechain as 0.2.2 over USB: yes. The card builder is gone since this morning (#1451, #1453), so there's no card placement left for 0.2.2 to differ in, and the USB package should simply carry the catalogue's version.

#1453 landed on main after your push. It removes the card composer and touches neither the base nor your files, and the branch still merges cleanly, so there's nothing to refresh for it.

One question: twwwy noted on the board that test_resident_engines fails on a clean main where the DSP assembler is present, because UNIT_WORDS still expects 1,009 words for Analog BD's engine and 0.1.7 builds fewer. Your branch keeps 1,009 and you report 44 OK in the toolchain image. Which is it on your side: does the chain's engine come to 1,009 again, or did that test not run? CI can't tell us, it has no assembler.

What's left is my run on the MKII (Analog BD keyed, Keep/Undo, removal). I'll take this one first, since it's level with main; #1165 rebuilds the base as well and goes in after it. I'll write here once it has run.

Zac-Kyoti and others added 2 commits November 12, 2026 13:52
# Conflicts:
#	sdk/machines/octatrack/base-binding.json
#	src/engine/assets/octatrack-base-variants.json
#	src/engine/assets/octatrack-base.elemod
#	src/engine/assets/octatrack-module-packages.json
…ds since 0.1.7

test_resident_engines expected 0.1.6's 1,009 (twwwy, on the board): it fails on main as on this
branch wherever the DSP assembler is present; CI has none. The chain does not change the engine.
Sidechain Compressor's USB package stays 0.2.2 on the rebuilt base (agreed on repeat98#1418).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Zac-Kyoti

Copy link
Copy Markdown
Contributor Author

@repeat98 Thanks, and twwwy is right. My "44 OK" was out of date: I ran it before merging Analog BD 0.1.7 into the branch and didn't run the simulator tests again after the merge. That was my mistake.

  • What it is: UNIT_WORDS expected 0.1.6's 1,009 words for Analog BD's engine, and 0.1.7 builds 1,004. On a clean main with the assembler present, test_real_units_relocate_alike_in_c_in_python_and_freshly_assembled fails with 1004 != 1009, and the same on this branch. The chain doesn't change the engine's size.
  • Fixed here (f607b9646): UNIT_WORDS[4] = 1004, with a line saying it's 0.1.7's engine. main gets the fix when this merges.
  • This time after merging current main: test_resident_* in the toolchain image, 44 OK (1 skipped).
  • Also merged main again, which had rebuilt its base. The base is rebuilt with the chain (browser .bin = build_core.py's, 789af3f8…), the base-bound packages are relinked, Sidechain stays 0.2.2 over USB as agreed, and pr:ready says Ready.

Over to your MKII run.

@repeat98

Copy link
Copy Markdown
Owner

Thanks Zac, and for setting the record straight on the 1,004 words. The test fix is right.

#1492 landed this morning and rebuilt the base again, so the branch conflicts with main in the base files once more (base-binding.json, the .elemod, the base variants and sources, the module packages). Could you merge main one more time (no rebase), rebuild the base, relink the base-bound packages and run pr:ready until it says Ready? This one still goes in first; the other base pull requests rebuild after it.

The red module-suites run isn't yours: a Zoom link test hit its 5-second timeout, in code this branch doesn't touch. It runs again with your push.

My MKII run comes on that refreshed build.

Zac-Kyoti and others added 2 commits November 13, 2026 12:51
# Conflicts:
#	public/octatrack-modules/analog-bassdrum/0.1.7-experimental.mwrm
#	public/octatrack-modules/output-matrix/0.1.1-experimental.mwrm
#	public/octatrack-modules/sidechain-compressor/0.2.2-experimental.mwrm
#	sdk/machines/octatrack/base-binding.json
#	sdk/machines/octatrack/module-adapters/controller-policies.json
#	sdk/machines/octatrack/module-adapters/machine-recipes.json
#	src/engine/assets/octatrack-base-sources.json
#	src/engine/assets/octatrack-base-variants.json
#	src/engine/assets/octatrack-base.elemod
#	src/engine/assets/octatrack-module-packages.json
…ages relinked; Analog BD's recipe recaptured (cd17acc9) and build.py pins refreshed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Zac-Kyoti

Copy link
Copy Markdown
Contributor Author

@repeat98 Refreshed (f806213b3): main merged with #1492 (no rebase), base rebuilt with the chain (browser .bin = build_core.py's, 7ad232eb…), base-bound packages relinked (Analog BD 0.1.7, Output Matrix 0.1.1, Sidechain Compressor 0.2.2, CC Map, Recorder Loop Fix), Analog BD's recipe recaptured against it (3b1c41ae → cd17acc9, image only), and build.py pins refreshed. Sidechain stays 0.2.2 over USB as agreed. test_resident_* in the toolchain image after the merge: 44 OK. pr:ready says Ready. Ready for your MKII run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants