Skip to content

fix: let the orchestrator read the artifacts it is told to synthesize - #340

Merged
saucam merged 3 commits into
mainfrom
fix/orchestrator-blackboard-scope
Sep 24, 2026
Merged

saucam merged 3 commits into
mainfrom
fix/orchestrator-blackboard-scope

Conversation

@saucam

@saucam saucam commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

Fixes #338.

The reported defect

compileGoalPack tells the orchestrator to "read each role's artifact from the blackboard, merge the findings, de-duplicate, and SHOW disagreement". Its default profile read two kinds:

orchestrator: { reads: ["spec", "findings"], writes: ["spec", "task-list"] },

So blackboard_read kind=research returned Role "orchestrator" may not read "research" — it reads: spec, findings. research, adr and diff were all unreachable, and synthesis collapsed into restating findings — the one kind it could get.

Why the narrowing was not a policy

service.ts justifies review's two-kind read set with "a panel whose members can read each other is not a panel; it's an echo". That is a property of peers. The orchestrator is not a panel member — §7 makes it the synthesizer and the only agent that reports to the owner. Applying peer isolation to it was an unintended consequence of one table serving both jobs.

It now reads every core kind, written as [...CORE_ARTIFACT_KINDS] so a seventh kind is readable by the synthesizer the day it's added. review is untouched: still no research, no adr, no peer's findings — there's a test that asserts the widening did not move by one kind.

Read scope isn't an obligation to read. §4's "holds an index, not the artifacts" is context economics, and the index carries per-artifact byte counts precisely so the orchestrator can choose; the constitution now says to coordinate from the index rather than keep a mirror of the board.

The second defect: the escape hatch had no door

reads/writes were accepted on the wire and honored by resolveRoleIo, but nothing user-facing produced them. Both paths from the issue's table are now real:

Path Before Now
CLI --role no syntax name:provider[:model][*count][+reads=a,b][+writes=c]
Pack role YAML no field reads: / writes: beside write/network/envelope
config.json no still no — see below
codeoid new probe /repo --collaborate "…" \
  --role orchestrator:claude \
  --role search:claude+reads=spec,adr+writes=research,extra/sources

Details that matter:

  • + splits first, before the *count suffix and the : walk, so neither has to know about scope. The cost is that + can't appear in a model id, which no backend's ids use.
  • +reads= is an empty list, and stays distinct from declaring nothing — the one distinction resolveRoleIo exists to make (declared-empty = touch nothing; absent = fall back to the §3 profile).
  • The pack's declaration is authoritative on the same terms as write — a spec that also declares one is an error, not a silent override. Conditionally, though: unlike write it's optional in the YAML, so a pack with no opinion leaves the spec's declaration (or the default profile) in place and every existing pack is unaffected.
  • The pipeline --role path rejects the scope segments rather than accepting and ignoring them. A phase's reads/writes aren't consumed until P4, and an accepted-but-unenforced fence is this design's own "an unenforced field is false security". Same treatment *count already gets.
  • Bounded at LIMITS.COLLABORATION_ROLE_SCOPE_MAX (16) in the CLI, the wire schema and validateCollaboration — the last one because an embedded frontend holds the SessionManager directly and never crosses parseClientMessage, so the schema's .max() was not the only door.

The third fix: one resolver, one formatter

The root cause wasn't really the table — it was that the constitution was never derived from the orchestrator's scope. childBrief has always called resolveRoleIo (which is why no child ever drifted); compileGoalPack called nothing.

Both now render from one blackboardScopeLines(io) over one resolveRoleIo call, so a narrowed orchestrator is told the truth rather than the default, and the constitution names the four blackboard tools it actually mounts. Making the disagreement unrepresentable, rather than merely absent today.

Confirmed while in the same table

write() checks writes and not reads, so search can publish research it can't read back. Deliberate, and now documented in place: read-back on write would hand review its own findings kind and from there its peers' entries, collapsing the property the file exists to hold. Nothing is lost — writes append rather than overwrite, and a role's own output is already in its transcript. A role that genuinely needs its prior version declares the kind in reads.

Deliberately not done

  • config.json — pipeline.modelTiers/modelRoles are machine policy (which model serves a class here). A blackboard scope is a property of the role, not of the machine, so the role YAML and the spec are its homes.
  • extra/<key> wildcards — the scope model is a list of kinds. A pack handing work off through extra/ names those keys; there's a test pinning that, since it's the one case the widened default doesn't cover.
  • The web create dialog — no per-role scope field; it relies on the default profile, which is what this PR corrects. §9 keeps that surface lean on purpose, and a scope editor is a separate UX change.

Verification

  • Repro closed at both layers: the service (search writes research → orchestrator reads it) and over the live MCP transport (all four kinds read back through a minted orchestrator token, and the index states the widened scope).
  • End-to-end: a --role spec's +reads=/+writes= through parseRoleSpec → the real wire schema → create → SessionInfo → plannedChildFor → the brief and the forRole fence.
  • Drift guard: the constitution contains You can READ: ${resolveRoleIo(ORCHESTRATOR_ROLE).reads.join(", ")} verbatim, plus a narrowed-orchestrator case.
  • Pack loader: reads/writes parse, absent stays absent, [] is preserved, bounds reject.
  • bun run typecheck · bun run lint · bun test → 2581 pass, 19 skip, 0 fail.

🤖 Generated with Claude Code

The compiled goal pack tells a collaboration's orchestrator to "read each
role's artifact from the blackboard, merge the findings, de-duplicate, and SHOW
disagreement". Its default profile read two kinds — `spec` and `findings` — so
`research`, `adr` and `diff` came back as `Role "orchestrator" may not read
"research"`. The instruction required what the fence refused, and synthesis
collapsed into restating the one artifact it could reach.

The narrow read set was borrowed from a property that does not apply here.
`review` reads exactly `spec`+`diff` because a panel whose members can read
each other is an echo, not a panel — that is about PEERS. The orchestrator is
not a panel member: §7 makes it the synthesizer and the only agent that reports
to the owner. So it now reads every core kind (spelled `[...CORE_ARTIFACT_KINDS]`,
so a seventh kind is readable the day it is added), and `review` is untouched —
still no `research`, no `adr`, no peer's `findings`.

Read scope is not an obligation to read. §4's "holds an index, not the
artifacts" is context economics, and the index carries per-artifact byte counts
precisely so the orchestrator can choose; the constitution now says to
coordinate from the index rather than mirror the board.

Two defects in the same table went with it.

The declared-scope escape hatch was unreachable. `reads`/`writes` were accepted
on the wire and honored by `resolveRoleIo`, but nothing user-facing produced
them: `parseRoleSpec` had no syntax and `roleSchema` had no field, so §3's
"adding a role stays a config change" was false for the one field that scopes
handoffs. A `--role` spec now takes
`name:provider[:model][*count][+reads=a,b][+writes=c]`, and a pack role YAML
takes `reads:`/`writes:` beside `write`/`network`/`envelope`. The pack's
declaration is authoritative on the same terms as `write` — a spec that also
declares one is an error, not a silent override — but conditionally, since
unlike `write` it is optional in the YAML, so an existing pack's roles keep
their default profile. `+reads=` is an empty list, which stays distinct from
declaring nothing. The pipeline `--role` path rejects the scope segments
instead of accepting and ignoring them: a phase's scope is not enforced until
P4, and an accepted-and-ignored fence is this design's own "an unenforced field
is false security".

And the orchestrator's constitution was not derived from its own scope at all.
`childBrief` has always rendered READ/WRITE from `resolveRoleIo`, which is why
no child ever drifted; the constitution rendered nothing, which is how this bug
existed in the first place. Both now go through one formatter over one
resolver, and the constitution names the four blackboard tools it mounts.

Confirmed while in the same table: `write()` checks `writes` and not `reads`,
so a role can publish a kind it cannot read back. Deliberate, and documented in
place — read-back on write would hand `review` its own `findings` kind and from
there its peers' entries. Nothing is lost: writes append, and a role's own
output is already in its transcript.

The web create dialog has no per-role scope field and keeps relying on the
default profile, which is what this changes; §9 keeps that surface lean on
purpose.

Fixes #338

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two rounds of review over the first commit. Everything below is a defect in
that commit or one it made reachable.

**A restarted orchestrator got its fence back but not its constitution.**
The biggest one, and the same bug in the other direction. `compileGoalPack`
ran only at create and nothing persists the rendered text, while resume always
re-attached the blackboard mount — so after a restart the orchestrator held all
four blackboard tools and reads on every core kind with no goal, no roster and
no statement of its own scope. Resume now recompiles from the same persisted
config the fence resolves against. Recompiled rather than stored on purpose: a
stored copy would be a snapshot of whatever the profile said the day it was
written, so upgrading the daemon would start the drift over. (A pack-adopted
collaboration's ETHOS and real pack id still don't survive — the adoption
itself isn't persisted. Out of scope, and noted in the code.)

**The constitution still hardcoded scope claims it should have derived.**
Two survived round one. `blackboard_write — publish your own artifact (the
shared spec, the task list)` was false under `+writes=`, and the synthesis
bullet told a `+reads=spec`-narrowed orchestrator to `blackboard_read_all` on
`findings` — a call its own fence refuses. Both derive from `io` now. The guard
test for this is scoped to the blackboard section deliberately: "Panels and
synthesis" naming `findings` is a fact about panels, not a scope claim.

**Deleting "you pass it deliberately" deadlocked every shipped pack.**
Replacing it with "name the artifact and let it read" assumed a role has a read
scope. `implementer`, `reviewer`, `adversary`, `navigator`, `verifier` — every
role in every installed pack — resolve to `{[], []}`, so the child was told it
could read nothing while the orchestrator was told to hand it nothing. The
rules now carry both paths: point at the artifact when the role can read it,
write what it needs into the task when it can't, and never use the second to
defeat the first.

**Pack-vs-spec scope precedence was per field, which was an escalation.**
A pack locking only `reads` left `writes` operator-overridable: `--pack p
--role review:gemini+writes=spec` let a reviewer publish the singleton `spec`
every other role reads. The mirror is worse — lock only `writes`, pass
`+reads=research`, and a reviewer sees the implementer's reasoning. A pack that
states either field now owns the role's scope as a unit.

**`split("+")` seized `+` across the whole grammar.**
Including role names, provider ids and model ids, on the pipeline path too, and
it reported the seizure as `unknown scope field "some"`. Model ids are
free-form operator input on several backends. A `+` now starts a segment only
when a field name or a `word=` follows it, so `reasoning:qwen:some+model`
parses as a model — verified against a live daemon — while `+reads` (no `=`)
and `+peeks=diff` still get their specific errors.

**The round-one `*count` hint truncated the user's own input.**
`value.replace(/\*.*$/, "")` is greedy, so `spec,diff*3,adr` suggested
`spec,diff` — valid, silently missing `adr`, and fatal much later as an agent
waiting on a handoff it was never scoped to read. The example is static now.

Also: a pack role's artifact kinds are validated when the pack LOADS, not only
when a collaboration adopts it (a role used by a pipeline phase never reaches
`validateCollaboration`, so a typo'd `reads: ["diffs"]` loaded clean and fenced
nothing); `.max()` aborts before `superRefine` so an oversized list doesn't
allocate an issue per entry; `resolveRoleIo` copies instead of handing out the
live `DEFAULT_ROLE_IO` array; `adoptPackRoles` copies instead of aliasing the
pack registry's own array; the "unknown artifact kind" sentence has one
formatter instead of three hand-copies; `ROLE_SCOPE_FIELDS` is one list instead
of four inline literals.

And a warning that didn't exist: a declared scope can dissolve panel
independence (a role that writes and reads `findings`, or writes `findings` and
reads `research`). §7's cross-critique round wants exactly that, so it stays
allowed — but it is reported on the create response rather than happening
quietly, because until this PR it took a hand-written WebSocket client and now
it takes one flag.

Verified end to end against a live daemon, not only in tests: the issue's own
repro — a `search` child publishes `research`, the orchestrator is asked to
read it — now returns the body, with the model reasoning "research falls within
my read scope" from the constitution. After a real restart the same
orchestrator still names its goal and recites `spec, research, adr, task-list,
diff, findings`. The resume guard is mutation-tested: it fails without the fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@saucam

saucam commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator Author

Two review rounds → 10 fixes folded in (78c7e04)

Ran two adversarial review passes over the first commit. Everything below was a defect in it, or one it made reachable.

The big one: the same bug on resume

compileGoalPack ran only at create and nothing persists the rendered text, while resume re-attached the blackboard mount unconditionally. So after a daemon restart the orchestrator came back holding all four blackboard tools and reads on every core kind — with no goal, no roster, and no statement of its scope. #338's asymmetry, reopened on every restart, and the create-path guard tests couldn't see it.

Resume now recompiles from the same persisted config the fence resolves against. Recompiled rather than stored deliberately: a stored copy is a snapshot of whatever the profile said the day it was written, so a daemon upgrade would restart the drift. Mutation-tested — the new guard fails when the fix is removed.

Known remaining gap, documented in the code: a pack-adopted collaboration's ETHOS and real pack id still don't survive a restart, because the adoption itself isn't persisted. Wider change than this issue.

Constitution claims that were still hardcoded

Two survived round one. blackboard_write — publish your own artifact (the shared spec, the task list) was false under +writes=; the synthesis bullet told a +reads=spec-narrowed orchestrator to blackboard_read_all on findings, a call its own fence refuses. Both derive from io now.

A regression the first commit introduced

Replacing "you pass it deliberately" with "name the artifact and let it read" assumed roles have a read scope. implementer, reviewer, adversary, navigator, verifier — every role in every installed pack — resolve to {[], []}. The child was told it could read nothing while the orchestrator was told to hand it nothing: deadlock. The rules now carry both paths, plus the rule not to use the second to defeat the first.

Pack-vs-spec precedence was an escalation

Per field, a pack locking only reads left writes operator-overridable — --pack p --role review:gemini+writes=spec let a reviewer publish the singleton spec every other role reads. Mirror case is worse: lock only writes, pass +reads=research, reviewer sees the implementer's reasoning. A pack that states either field now owns the role's scope as a unit.

split("+") seized + across the whole grammar

Role names, provider ids, model ids — on the pipeline path too — and it reported the seizure as unknown scope field "some". Model ids are free-form operator input on several backends. A + now starts a segment only when a field name or word= follows, so reasoning:qwen:some+model parses as a model while +reads and +peeks=diff keep their specific errors.

The round-one *count hint truncated the user's input

value.replace(/\*.*$/, "") is greedy: spec,diff*3,adr suggested spec,diff — valid, silently missing adr, fatal much later as an agent waiting on a handoff it was never scoped to read. Static example now.

Smaller, same commit

Pack role kinds validated at load (a role used only by a pipeline phase never reaches validateCollaboration, so a typo'd reads: ["diffs"] loaded clean and fenced nothing) · .max() aborts before superRefine · resolveRoleIo copies instead of handing out the live DEFAULT_ROLE_IO array · adoptPackRoles copies instead of aliasing the pack registry's array · one formatter for the "unknown artifact kind" sentence instead of three hand-copies · one ROLE_SCOPE_FIELDS instead of four inline literals · doc drift in §4 and role-model-binding §2.3.

New warning

A declared scope can dissolve panel independence (a role that writes and reads findings, or writes findings and reads research). §7's cross-critique round wants exactly that, so it stays allowed — but it's now reported on the create response instead of happening silently, because until this PR it took a hand-written WebSocket client and now it takes one flag.

End-to-end, against a live daemon

Not just tests — codeoid start --local on an isolated HOME, real Claude backend:

Check Result
--role search:claude+peeks=spec CLI: unknown scope field "peeks"
--role search:claude+reads=diffs daemon: Role "search" reads unknown artifact kind "diffs" — valid: …
--role search:claude+reads=spec*3 CLI: the fan-out count goes on the backend, before any scope
--role search:qwen:some+model parsed as a model id, not eaten by the grammar
The issue's repro — search publishes research, orchestrator reads it returned the body; the model's own reasoning: "research falls within my read scope and should succeed"
After a real daemon restart orchestrator still names its goal and recites spec, research, adr, task-list, diff, findings

bun run typecheck · bun run lint · bun test → 2590 pass, 19 skip, 0 fail (+9 tests).


Out of scope, flagged separately: several review passes reported RESUME_MAX_SESSIONS 50→200 (4× the resident-scrollback ceiling against an unchanged 20s deadline). That is an uncommitted local working-tree edit on this machine, not part of this PR — it is deliberately excluded from both commits. Also pre-existing and untouched: task-list has two default writers but no slot in MULTI_WRITER_KINDS; resume drops a pack-adopted child's network/envelope; roleSchema isn't .strict(), so a typo'd key is silently stripped; the web create modal still has no per-role scope field.

…open

A gap sweep over the previous commit found the declared-scope feature had zero
coverage at the one place the daemon applies it, and confirmed it by mutation:
replacing the scope argument in `#blackboardMountFor` with `undefined` left all
2585 tests green.

Worse than untested. The scope came from
`collaboration.roles.find((r) => r.name === child.roleName)` — an exact-match
lookup in a module that compares role names case-insensitively everywhere else.
A miss yields `undefined`, `resolveRoleIo` falls back to the §3 profile, and a
`+reads=`-narrowed reviewer comes back on a WIDER scope than it declared with
its brief still stating the narrow one. Failing open, silently, at the seam
where a declared scope becomes a real fence.

`PlannedChild` already carries `reads`/`writes`, and it is what `childBrief`
resolves its READ/WRITE lines from, so the lookup was a second derivation of a
fact already in hand. Gone: the mount now reads the same field the brief does.
The orchestrator's synthetic `PlannedChild` carries its own declared scope for
the same reason.

The new guard observes what the daemon MINTS — `RoleBlackboard` exposes its
resolved scope, so spying on `mint` pins the fence itself rather than
rebuilding a handle and re-testing `forRole`. Mutation-tested: it fails when
the argument is dropped.

Two more from the same sweep.

`blackboard/types.ts` still carried the invariant this change makes false —
"the orchestrator holds an index ... never the artifact bodies" and "all the
orchestrator ever needs". That file defines the vocabulary, so it is where the
next person hardening scope would look, and it would have argued them straight
back into #338. It now says what §4 actually means: context economics, not
permission.

And the constitution gave the orchestrator no opening move. Every default
worker read set is rooted at `spec`, every child brief ends "wait for
instructions", and nothing told the orchestrator to publish one — so a goal
whose first dispatch is `search` stalls on turn one with a searcher reading
"no spec has been written on this goal yet" and no scope to read anything
else. Derived like every other claim in that section: an orchestrator that
cannot write `spec` is not told to.

Also: the "share one formatter" guard asserted nothing — `toContain("You can
READ: ")` is satisfied by a hardcoded "You can READ: spec, findings", which is
the #338 bug verbatim. It compares against `resolveRoleIo` on both sides now,
and checks the two sets differ so it cannot pass on a formatter that ignores
its argument.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@saucam

saucam commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator Author

Gap sweep → 39bb737 (CI green)

The round-2 sweep found one more real defect after the fixes landed, and mutation-confirmed it.

The declared-scope feature had zero coverage at the one place the daemon applies it. Replacing the scope argument in #blackboardMountFor with undefined left all 2585 tests green — every hop around the seam was tested (parse, validate, adopt, plan, brief, forRole), but not the call that hands the manager's derivation to the fence.

Worse than untested: the scope came from collaboration.roles.find((r) => r.name === child.roleName), an exact-match lookup in a module that compares role names case-insensitively everywhere else. A miss yields undefined → resolveRoleIo falls back to the §3 profile → a +reads=-narrowed reviewer comes back on a wider scope than it declared, with its brief still stating the narrow one. Failing open, silently, at the seam this PR exists to make trustworthy.

PlannedChild already carries reads/writes and is what childBrief resolves from, so the lookup was a second derivation of a fact already in hand. Removed — the mount now reads the same field the brief does, which is the PR's own thesis applied to the place I'd missed.

The new guard spies on mint and inspects what the daemon actually produces (RoleBlackboard exposes its resolved scope), rather than rebuilding a handle and re-testing forRole — a fair criticism of my first attempt. Mutation-tested: it fails when the argument is dropped.

Two more from the same sweep

  • blackboard/types.ts still carried the invariant this change falsifies — "the orchestrator holds an index … never the artifact bodies" and "all the orchestrator ever needs". That file defines the vocabulary, so it's where the next person hardening scope would look, and it would have argued them straight back into Orchestrator cannot read the artifacts its own constitution instructs it to synthesize #338. It now states what §4 actually means: context economics, not permission.
  • The constitution gave the orchestrator no opening move. Every default worker read set is rooted at spec, every child brief ends "wait for instructions", and nothing told the orchestrator to publish one — so a goal whose first dispatch is search stalls on turn one with a searcher reading "no spec has been written on this goal yet" and no scope to read anything else. Derived like every other claim in that section: an orchestrator that can't write spec isn't told to.

Also fixed a guard I wrote that asserted nothing: toContain("You can READ: ") is satisfied by a hardcoded "You can READ: spec, findings" — the original bug verbatim. It compares against resolveRoleIo on both sides now, and checks the two sets differ so it can't pass on a formatter that ignores its argument.

typecheck · lint · bun test → 2592 pass, 19 skip, 0 fail.

Left alone deliberately

Pre-existing, each changing existing read semantics and belonging in its own PR: #ownSlot slots only findings, so a fan-out on diff/research silently collapses to the last writer; task-list has two default writers and no slot; roleSchema isn't .strict(), so a typo'd key is stripped silently; the web create modal still has no per-role scope field; PhaseDef.reads/writes has a divergent wire shape (scalar writes) that P4 will need to reconcile.

@saucam
saucam merged commit cabdd31 into main Sep 24, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Orchestrator cannot read the artifacts its own constitution instructs it to synthesize

2 participants