Skip to content

FR: events emit projections — resolve the ADR-0007 vs ADR-0028 contradiction (a stream is a channel, not a source) #212

Closed

Description

@dmealing

Summary

There is a live contradiction between two accepted decisions, and it should be resolved before a streaming/CDC backend lands:

  • ADR-0007's source catalog includes source.event @topic @role:publish (CDC / outbox) — a stream as a source.
  • ADR-0028 rules that "message topics/queues are channels, not sources — they live at the surface layer as binding.* on operations (AsyncAPI's model), never in source.*."

Both are currently on the books.

Resolution — toward ADR-0028: a stream is a channel, not a source

A source.* answers "where can this object's data be READ from" — it must be addressable state the tool can get-by-id against and that a verify --db-style drift check can inspect as a schema. A topic is transport: you cannot get-by-id against it, and no drift check can inspect "its contents." So:

  1. Amend ADR-0007's catalog: drop event as a source paradigm (and remove the publish role's source-side home).
  2. Model emission at the surface layer FR-024 §9 already reserved: an event/streaming api sibling (api.eventing / operation.event / binding.messaging) whose payload references a projection.

This lands a clean CQRS + events symmetry: queries return projections, commands take values, events emit projections. A CDC event's shape is a projection (passthrough/computed origins evaluated at emit time); only its exposure is a channel binding instead of a source — the projection concept still does all the shape work.

Two edge cases, pre-decided

  1. Windowed / stateful stream aggregation (Kafka-Streams-style folds over history) violates the meta migrate: aggregate/monitoring read-model views can't be modeled as projections — 4 missing origin/field capabilities #195 determinism contract (a pure derivation over the current entity graph). It is stream processing, not projection — stays out permanently.
  2. A compacted changelog treated as a state store (KTable) is functionally addressable. If ever needed, it enters as an ordinary read-only-@kind paradigm source — because the doctrine has a principled escape: a stream becomes a source exactly when it is treated as addressable state. Record this clause in the ADR so the door is principled, not ad hoc.

Scope

Amend ADR-0007 (remove the event paradigm + the publish source role); design the reserved api eventing sibling (operation.event + binding.messaging + a payload→projection reference); record the "stream becomes a source when addressable" escape clause. No change to the projection/entity/value taxonomy — this is purely about where event exposure lives (surface layer, not source layer).

Activity

  1. dmealing commented on Aug 5, 2026

    @dmealing
    MemberAuthor

    Ruling (2026-08-05): RATIFY, with three amendments — plus one sub-decision this surfaced

    The resolution above is ratified: a stream is a channel, not a source. Drop event from
    ADR-0007's catalog, model emission at the surface (api.eventing / operation.event /
    binding.messaging) with a payload→projection reference, and keep the escape clause. Three
    amendments follow, then a sub-decision that needs its own ruling.


    Amendment 1 — the doctrine's test is wrong in both directions; drop half of it

    As written the test is a two-prong AND: a source.* must be addressable by id and
    drift-inspectable as a schema. Swept across all twelve catalog entries, prong (b) breaks it:

    It keeps a schema-registry-backed Kafka topic (Confluent exposes key/value schemas — more
    inspectable than most of the catalog) and kills source.memory (nothing exists to drift
    against).

    Both are the wrong answer, from the same prong. It also wrongly maims document
    ($jsonSchema optional), keyValue (Redis has none), objectStore (parquet yes, CSV no),
    timeSeries (Timescale yes, Prometheus no) and graph (constraints optional).

    Prong (a) alone produces exactly the intended result. Ratified wording for the ADR:

    A source.* binds an object to addressable state at rest — a place where a current value of
    an instance can be read on demand by a declared key or address, not merely observed as it flows
    past. Drift-inspectability follows per paradigm from whatever schema authority the backend
    exposes (information_schema, index mapping, schema registry, parquet footer) and may be
    absent; it is a verify capability, never an admission criterion.

    Under this, only event's flow kinds (topic / stream) fall — eventStore and changelog are
    precisely the escape clause. Every other catalog entry survives, and "get-by-id" generalizes to
    "get-by-declared-key/address" (keyValue composite keys, objectStore paths, timeSeries series+time).

    Two intuitions worth correcting for the record: source.vector survives (Qdrant/Pinecone have
    fetch-by-id; similarity is the query path, not the only read), and so does source.search
    (GET /_doc/id plus GET /_mapping).

    Amendment 2 — removing publish is NOT docs-only

    @role: publish is live registered vocabulary: fixtures/registry-conformance/expected-registry.json
    (line 3746), spec/metamodel/db.json:44, and five ports' embedded definitions (C# and Python carry
    committed SpecMetamodel copies to sync). Zero fixtures use it — only primary and replica
    appear anywhere — so the churn is tiny, but it is lockstep, technically breaking, and belongs in the
    one declared pre-1.0 window alongside #210's registry-string change. If that batch never happens
    before 1.0, publish stays registered-but-inert and dies at 2.0. Harmless either way.

    Amendment 3 — one-word consistency fix in FR-024 §7

    Its channel row says an operation referencing a value. Under the ratified symmetry it
    references a projection.


    Sub-decision (NEW — needs a ruling): what is @role actually for?

    Investigating Amendment 2 turned up something that changes how publish should be read.

    @role is load-bearing for exactly one invariant. validate-source-roles.ts enforces: at
    least one source must be primary; no more than one may be; and a primary whose @kind is
    read-only is ERR_ENTITY_PRIMARY_SOURCE_READONLY (ADR-0028 B4b). That is the whole of it.

    Nothing downstream reads the value. The write-through CQRS feature — the canonical multi-source
    case, and the one ADR-0007's own rules text cites — keys on @kind, not @role:

    export function isProjection(entity)   { return hasReadOnlyKindSource(entity) && !hasWritableKindSource(entity); }
    export function isWriteThrough(entity) { return hasReadOnlyKindSource(entity) &&  hasWritableKindSource(entity); }

    So the loader only ever asks "primary or not", and codegen does not ask at all. Across every
    fixture in the repo: primary × 9, replica × 11. index, cache, publish and mirror have
    zero uses and zero readers.

    That reframes Amendment 2: removing publish is not removing a capability, it is deleting one of
    four never-read enum members. Which raises the question the doctrine does not answer — if
    publish goes on doctrine, what justifies index, cache and mirror staying?
    They survive
    the addressability test fine (an ES index and a Redis cache are both addressable state), but they
    are unused, unread, and indistinguishable from replica to every consumer.

    Options:

    • A (recommended) — shrink @role to primary | replica; document index / cache /
      publish / mirror as reserved, not registered. This is exactly the ADR-0040 precedent
      (index.fulltext / index.vector / index.spatial are documented but unregistered on
      YAGNI-plus-vocab-freeze grounds). One registry change instead of two, same batch, no extra cost.
      Keeping unread members registered obliges five ports to carry them in lockstep forever for zero
      benefit, and invites an adopter to author @role: cache and get no behaviour — worse than an
      honest "not supported yet".
    • B — remove only publish per the doctrine; leave the other three. Minimal, but leaves the
      inconsistency: three dead members kept, one deleted, on grounds that do not distinguish them.
    • C — leave all six; document that only primary is read. Cheapest now; hardest to undo after
      the 1.0 freeze.

    Argument for C/B worth weighing: @role may be legitimately descriptive — an author
    documenting that a source IS a cache has value even when no generator reads it. That is a real
    point, but if descriptiveness is the justification then the right home is the registered
    attr.properties bag (ADR-0023's chartered escape hatch), not four dead enum members in a
    byte-checked manifest.

    This sub-decision rides the same batch as Amendment 2 either way, so deciding it costs nothing
    extra — but it should be decided explicitly, not carried along implicitly with publish.


    Scope and sequencing

    Now — docs only, no quiet-clock reset Amend ADR-0007 (drop event from the catalog + the field-attr table; record the refined test and the escape clause), sync the source-v2 design doc, cross-ref ADR-0028, fix FR-024 §7's channel row. ADR-0035 §1 excludes "reserved and not yet in the registry" from the compat surface.
    Rides the #210 consolidation batch The @role registry change — publish removal at minimum, or the full shrink under option A.
    1.1, rides FR-024 (already RATIFIED-DEFER) The api.eventing / operation.event / binding.messaging build. None of api.*, operation.*, binding.* is registered today (confirmed empty), so it is additive by construction and never window-sensitive.

    What would reverse this ruling

    • An adopter whose primary persistence is an event store (get-by-aggregate-id + fold). Replay-derived
      state strains "state at rest"; the escape clause still admits the store, but the doctrine's centre
      of gravity would be misplaced.
    • The escape clause being exercised early and repeatedly (multiple changelog-as-store adopters
      before any eventing surface ships) — evidence that default-out chose the wrong default.
    • The FR-024 eventing build finding event payloads routinely need shapes underivable from the
      current entity graph
      .

    Not a falsifier: CDC before/after envelopes. Both images are instances of one projection shape
    evaluated at two moments; the envelope is surface-owned structure. The real break would be a payload
    needing genuinely historical derivation — at which point "events emit projections" weakens to
    "events emit declared shapes".

  2. added a commit that references this issue on Aug 5, 2026
  3. dmealing commented on Aug 5, 2026

    @dmealing
    MemberAuthor

    Sub-decision RULED (2026-08-05): option A, plus doctrine

    Shrink source.rdb @role to allowedValues: ["primary", "replica"] in the #210 consolidation
    batch. Document index / cache / publish / mirror as reserved-not-registered (the ADR-0040
    treatment), and write the re-entry bar into the ADR-0007 amendment: a role member enters the
    registry only when a shipping consumer dispatches on it.

    The cross-port gate is closed

    The "nothing reads the value" finding above was TypeScript-only. It generalizes exactly — every
    read of @role in all five ports is an equality test against primary:

    • Java — MetaObject.java:950,968 (findPrimaryWritableSource / findPrimaryReadOnlySource,
      ROLE_PRIMARY.equals(...) ANDed with kind-derived writability) and ValidationPhase.java (the
      one-primary invariant). OMDB has zero role usage — no replica read-routing exists.
    • Kotlin — KotlinGenUtil.kt:103 is explicitly "Own-only + role-agnostic"; its own comment
      states the replica view is found by read-only @kind, not by role.
    • C# — MetaSource.cs:62 defines Role; the only production consumers are
      ValidationPasses.cs:1962,1989. MetaObjects.Codegen / .Render never read it.
    • Python — object_manager.py:625 selects the primary by role; the write-through read path
      _read_source_name is documented "OWN + role-agnostic" and finds the replica by kind.

    Across five ports, @role's entire consumed information content is one bit — "is this the
    primary." replica, index, cache, publish, mirror are indistinguishable to every consumer.
    Fixture sweep: primary ×12, replica ×12, the other four ×0.

    Why not a @primary: boolean (the sharper option)

    Because replica is structurally load-bearing, despite no consumer reading the word.
    @role defaults to primary when omitted (all five ports), and the one-primary invariant rejects
    two primaries — so any second source must carry an explicit non-primary role or the model
    fails to load. replica is the registered opt-out that makes every multi-source object declarable
    at all, and it is the taught authoring form for write-through (@kind: view @role: replica).
    The enum cannot shrink below two members without redesigning the default.

    @role also survives ADR-0037's ordered test, narrowly, as a step-3 configuration attribute.
    Primacy is not step-0 derivable in general: the obvious derivation ("the writable source is
    primary") reproduces every current use in this repo, but primacy is a tie-breaking designation,
    and table(primary) + table(replica) — two same-writability sources — is legal today and becomes
    inexpressible under derivation. ADR-0023's derive-don't-declare covers facts that are always
    computable; primacy is only contingently so.

    A boolean would additionally bake the one-bit reading into the authored surface, force a rename
    migration on 12 fixture uses plus every adopter write-through model now, and make the routing
    direction a re-modelling later — for zero behaviour change. The two-member enum is the honest
    freeze; the boolean is a false economy.

    What decided it

    1. Zero readers, zero writers, five-port lockstep carry cost. Four of six members have no
      fixture use and no consumer in any port. Keeping them obliges five ports, two committed
      SpecMetamodel copies and a byte-checked manifest to carry them forever — and lets an author
      write @role: cache and silently get replica behaviour, which is worse than an honest
      "reserved, not supported".
    2. The freeze is asymmetric (ADR-0036: the pre-1.0 consolidation window is the only time).
      Removing a registered member post-1.0 is a 2.0 event; re-adding a reserved one post-1.0 is
      additive. Pruning now is the reversible move; keeping is the irreversible one.
    3. The precedent is exact, and the template matters. ADR-0040 gave index.fulltext /
      index.vector / index.spatial this treatment on identical grounds. And because @role is
      registered on source.rdb specifically, the shrunk registration is what every future paradigm
      in this issue's catalog will copy — better it carry two proven members than six speculative ones.

    Recorded finding, not acted on

    @role today is a designation mechanism, not a routing mechanism. ADR-0007's rules text says
    consumers "route by @role — primary = CRUD; index/cache/publish = derived behaviour", but no port
    ever built that dispatch while the vocabulary sat registered. The amendment should state the
    current reality plainly and set the consumer-reads-it bar, rather than leave prose implying
    behaviour that does not exist.

    Strongest argument against

    ADR-0007 designed @role for future runtime routing; pruning forecloses a designed direction. And
    even unread, @role: cache is honest author documentation.
    Real, and the direction was designed
    rather than accidental. It loses because reserved-not-registered defers rather than forecloses —
    re-entry is additive, and registration demonstrably did not summon the feature in four ports over
    the project's life. And if descriptiveness is the justification, ADR-0023 already charters the home
    for consumer-meaningless author annotations: the attr.properties bag, not four members of a
    byte-checked enum whose presence implies behaviour that does not exist.

    Before the batch lands — required check

    Zero uses were verified in this repo only. These are shipped registered members, so removal is
    breaking for anyone already authoring them: re-run ADR-0036's adopter scan plus the public
    reference app before landing.

    Execution note for the batch

    The shrink touches spec/metamodel/db.json — both allowedValues and the description string,
    which enumerates all six and is byte-checked — plus fixtures/registry-conformance/expected-registry.json,
    TS constants, Java MetaSource.VALID_ROLES / RdbSource.withEnum, C# SourceConstants.SOURCE_ROLES,
    Python source_constants.py, and the C# / Python committed SpecMetamodel copies. The
    ROLE_INDEX / CACHE / PUBLISH / MIRROR constants are deleted, not orphaned.

    What would reverse this

  4. added 3 commits that reference this issue on Aug 5, 2026
  5. dmealing commented on Aug 8, 2026

    @dmealing
    MemberAuthor

    Shipped in 0.21.0 / 7.21.0 — a coordinated MINOR across all four registries (npm 0.21.0 · PyPI 0.21.0 · NuGet 0.21.0 · Maven 7.21.0).

    Merged as #276; commits 93955c35 (the shrink) and cb991a25 (review follow-ups). local-ci green 9/9.

    Migration guide: docs/features/migrations/value-assembly-origins-and-source-role-shrink.md, §3 covers @role.

    The doctrine half — already landed before this cut

    All three amendments from the ruling shipped ahead of the registry change, per the split the ruling itself specified (ADR-0035 §1 excludes reserved-but-unregistered from the compat surface, so no quiet-clock reset):

    • ADR-0007 Amendment 1 — the single-prong admission test: a source.* binds an object to addressable state at rest, readable on demand by a declared key or address, not merely observed in flight. Drift-inspectability is a per-paradigm verify capability, never an admission criterion. Only event's flow kinds fall; eventStore/changelog are the escape clause; vector and search survive.
    • ADR-0007 Amendment 2 — @role is a designation, not a routing mechanism, with the re-entry bar: a role member enters the registry only when a shipping consumer dispatches on it.
    • ADR-0028 amendment and FR-024 §7 — the channel row references a projection, landing the CQRS symmetry: queries return projections, commands take values, events emit projections.

    The registry half — this release

    source.rdb @role's allowedValues shrinks to ["primary", "replica"]. index, cache, publish and mirror become reserved-not-registered (the ADR-0040 treatment: documented on the axis, absent from the registry). A legacy use now fails load with ERR_BAD_ATTR_VALUE — no new error code, no new validation pass; the existing generic allowedValues check does the work.

    Per-port constants for the four retired members are deleted, not orphaned, across all five ports. Cross-port verification confirmed the ruling's premise held: the only remaining non-test readers are the intended registration/membership sites, so no consumer dispatch exists and the re-entry bar is untripped.

    The adopter scan was re-run immediately before landing — zero uses of the four retired members across this repo, the public reference app, and downstream consumer models.

    One real bug found in review

    The Java loader's @role rejection message was a hand-written literal listing all six members, so after the shrink it emitted:

    @role 'publish' is not a valid value; allowed: primary, replica, index, cache, publish, mirror

    — naming the rejected value among the allowed ones, on the exact path the new conformance fixture exercises. It was also a cross-port diagnostic divergence, since TypeScript derives its equivalent from the registry spec. It now derives from MetaSource.VALID_ROLES via a sorted TreeSet, so it cannot drift again.

    The generalizable lesson, recorded because it recurred: after retiring a vocabulary member, sweep for the member values as free text, not just the constants that held them. The original sweep matched constant names only.

    Gates

    New conformance fixture error-source-role-reserved (@role: "publish" → ERR_BAD_ATTR_VALUE), TDD RED-first against the six-member registry. Registry manifest diff confined to the @role attribute block; metamodel-docs diff to the @role row; all codegen golden output byte-identical — roles never dispatched, so the golden suite is the proof.

    Still open, by design

    The eventing surface (api.eventing / operation.event / binding.messaging) rides FR-024 to 1.1. All three node types are unregistered today, so it is additive by construction and nothing about this cut blocks it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:metamodelMetamodel vocabulary / loaderenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions