Conversation
…pecs DESIGN_PHASE3.md T1 and T13, the two tasks with no dependencies. Nothing here changes behaviour: `single` tenancy is the default everywhere and the `default` AgentSpec reproduces Phase 2's argv byte for byte. shared/tenancy.py (T1) `userkey(issuer, userid)` is the only place an identity becomes a name. The slug is deliberately lossy so keys stay readable in `kubectl get kafkatopics`; that is safe only because 32 bits of SHA-256 over the canonical form are appended, so two identities that slug identically still differ. The issuer is inside the hashed bytes as well as the prefix — without it, configuring a second issuer is a way to impersonate an identity on the first. The digest input is joined with \x1f so the issuer/userid boundary is unambiguous. Canonicalisation is per-issuer and minimal: GitHub folds case (and must agree with `ghauth.is_allowed`, which a test pins), email lower-cases only the domain per RFC 5321, static is byte-exact. Erring toward two tenants for one human is wasteful; the other direction is a leak. `TopicSet` is the one place §3.1's layout is written down, and refuses to name a topic without a userkey in multi mode rather than falling back to a shared one. `ntfy_topic` is HMAC-derived so the name carries no identifier (§4.1). eventrunner/agentspec.py (T13) One TOML file per agent, carrying the tool policy that §7.4 depends on — which is why §9 puts this before triggers. Validation refuses rather than coerces: a policy that silently drops the clause it could not parse would hand a trigger-driven agent the tool the operator meant to remove. A missing non-default agent is an error event, never a silent fallback to `default`. `build_cmd` takes a spec. The request still wins on `max_turns`/`model` (Phase 0 wire parameters), but the tool policy, permission mode and system prompt are spec-only: a request that could widen the sandbox is a request that can escape it. Tests: 105 new, 656 total passing, no existing test modified. The argv-equality test compares against a literal transcription of Phase 2's `build_cmd` rather than a call into the current one, so it catches a regression instead of following one. The `claude --help` flag assertion covers only the flags Phase 3 adds. `--max-turns` is supported but undocumented in `--help` on 2.1.270, so asserting the pre-existing flags would fail against a working deployment. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
…igned tenancy DESIGN_PHASE3.md T2, T3, T4 and T22. Still no behaviour change in the default configuration: `EB_TENANCY_MODE=single` is the default, and the 656 tests that existed before this branch all pass untouched. That is the property §9 asks steps 1-4 to preserve — refactors with tests, everything risky behind a flag that is off. T2 — TopicSet threaded through (§3.2) One set, built once in `__main__` and handed to the producer, the responses consumer, both mirrors and the config, so §3.1's layout appears exactly once. `publish_request` takes a `userkey` and resolves the topic from it; one producer can send to any topic, so there is no second producer and no connection per user. The explicit-topic-list option, not `subscribe(pattern=...)`. A pattern is picked up by metadata refresh (5 min by default), so a new user's first response can be published before anyone is subscribed and with `auto_offset_reset=latest` it is never read — the page stays empty and no error appears anywhere. The blocking `ensure_subscribed` that closes the remaining race is T6. GroupMirror takes the full topic list directly: it is one-shot at startup, so there is no new-user race to lose, and a tenant whose topic is not provisioned yet is skipped with a log line rather than costing everyone their group history. T3 — the registry and Caller (§2.4, §2.5) `Caller` carries the tenancy key so no caller derives it twice. `submitter_iss` is deliberately not the same as `issuer`: a static identity needs an issuer for its key to separate from a GitHub login of the same name, but Phase 2 made an absent `ce_submitteriss` mean "an operator typed this", and promoting it would upgrade an assertion into a verified claim. The registry refuses rather than skips — the opposite of `auth.parse_tokens`, because a skipped user is one whose topics exist and whose events have nowhere to go. A recorded userkey that disagrees with this build's derivation is a startup refusal naming both values, and two identities colliding on one key is refused, which is what covers §2.3's 32-bit birthday bound. `multi` mode refuses anonymous with 401 and an unregistered user with 403. Both disagree with the single-tenant defaults on purpose; `single` is byte-identical to Phase 2. T4 — SIGNED_ATTRS += userkey, depth (§8.3) One change, both attributes, because adding one changes canonicalisation and a signer and verifier on different versions disagree about every signature. Both have to be covered rather than merely present: a mutable `userkey` lets anything with topic write access file events into another user's history, and a resettable `depth` is not a hop limit. T22 — the Secret-vs-ConfigMap rule, pinned in test_manifests.py The secret-path variables are declared ahead of the tasks that read them (T8-T16) so the guard is in place when they land. The classification test only asserts the ConfigMap side against live code, and says so, rather than looking stronger than it is. Known gap, stated rather than hidden: `/continue` resolves the owning tenant from the recorded submitter so a resume reaches the right runner, but does not yet check that the caller IS the owner — owner-scoped reads and `?k=` keys are §6.2/§4.4 (T7/T9, step 5). Until those land, `multi` mode's `/continue` is as open as Phase 2's. The lookup also cannot distinguish a GitHub `alice` from a static `alice`; `multi` mode refuses both anonymous and static submissions, so GitHub is the only issuer that reaches it today, and this moves to T5's global index when that exists. Tests: 66 new (722 total), ruff clean. Verified both services start in every tenancy configuration and that all four refusals print readable messages rather than tracebacks. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
…till design The Phase 3 design doc said "nothing in this document is implemented yet" and the agentdocs index said "Design only — not implemented". Both are now wrong in a way that matters: a reader who trusts them will either re-implement steps 1-4 or assume `EB_TENANCY_MODE=multi` gives them isolation it does not yet give. So the status lines name the tasks that landed (T1, T2, T3, T4, T13, T22 = §9 steps 1-4) and say plainly that everything from step 5 on — per-user stores, owner-scoped reads, transcript auth, `k8s_tenant.py`, ntfy isolation, triggers, fetched skills, Kafka ACLs — is still design only. The component README gains a Phase 3 configuration table covering only the six variables whose code exists, with two notes that are the point of including it at all: `multi` mode is not yet isolation anyone should present as such (the HTTP reads and `/continue` are still as open as Phase 2's, because steps 5-8 are not in), and adding a user is an operator action because EventBridge has no Kubernetes client. Verified against the code rather than assumed: only `POST /v0/agents` and `POST /v0/groups` call the auth path, so every GET, `/continue` and `/transcript` is open in both modes today. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
…rashing
Found reviewing the previous commit. In `multi` mode `_owner_userkey` returned
`None` both when no key was needed (single mode) and when the owning tenant could
not be determined. The second `None` flowed into `publish_request` ->
`TopicSet.requests(None)`, which raises `ValueError` by design — publishing a resume
to a shared topic would run it on another tenant's runner, under that tenant's
credential — but nothing caught it, so the request died as a WSGI 500 with a stack
trace instead of a status code.
Three ways to reach it, none of them abuse:
* a user removed from the registry;
* a `prompts` row whose `submitter` is NULL — the column is nullable and pre-dates
auth, so these rows exist in any store that outlived Phase 2;
* a correlation back-filled from the topic by `RequestsMirror`, which records no
submitter at all.
`_owner_userkey` now returns `(userkey, unresolved_reason)` with exactly one set, so
the two `None` cases are distinguishable, and both `/continue` paths check it before
writing anything — no touched session row and no orphan prompt row claiming a turn
that never ran. The JSON route answers `503` naming what could not be resolved,
matching how §2.5 treats a user whose topics do not exist. The HTML form route
answers `503` with a plain-text body rather than redirecting: a silent `303` back to
the transcript page would look like the turn was accepted and then vanished, which is
the Phase 2 §6.1 symptom class this phase keeps trying to avoid.
Only reachable with `EB_TENANCY_MODE=multi`, so the default path was never affected.
13 new tests (735 total). Verified they are load-bearing by reintroducing the bug:
the four refusal tests fail without the guard and pass with it.
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
… index
DESIGN_PHASE3.md T5 (§6.1, §2.6), the keystone the rest of step 5+ depends on: T7
(owner-scoped reads), T8 (transcript auth), T15 (triggers) and T21 (deletion) are all
blocked on it. `EB_TENANCY_MODE=single` remains the default and reproduces Phase 2.
eventbridge/owner_index.py — the one deliberately cross-tenant table
`correlationid -> userkey`, UNIQUE on `correlationid` **alone**. That is the part to
get right: `(userkey, correlationid)` is the correct primary key inside a tenant's
store and exactly the wrong constraint here, because two tenants minting one id
produce two distinct tuples, never conflict, and the collision passes through
silently — which is the failure the guarantee exists to prevent.
It has to be cross-tenant: the lookup that decides which tenant owns a correlation
must happen before a per-user store can be chosen. It is also what lets
`sessionuuid = uuid5(NAMESPACE, correlationid)` stay UNSALTED (§2.6) — salting it
would break every existing session for a collision this constraint removes.
`owner_of` returns `(userkey, known)` because a bare `None` conflates "the shared
tier owns this" with "never seen", and §6.2's 404 rule needs them distinguished.
eventbridge/store_registry.py — per-user stores, LRU-bounded
Separate FILES, not one database with a `userkey` column. `store.py` has 28 methods
and every one touches tenant data; a `WHERE userkey = ?` remembered 28 times is one
that will eventually be forgotten, with a user reading someone else's conversation
as the symptom. Separate files make the mistake structurally unavailable — the
connection IS the tenant.
Two connections per tenant in WAL mode (main + -wal + -shm) bounds this near 150-200
tenants under a 1024 soft RLIMIT_NOFILE, hence `EB_MAX_OPEN_STORES=64`. A store with
live SSE subscribers is PINNED: evicting one makes an open page stop updating with no
error anywhere. If everything is pinned the cache exceeds its ceiling rather than
breaking a viewer — over-budget degrades and is visible on /healthz; a dead stream
does not recover.
Minter now consults the index (§2.6)
One indexed SELECT per mint replaces the startup seeding loop. With N per-user stores
the old loop meant N SQLite opens before the socket binds — 100 tenants x 10,000
correlations for a guarantee one SELECT gives directly. The claim happens inside the
retry loop, so a candidate lost to a concurrent worker retries instead of both callers
believing they own it.
`mint_for()` passes `userkey` only to minters that accept it, by signature inspection
rather than by catching TypeError around the call — a working `mint()` can raise
TypeError from its own body, and swallowing that would silently drop the ownership
claim.
Routing
Responses are filed by the event's OWN signed `ce_userkey`. One with no userkey in
multi mode lands in `shared/` and increments `unattributed` (now on /healthz): never
guessed into a tenant's store, never dropped. `/continue`, the reads, the group routes
and the deadline sweeper all route through the index — the sweeper across every tenant,
because a stalled batch belonging to an untouched tenant is exactly what deadlines are
for and also the least likely to be in the LRU.
This retires the submitter-derived owner lookup from the previous commit, and with it
the limitation that it could not tell a GitHub `alice` from a static `alice`.
Single-tenant mode keeps its files exactly where Phase 2 left them — directly in the
bridge root, not under `shared/` — so an upgraded deployment's sessions and transcripts
keep showing up in the UI instead of appearing to vanish.
Two bugs found and fixed while building this:
* **Eviction could return a closed store.** With every older entry pinned, the LRU
took the store it had just opened and handed the caller a closed one — surfacing
later as `ProgrammingError: Cannot operate on a closed database` from somewhere that
looks unrelated to caching. Fixed with `protect=`, with a regression test.
* **`except Exception` hid a signature mismatch.** Adding `userkey=` to
`publish_group_event` made every group event stop publishing, and the broad handler
around it turned that into a batch that silently never completed. TypeError now
re-raises at both call sites.
`Store.close()` added (it had none), and it wakes subscribers first so a viewer blocked
on `Event.wait()` notices shutdown instead of hanging to its own timeout.
Tests: 49 new (785 total), ruff clean. Verified both modes start and that the on-disk
layout is upgrade-safe.
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
… them
Found reviewing T5. `StoreRegistry` eviction called `Store.close()`, which created a
use-after-close race on every multi-call read path: a handler resolves a store, and
while it is still making calls a *concurrent request for a different tenant* pushes
the cache over `EB_MAX_OPEN_STORES` and closes it underneath. The next call fails with
sqlite3.ProgrammingError: Cannot operate on a closed database
from a line that looks nothing to do with caching. `get_html` alone makes five calls on
one store, so the window is wide rather than theoretical, and reproducing it needs only
`max_open=1` and two tenants.
Eviction now drops the registry's reference and lets CPython close the connections when
the last holder goes away — exactly the lifetime that is safe. File descriptors are
reclaimed slightly later than an explicit close; that is the right trade, because the
ceiling is a soft budget while a closed connection under an in-flight request is a 500.
`_sse_generator` additionally now subscribes BEFORE the replay read. Pinning is what
keeps a subscribed store in the cache so later readers get the same object and see the
subscriber's notifications, and the old order left the store unpinned between
`_store_of()` and `subscribe()`. Reordering is harmless: the Event only ever means
"there may be something new", every read is `since_seq=sent`, and an already-set Event
costs one extra query.
Worth noting what the first fix attempt got wrong, since it is the more interesting
mistake: pinning only the SSE path would have closed the case that was easiest to see
while leaving every ordinary read exposed. The bug was in the eviction contract, not in
one caller.
Two regression tests, one asserting an evicted-but-held store stays usable for both
reads and subscribe, one pinning the subscribe-before-replay order. 787 passing.
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
… failures Three bugs from an adversarial review of T5. All three were confirmed by driving the real code, and all three are multi-tenant-only — the default `single` path is unaffected. **1. Restart-orphaned groups were never settled (`_settle`).** It called `maybe_complete(gid)` with no `userkey`, so it looked in the SHARED store for a group that lives in a tenant's store, found nothing, and returned False. Every group a restart left unfinished stayed unfinished forever, and the failure mode is silence — which §21.6 calls the worst one. The mirror now resolves the owner per group from the ownership index. **2. Replayed member events landed in the shared store.** `on_member_event` reads `ce_userkey` off the event, and any response published *before* Phase 3 has none — so member rows went to `shared/` while the group row sat in the tenant's store, leaving the group's counts permanently wrong and `_decorate_member` reading from a third place. Pre-existing Kafka history is exactly what this mirror exists to replay, so that is the normal upgrade path, not an edge case. The mirror now back-fills a missing `userkey` from the index. It only ever FILLS: an event carrying its own `userkey` keeps it, because that attribute is signed (§2.6) and the index is not authoritative over a signed assertion. Both of these had one root cause — the mirror knew about per-user topics but not about which tenant a replayed group belonged to. **3. A broken index surfaced as "correlation ID space exhausted".** `mint()` caught `Exception` around `claim` to treat a lost race as a retry, so a genuinely broken index (disk full, database locked, schema missing) was retried 2000 times and then reported a message that sends the reader to the word lists rather than the database. Now catches `Collision` specifically; everything else belongs to the caller. Also closes a test-strength gap the review found: deleting the `_has_subscribers` pinning check entirely still passed every `test_store_registry.py` test. The existing tests asserted object identity and counters, not the notification path that is the actual reason for the rule — now that eviction only drops a reference, a held store keeps working, so what actually breaks is that the *writer* gets a new Store whose subscriber map is empty and the viewer is never notified. The new test asserts that end to end; removing the pinning check now fails 4 tests instead of 0. Tests: 8 new (795 total), ruff clean. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
…ve you The status lines said steps 5-12 were design only; T5 (per-user stores and the global ownership index) is now in the tree, so they were wrong in the direction that matters — a reader would either re-implement it or, worse, read "per-user stores" as "isolation". The distinction the docs now draw explicitly: each tenant's sessions, responses and transcripts DO live in their own SQLite files, and a read routes to the owning tenant's store — but nothing yet checks that the *caller* is that owner, because owner-scoped reads (T7) and transcript auth (T8) are not implemented. So `multi` mode's reads, `/continue` and `PUT /transcript` remain as open as Phase 2's. Also adds `EB_MAX_OPEN_STORES` to the component README's Phase 3 table with the descriptor arithmetic that explains the default, and to `test_manifests.py`'s classification list so T22's "every Phase 3 variable is on one side of the Secret-vs-ConfigMap rule" check keeps covering it. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
aslom
left a comment
There was a problem hiding this comment.
Summary
Careful, well-reasoned work — 37 files, +4618/−153, implementing §9 steps 1–4 plus T5,
T13 and T22, with four fix(...) commits for defects found during the work and docstrings
that argue for their decisions rather than restating them. The two modules the design is
most specific about are faithful to it: userkey() puts the issuer inside the hashed bytes
with a \x1f separator for the right reason, and owner_index.py declares
correlationid TEXT PRIMARY KEY with a comment on why it must not be
(userkey, correlationid) — the #881 correction landing exactly as specified.
Requesting changes for five things, two of which I would not want merged in any form
and one of which affects the default configuration.
The single-mode regression is the serious one. __main__.py deletes the
Minter seeding loop and nothing backfills its replacement — OwnerIndex has no
seed-from-store path. On the first start after an upgrade the index is empty while
sessions.sqlite is full, so exists() reports ids as free that are in use. The space is
50 × 50 × 10,000 = 25M, so with ~1,000 existing correlations a mint has ~1-in-25,000 odds
of reissuing a live id, and the result is upsert_session overwriting a session and the
new prompt appending to somebody's old conversation. Phase 2's odds were zero. This
contradicts the headline guarantee, and no test covers the state that causes it because
the fixtures build index and store together.
Then two that make multi incorrect rather than merely incomplete. create_group
passes userkey to submit_members but not to groups.create(), so the group row lands
in the shared store while its members land in the tenant's — the batch reports 0 members
forever and never completes, and a retried POST returns members: []. And
close_group/cancel_group perform no authentication at all (_caller is called in only
three places in that file, neither of them these), then resolve any tenant's group from the
global index and mutate it. T7 covers reads staying open; it does not cover unauthenticated
cross-tenant mutation, which is new here.
Two more. agent is absent from SIGNED_ATTRS while ce_agent selects the AgentSpec
that supplies --permission-mode and the tool allowlists — so a forged attribute widens
the sandbox and the signature still verifies, which makes the signed configuration worse
than the unsigned one. And userkey reaches a filesystem path join unvalidated, from an
inbound Kafka header: .. escapes users/ into another tenant's store or out of the tree
entirely, verified by construction. tenancy.py is the only place that knows the key's
shape and exports no validator, which is the root of both.
Everything above was reproduced against 16158e50 rather than inferred. Six suggestions
and four nits follow, and two documentation comments.
On design conformance, which was asked for: the code is not fully reflected in
DESIGN_PHASE3.md. Six gaps, two of them contradictions — §6.1 still places
single-tenant stores in shared/ where the code uses the bridge root, and still asserts
"Closing is safe" about an LRU that commit 49aed05 proved unsafe. owner_index.py
appears in the document exactly once, in the status header this PR adds. Inline on that
header.
I have also set out what IMPLEMENTATION_REPORT3.md and README_PHASE3.md should contain,
and why the two Phase 2 documents are still owed — DESIGN_PHASE2.md §6 is a
test-results-and-findings report filed inside a design document, and its §7 "Running it?"
points at README_PHASE1.md for a phase that added the device flow, the approved-user
list, the break-glass path, a 300 s revocation window and the keyset rollout.
Author: mrsabath (MEMBER — maintainer, normal review posture)
Areas reviewed: Python (24 files), tests (12), Markdown (3). No YAML, Helm/K8s, Dockerfile, shell or CI in the diff.
Agent/IDE config (.claude/.vscode): none — §3.5a gate run for both +++ b/ and rename to forms, no match.
Secrets scan: clean. No CI workflow, dependency manifest, pyproject.toml, uv.lock or Dockerfile touched.
Commits: 8, every one Signed-off-by, DCO passing, all conventional prefixes.
CI status: all 11 checks passing — which is worth noting against the findings above: the suite cannot see the two multi defects, because every multi-mode group test calls GroupService.create directly with the userkey the handler fails to supply, and test_continue_tenancy.py mocks the producer so no topic string is ever asserted.
Checked and clear
- "No assertion was weakened" — true.
test_auth.pyandtest_groups.pyhave one identical hunk each, widening a fake producer'spublish_group_eventsignature and recording the new argument. Nothing removed or relaxed;test_manifests.pyis purely additive. - Hidden skips — none. The only skip is
test_agentspec.py's disclosedskipif(shutil.which("claude") is None). Worth knowing that this is the test pinning §5.5's CLI flags, so it does not run in CI. - Test strength generally —
test_tenancy.py's collision corpus,test_owner_index.py'stest_the_constraint_is_on_correlationid_aloneand its concurrency test,test_store_registry.py's pinning and eviction regressions, andtest_agentspec.py's literal Phase-2 argv transcription are all real and would fail on a regression. The two gaps are the ones noted inline. - §8.3's coordination warning is narrower than written, in your favour.
canonical()omits absent attributes and sorts by name, and nothing setsdepthyet, so for every event a default deployment signs today the canonical form is byte-identical across this change. The two services do not have to move together unlessmultiis on. - The
GET /v0/groupsquestion in the PR body is already answered by §6.2, which has that row — and also commits to filtering the list by owner, which this PR does not yet do. - The test arithmetic needs reconciling: the body says "656 baseline + 139 new", but this PR adds 192
def test_functions and removes none, and parametrisation only raises the collected count.
| return self._root | ||
| if not userkey: | ||
| return self._root / SHARED | ||
| return self._root / "users" / userkey |
There was a problem hiding this comment.
[must-fix] userkey reaches this path join with no validation, and in multi mode it
arrives from an inbound Kafka header (for_event → event.get(ce.EXT_USERKEY), line 141).
A userkey containing .. escapes users/:
'gh-alice-a1b2c3d4' -> /data/eventbridge/users/gh-alice-a1b2c3d4
'../users/gh-victim-0bad0bad' -> /data/eventbridge/users/gh-victim-0bad0bad # another tenant
'../../../../tmp/pwned' -> /tmp/pwned # outside the tree
'..' -> /data/eventbridge # the bridge root
Store.__init__ does base.mkdir(parents=True, exist_ok=True), so the directory is
created, not rejected. A forged ce_userkey therefore files events directly into a
victim's responses.sqlite, or drops SQLite files anywhere the process can write.
Two reasons this is reachable today rather than theoretical:
userkeyis inSIGNED_ATTRS, butEB_REQUIRE_RESPONSE_SIGNATUREdefaults to
false, and in audit mode the event is stored unchanged. Verification is what would
catch the forgery and it is off by default.- §3.6 already concedes that in Tier A anything with network access to the shared broker
can write to any topic. That is an accepted property for reading events; it must not
also be a filesystem write primitive.
Note the invalid key is truthy, so it bypasses the unattributed branch above and
goes straight to the join.
The fix belongs in tenancy.py, which is "the ONLY place an identity becomes a name" and
currently exports no validator — so no consumer can check the shape the producer
guarantees:
# shared/tenancy.py
USERKEY_RE = re.compile(r"^[a-z]{2}-[a-z0-9][a-z0-9-]{0,23}-[0-9a-f]{8}$")
def is_valid_userkey(value: str | None) -> bool:
return bool(value) and bool(USERKEY_RE.fullmatch(value))Then treat an invalid key exactly like a missing one — shared/ plus unattributed —
so a forged value is counted and visible rather than acted on. Worth a round-trip test
asserting is_valid_userkey(userkey(iss, uid)) over the same adversarial corpus
test_tenancy.py already builds.
For what it is worth, agentspec.py gets this right 150 lines away: _resolve_under
refuses a path that "escapes the agent directory" and reasons explicitly about
../../../etc/shadow, and validate_name enforces a DNS-1123 label. The same gate is
missing on the one input that is genuinely attacker-supplied.
| self._c.execute( | ||
| "INSERT INTO correlation_owner(correlationid,userkey,created_utc) " | ||
| "VALUES (?,?,strftime('%Y-%m-%dT%H:%M:%SZ','now'))", | ||
| (correlationid, userkey)) |
There was a problem hiding this comment.
[suggestion] The uniqueness guarantee is enforced by the PRIMARY KEY, but the
error type callers depend on comes only from the in-process lock. If two writers ever
race past it, the bare INSERT raises sqlite3.IntegrityError, not Collision — and
Minter.mint catches Collision specifically, with a comment explaining (correctly) why
it must not catch Exception. So that path would surface a 500 rather than retrying.
Today the lock holds: one process, RLock, single-replica. It is load-bearing in a way
the docstring attributes to the constraint, though, and a second writer is exactly what
§6.5's deletion path and any future sweeper add. Letting the database be the authority
removes the dependency:
cur = self._c.execute(
"INSERT INTO correlation_owner(correlationid,userkey,created_utc) "
"VALUES (?,?,strftime('%Y-%m-%dT%H:%M:%SZ','now')) "
"ON CONFLICT(correlationid) DO NOTHING", (correlationid, userkey))
if cur.rowcount == 0:
row = self._c.execute("SELECT userkey FROM correlation_owner WHERE correlationid=?",
(correlationid,)).fetchone()
if row[0] != userkey:
raise Collision(...)That keeps the idempotent-for-same-owner behaviour and makes Collision the only way a
conflict can present, lock or no lock.
| if userkey is None: | ||
| rows = self._c.execute( | ||
| "SELECT correlationid FROM correlation_owner WHERE userkey IS NULL " | ||
| "ORDER BY created_utc DESC LIMIT ?", (limit,)).fetchall() |
There was a problem hiding this comment.
[suggestion] created_utc is written at second granularity
(strftime('%Y-%m-%dT%H:%M:%SZ','now')), so ORDER BY created_utc DESC LIMIT ? is not a
deterministic "newest first" — a batch submitted inside one second has arbitrary order
among its rows, and with LIMIT smaller than that batch it returns an arbitrary subset.
GroupService.submit_members publishes a whole batch in a loop, so a 100-member group is
precisely the case that lands in one or two seconds.
Everything else on the event path uses millisecond precision (ce.now_iso() sets
timespec="milliseconds"), so this is also an inconsistency rather than only an
imprecision. Either match that, or add rowid as a tiebreaker (ORDER BY created_utc DESC, rowid DESC), which is free and total.
| not the first. | ||
| """ | ||
| with self._lock: | ||
| self._c.execute("DELETE FROM correlation_owner WHERE correlationid=?", |
There was a problem hiding this comment.
[nit] The docstring contradicts the code: "Note what this does NOT do: freeing the id
for reuse is deliberate" — but the DELETE does free the id, since exists() is the
Minter's only check and it will no longer see it.
Worth settling before T21 builds on this, because reuse has a consequence beyond tidiness:
sessionuuid = uuid5(NAMESPACE, correlationid) is unsalted by design (§2.6), so a reused
correlationid derives the same session uuid as the deleted one. A claude
transcript left on a runner's volume, or a checkpoint that outlived the delete, is then
resumable by the new correlation — a different user's conversation continuing into
somebody else's agent. Either keep a tombstone so ids are never reissued, or say
explicitly that deletion must also purge every transcript keyed on that uuid.
| if "@" in s: | ||
| local, _, domain = s.rpartition("@") | ||
| return f"{local}@{domain.lower()}" | ||
| return s |
There was a problem hiding this comment.
[nit] The docstring says static identities are "byte-exact: an operator typed them",
but the @ branch above this one catches them first — a static EB_AUTH_TOKENS name of
Ops@Example.COM canonicalises to Ops@example.com, not byte-exact. Harmless in
practice (and arguably the nicer behaviour), but the third bullet claims something the
code does not do, and this function's whole contract is that each branch is a deliberate
per-issuer claim. Either reorder on issuer == "static" or say the email rule applies to
any address-shaped identifier regardless of issuer.
| and returned False — leaving every restart-orphaned group unfinished forever, with | ||
| silence as the symptom (§21.6's worst failure mode).""" | ||
| svc, owners, stores, _ = _svc(tmp_path) | ||
| gid, _ = svc.create(label="b", expected=1, userkey=UK) |
There was a problem hiding this comment.
[suggestion] This is the suite's multi-mode blind spot, and it is why the
create_group bug above survived 192 new test functions.
Every multi-mode group test calls GroupService.create(label=..., userkey=UK) directly
(also 113, 130), supplying by hand the one parameter Handlers.create_group fails to pass.
So the suite proves the tenanted path works when userkey is given, which is not the
property that matters — nothing drives Handlers.create_group in multi at all.
test_continue_tenancy.py:51 has the matching gap from the other side: producer = MagicMock(), and every "publishes to the owning tenant's topic" assertion checks
publish_request.call_args[1]["userkey"] — never a topic name. The file's stated hazard is
that "publishing to a shared topic would run the turn on another tenant's runner", and no
assertion anywhere covers it; test_topicset_wiring.py checks TopicSet and Producer
separately but never the handler → producer → topic chain.
One test per route that goes through Handlers with a real Producer against a fake Kafka
and asserts the topic string would have caught both the create_group defect and the
continue_agent 500. That is a better investment than more unit tests on the pieces, which
are already well covered.
| # costs one GitHub call rather than one each. | ||
| self.logins = ghauth.LoginCache(cfg.github_cache_ttl_s) | ||
|
|
||
| def _agent_for(self, caller, body) -> str | None: |
There was a problem hiding this comment.
[nit] str(named).strip() accepts any JSON value — a dict, a list, "../../etc" —
and ships it as ce_agent. agentspec.NAME_RE and validate_name already exist and
enforce a DNS-1123 label; the HTTP boundary is where that check belongs.
As it stands a bad agent name is answered 202 Accepted and surfaces minutes later as an
asynchronous phase=error event, which the submitter may never look at, instead of a
400 on the call that made the mistake.
| f"got {cfg.tenancy_mode!r}. See DESIGN_PHASE3.md §8.1.") | ||
| cfg.topic_prefix = e("EB_TOPIC_PREFIX", cfg.topic_prefix) | ||
| cfg.user_registry_path = e("EB_USER_REGISTRY_PATH", cfg.user_registry_path) | ||
| cfg.max_open_stores = int(e("EB_MAX_OPEN_STORES", str(cfg.max_open_stores))) |
There was a problem hiding this comment.
[nit] Unguarded int(), so EB_MAX_OPEN_STORES=lots is a bare ValueError traceback
at startup while EB_TENANCY_MODE seven lines above raises a readable SystemExit. The
whole startup path is otherwise careful to print messages rather than tracebacks (the PR
body says so explicitly), so this is the odd one out.
Line 196 is the related gap: EB_TOPIC_PREFIX is unvalidated, so a prefix containing _,
. or a space produces exactly the illegal or JMX-colliding topic names that
tenancy.py's _slug docstring warns about — a topic a.b_c and a topic a_b.c collide
in Kafka's metric names and one silently overwrites the other's. The prefix is the one
component of those names _slug never sees.
| session = event["sessionuuid"] | ||
| mode = event.get("mode", "start") | ||
| prompt = payload.get("prompt", "") | ||
| max_turns = payload.get("max_turns") or spec.max_turns |
There was a problem hiding this comment.
[nit] payload.get("max_turns") or spec.max_turns replaced
payload.get("max_turns", 3), so a request with max_turns: 0 now silently gets the
spec's value instead of 0. The spec side rejects zero
(test_a_nonsensical_max_turns_is_refused), so the two paths disagree about the same
nonsensical input — one refuses it, the other rewrites it. is not None makes them agree,
or validate it at the HTTP boundary alongside the agent name above.
| """§4.1 is the whole design: the name travels to a phone.""" | ||
| t = T.ntfy_topic("kev1", T.userkey("github", "mrsabath"), SECRET) | ||
| assert "mrsabath" not in t | ||
| assert "gh" not in t.removeprefix("kev1-") |
There was a problem hiding this comment.
[nit] This assertion is flaky by construction. t.removeprefix("kev1-") is 26
characters of lower-cased base32, whose alphabet is a-z2-7 — so it contains g and h,
and the chance that "gh" appears somewhere in 26 characters is roughly
25 × (1/32)² ≈ 2.4%. It passes deterministically for this fixture's secret and
userkey, and will fail for about one in forty other combinations — including whatever
secret a deployment actually uses, if this ever runs against one.
It also adds nothing: assert "mrsabath" not in t on the line above is the property §4.1
cares about. The issuer prefix is two characters, so testing for its absence in a
130-bit digest is testing that a random string does not happen to contain a common bigram.
Dropping the line is the fix.
Every finding reproduced against 16158e5 before fixing, and every fix has a regression test in `tests/test_review_fixes.py` that fails without it (verified by mutation, one fix at a time). They are filed together because they share a cause worth naming: all five were invisible to 192 test functions, because the fixtures assembled the collaborators by hand with the arguments the production call sites fail to supply. **1. `single`-mode regression: the ownership index was never seeded.** The serious one, and against the exact guarantee this phase claims. Dropping Phase 2's `Minter` seeding loop left nothing to populate its replacement, so on the first start after an upgrade `owners.sqlite` is empty while `sessions.sqlite` is full and `exists()` reports live ids as free. With ~1,000 existing correlations a mint has roughly 1-in-25,000 odds of reissuing one, and then `upsert_session` overwrites a session and the new prompt appends to somebody's existing conversation. Phase 2's odds were zero. `OwnerIndex.seed_from()` plus a one-time backfill from every store at startup. §2.6 is right that per-start seeding is the wrong trade; it accidentally argued away the one-time backfill too, and §6.1a now records the distinction. **2. `create_group` did not pass the userkey.** `submit_members` got it and `groups.create()` did not, so in `multi` the group row and the groupid's owner claim landed in the shared store while every member row landed in the tenant's: 0 members forever, `maybe_complete` never fires, the batch never completes and never notifies. A retried POST with an Idempotency-Key read the tenant store `create` never wrote and returned `members: []` — worse than a second batch, because it looks like success. **3. `close_group`/`cancel_group` were unauthenticated cross-tenant mutations.** Neither called `_caller`; both resolved any tenant's group from the global index and acted on it. T7 covers *reads* staying as open as Phase 2's — Phase 2 had no tenants to cross and no cross-tenant mutation anywhere, so this was a capability being introduced, not an openness preserved. Now gated by `_authorize_mutation`, answering `404` for another tenant's group per §6.2's enumeration rule. Single-tenant mode is untouched. **4. `agent` was outside `SIGNED_ATTRS`** while `ce_agent` selects the AgentSpec that supplies `--permission-mode` and the tool allowlists — §5.1's "sandbox". A forged value widened the sandbox and the signature still verified, making a signed deployment worse than an unsigned one because an operator believes it is attested. Added in the same change as `userkey`/`depth`, per §8.3's one-canonicalisation-break rule. **5. `userkey` reached a filesystem path join unvalidated**, from an inbound Kafka header, and `Store.__init__` calls `mkdir(parents=True)` — so `..` created and wrote to a directory inside another tenant's store, or outside the tree. `tenancy.is_valid_userkey()` is the gate, exported from the module that is "the ONLY place an identity becomes a name" so a consumer can check the shape the producer guarantees. An invalid key is treated as a missing one — `shared/` plus `unattributed` — so a forgery is counted and visible rather than acted on. Note it is *truthy*, so the pre-existing `not userkey` check let it past. Also from the review: - **A known-but-unowned correlation 500'd on `/continue`.** `owner=None, unresolved=None` is falsy, so the handler proceeded, wrote a prompt row, then `TopicSet.requests(None)` raised — a traceback with an orphan row already committed, so the conversation showed a turn that was never submitted. Reading and publishing have different requirements, so `_publishable_owner` is now separate from `_owner_userkey`: shared-tier data stays readable, and a turn for it is refused before anything is written. - `claim()` uses `ON CONFLICT DO NOTHING` + a follow-up read, so `Collision` is the only way a conflict presents whether or not two writers race the in-process lock — the bare `INSERT` would have raised `IntegrityError`, which `Minter.mint` deliberately does not catch. - `created_utc` at millisecond precision (matching `ce.now_iso()`) with a `rowid` tiebreaker; at second granularity a 100-member batch had arbitrary order among its rows. - A bad `agent` name is a `400` on the call that made the mistake, not a `202` followed by an async `phase=error` the submitter may never read. - `max_turns: 0` is no longer silently rewritten to the spec's value while the spec path rejects zero. - `EB_TOPIC_PREFIX` validated (it is the one topic-name component `_slug` never sees, and `.`/`_` there reproduce the JMX metric collision) and `EB_MAX_OPEN_STORES` refuses readably instead of raising a bare `ValueError`. - Dropped a flaky assertion: `"gh" not in` 26 characters of base32 fails ~2.4% of the time by construction, and the `"mrsabath" not in` line above it is the real property. - `_canonicalise`'s docstring no longer claims static identities are byte-exact when the `@` branch catches address-shaped ones first. Design doc, where the code had disproved it (§9's "the code is the authority" rule): §6.1's layout corrected to put single-tenant stores in the bridge root; "Closing is safe" replaced with what commit 49aed05 established, including that the first fix attempt only pinned the SSE path and left the class open; new §6.1a documents `owner_index.py`; §2.6's "one new extension attribute" corrected to four, with which three are signed and why; §3.1 records the single-mode `events`/`dead` names. The status header now warns that `multi` does not work end to end until T6 — no response is consumed, which is more useful than "not yet isolated". Tests: 45 new (840 total), ruff clean. Still outstanding from the review and deliberately not in this commit: the T22 subset check, `RequestsMirror`/`NtfyPublisher` `stores=`, the handler-to-topic integration test, the `forget()` tombstone decision, and the four documents. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
|
Thanks — this was a genuinely useful review. All five must-fixes are in The Worth recording why I walked into it, because the design contributed: §2.6 argues
On the path traversal — I took your suggestion as written, including treating an invalid On Also addressed from your suggestions and nits: the known-but-unowned You were right about the flaky test, including the arithmetic. And right about my test count. It is +217 unique test functions (525 → 742; 845 On design conformanceFixed in-tree, under §9's "the code is the authority where implemented" rule:
Two corrections accepted: §6.2 does have the Deliberately not in this commitYour remaining suggestions, which I would rather do as a reviewable second pass than bury The four documents I have not started. Your argument that the findings are freshest now is All 11 checks passing on |
…etion Round two of the review, covering what was deferred from 6ebdd80. **T22 now checks the direction that enforces the rule.** `CONFIGMAP_VARS ⊆ found` catches a stale entry and nothing else; the direction that catches a *new* variable classified as neither is `found ⊆ CONFIGMAP_VARS ∪ SECRET_PATH_VARS`, which is the case the test exists for. Scoped to Phase 3's variables — the 36 that pre-date §8.3 are listed explicitly, with `test_the_baseline_list_is_still_accurate` keeping that list honest rather than letting it silently widen the exemption. Verified by deliberate failure, as asked: injecting an unclassified `e("EB_UNCLASSIFIED_X")` read into `config.py` turns the test red with the variable named. Also replaced the hardcoded ten-space indentation in the inlined-secret check with `\s*`, matching the precedent elsewhere in the same file — the old pattern would miss the same variable inlined in an initContainer, a sidecar or a deeper patch. **`RequestsMirror` and `NtfyPublisher` now take `stores=`.** Both knew the per-tenant topic names and still wrote to the shared store, which does look like the sweep in 77807b6 not reaching them. The mirror back-filled every prompt into `shared/`, so a correlation whose request the bridge did not originate showed a BLANK PROMPT on its owner's page — the exact gap the mirror exists to close. ntfy's `_last_prompt` and `_last_assistant_text` read only the shared store, so a tenant's notification arrived without the prompt or the reply, which §4.1 says is the content that lets a phone that cannot reach EventBridge still see the result. Its group-existence check is routed too. **A handler→topic integration test**, which the review argued was a better investment than more unit tests on the pieces. It is: faking at `KafkaProducer` rather than at `Producer` means the real handler, the real `Producer` and the real `TopicSet` all run, and the assertion is *which topic the bytes went to*. Confirmed by reverting each fix that it catches both defects it was written for — the `create_group` missing `userkey` and the `/continue` 500 on an unowned correlation. That test immediately earned itself twice over: the mutation run surfaced that `_publish_started`/`maybe_complete` swallow `ValueError` from `TopicSet` into a log line, so a caller forgetting a `userkey` produced a group that silently never announced itself — the same shape as the `TypeError` already guarded there. Both guards now cover both. **Deletion tombstones rather than freeing the id.** This is the one I said needed thought rather than a quick answer, and the answer is that reuse is unsafe. Verified: `forget()` used to `DELETE` the row, `exists()` is the `Minter`'s only uniqueness check, and `sessionuuid = uuid5(NAMESPACE, correlationid)` is unsalted by design — so a reissued `correlationid` derives *the same session uuid*, and a `claude` transcript left on a runner's volume becomes resumable by the new correlation. That is one user's conversation continuing inside somebody else's agent. So `forget()`/`forget_tenant()` keep the row, set `deleted_utc` and clear `userkey` — the id is reserved forever, and the tombstone discloses nothing about whose it was. The alternative is guaranteeing every transcript keyed on that uuid is purged everywhere, including volumes this process does not own, which is not a guarantee EventBridge can make. Cost is one short row per deleted correlation. Migrated with an `ALTER TABLE`, the same idiom `store.py` uses for `prompts.submitter`, and a test covers an index created before the column existed. The reviewer was also right that the docstring contradicted the code: it claimed "freeing the id for reuse is deliberate" about a `DELETE` nobody had reasoned about. Design doc: §6.1a records the tombstone decision and why, since the index is what enforces it. Tests: 855 passing (+13), ruff clean. Still outstanding: the four phase documents (`IMPLEMENTATION_REPORT3.md`, `README_PHASE3.md`, and the two owed Phase 2 documents). Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
|
Round two is in T22 now checks the direction that enforces the ruleYou were right that it could not fail for its stated reason. Scoped to Phase 3's variables, with the 36 that pre-date §8.3 listed explicitly — and I took the deliberate-failure suggestion literally: injecting an unclassified
|
Asked for in the #883 review, and the argument for writing it now rather than later was the most persuasive one there: the findings are attributable to specific commits today and will not be in a month. 361 lines, following `IMPLEMENTATION_REPORT1.md`'s contract — measured figures rather than estimates, failures with root causes, and an explicit account of what is NOT verified. Phase 0 and Phase 1 each ship three documents; Phase 2 shipped none, and the gap shows in that there is nowhere to put the §6.1 split-consumer trap except inside a design document. This starts closing that for Phase 3. **Measured rather than asserted.** §6.1 estimates the per-tenant descriptor cost and derives a ceiling; measured it is 6.0 descriptors per open store (10 tenants, 60 fds), which puts ~680 tenants under this host's 4096 soft limit and ~170 under the 1024 the design assumed — so squarely inside its "150-200" once the limit is held constant. The ownership index costs 27.4 µs per `claim`, 1.7 µs per `exists`, and 270 ms to seed 10,000 ids. That last figure is the one that matters: it settles §2.6's objection to seeding, because the *one-time* backfill it appeared to rule out costs 270 ms once, while the *per-start* seeding it correctly rejects is what the index exists to avoid. **Seven deltas from the design, recorded as decisions rather than drift** — the single-tenant store location, eviction by reference-drop, tombstoned deletion, the single-mode `events`/`dead` names, `agent` joining `SIGNED_ATTRS`, the exported userkey validator, and prefix validation. **The findings, ordered by what they cost to find.** Three are worth the document on their own: - The `single`-mode regression shipped behind a headline claiming it could not happen, with the reproduction (50 of 50 live ids reported free), the 1-in-25,000 arithmetic over a verified 50x50x10,000 id space, and the three reasons it is instructive: it was in the default configuration, the suite could not see it because no fixture simulated an upgrade, and §2.6's wording contributed. - The eviction contract, where the lesson is the *first* fix: pinning only the SSE path addressed the case that was easy to picture and left the class open. - `except Exception` converting three separate programming errors into log lines, on three separate occasions, which is why it is filed as one finding with the pattern extracted. Also records the suite's structural blind spot as a finding in its own right — a fake placed at the boundary under test cannot test that boundary — since that is what let two defects survive 200-odd new test functions. **§5 is the longest section, deliberately.** `multi` does not work end to end until T6 (no response is consumed); no cluster, no two-tenant deployment, no Kafka; reads not owner-scoped; Tier A is not isolation; the per-tenant throughput ceiling §10 asks for is still unmeasured; the `claude --help` flag assertion does not run in CI; `seed_from` is capped at 10,000 per store; and `is_valid_userkey` is shape validation, not authorisation. Every figure in the report was measured on this branch, every cited commit verified to be on it, and every cited design section verified to exist — §21.6 turned out to be `DESIGN_PHASE1.md`'s and is now qualified as such. Two numbers in my own PR description were wrong and are corrected here: the test delta is +232 unique functions (525 to 757), not the 139 or 217 claimed earlier. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
Implements
agentdocs/DESIGN_PHASE3.md§9 rollout steps 1–4, plus T5 — thefoundation for per-user isolation. Extends the Phase 2 identity work (#879) rather than
replacing any of it.
The design is a 12-step, 22-task phase, far larger than Phase 0/1/2 were individually.
§9 singles out the first four steps as the part that is safe to merge while
multihasnever been switched on anywhere: "the first four steps are refactors with tests, and
everything risky is behind a flag that defaults off." T5 is the keystone that everything
from step 5 on depends on, so it is here too.
What landed
shared/tenancy.py—userkey(),TopicSet,ntfy_topic()TopicSetthreaded through the producer, responses consumer and both mirrorsCaller, the user registry,ce_userkeystamped on requestsSIGNED_ATTRS+=userkey,depth— one change, per §8.3correlationid → userkeyindex,Minteruniquenesseventrunner/agentspec.py+build_cmdas a function of the spectest_manifests.pyNot implemented, deliberately: T6 (
ensure_subscribed), T7 (owner-scoped reads), T8(transcript auth), T9–T12 (capability keys, ntfy isolation,
k8s_tenant.py), T14–T19(triggers, fetched skills), T20 (Kafka ACLs), T21 (deletion).
The additive guarantee
EB_TENANCY_MODE=singleis the default and reproduces Phase 2 byte for byte. The 656tests that existed before this branch pass untouched — three test fakes were widened
to match a producer signature, but no assertion was weakened or removed. That is the
check §9 step 2 asks for, and it is why the largest diff in the phase is also the least
risky.
The
defaultAgentSpec is held to the same standard:test_default_spec_reproduces_phase2_argv_exactlycompares against a literaltranscription of the pre-Phase-3
build_cmd, not a call into the currentimplementation. Single-tenant mode also keeps its SQLite files exactly where Phase 2 left
them — directly in the bridge root, not under
shared/— so an upgraded deployment'ssessions and transcripts keep showing up instead of appearing to vanish.
Decisions worth a reviewer's eye
userkeyis the security control, not decoration._slugis lossy sokeys stay readable in
kubectl get kafkatopics; that is safe only because 32 bits ofSHA-256 over the canonical form are appended.
a.b@x.com,a_b@x.comanda-b@x.comall slug to
a-b-x-comand must not merge. The issuer is inside the hashed bytes too —otherwise a second configured issuer impersonates the first.
userkeycolumn (§6.1).store.pyhas28 methods and every one touches tenant data; a
WHERE userkey = ?remembered 28 timesis one that will eventually be forgotten. Separate files make the mistake structurally
unavailable — the connection is the tenant.
correlationidalone (§2.6).(userkey, correlationid)is right inside a tenant's store and exactly wrong here: two tenantsminting one id produce two distinct tuples, never conflict, and the collision passes
silently. That constraint is also what lets
sessionuuidstay unsalted.subscribe(pattern=…)(§3.2). A pattern is picked up bymetadata refresh (5 min default), so a new user's first response can be published before
anyone is subscribed — and with
auto_offset_reset=latestit is never read, with noerror anywhere.
max_turns/modelstill come from therequest (Phase 0 wire parameters); permission mode, tool lists and system prompt are
spec-only.
Bugs found in self-review and fixed
Each was confirmed by executing the real code, and each has a regression test that fails
without the fix:
/continuereturned a WSGI 500 where the owning tenant could not be resolved —reachable via a deregistered user, a NULL
submitter, or a mirror-backfilledcorrelation. Now a
503(or404) before anything is written.(
get_htmlmakes five calls) could have its store closed by a concurrent request foranother tenant →
ProgrammingError: Cannot operate on a closed database. Eviction nowdrops the reference and lets CPython close at the last holder. The first attempt at
this fix only pinned the SSE path — which closed the easiest case to see and left the
class open; the bug was in the eviction contract, not one caller.
GroupMirrornever learned about tenancy — restart-orphaned groups were settledagainst the shared store (so never settled at all, with silence as the symptom), and
replayed pre-Phase-3 events, which carry no
ce_userkey, put member rows inshared/while the group row sat in the tenant's store. That is the normal upgrade path.
except Exceptionhid two things: a signature mismatch that silently stopped everygroup event from publishing, and a broken ownership index reported as "correlation ID
space exhausted".
Two limitations stated rather than implied
multimode is not yet isolation anyone should present as such. Per-user storesexist and reads route to the owning tenant's store, but nothing checks that the
caller is that owner — T7/T8 are not in, so reads,
/continueandPUT /transcriptare as open as Phase 2's. §10 is also blunt that even once those land, separately-named
topics without Kafka ACLs are organisation, not isolation.
GET /v0/groupsnow requires an authenticated caller inmultimode, because thelist must be scoped to someone. §6.2's route table does not mention it; flagging
because it is a route that was open in Phase 2 and is the HTML page's own poller.
multialready refuses anonymous callers atresolve_caller, so the practical blastradius is small — but say so if you would rather it waited for T7.
Testing
(525 on
upstream/mainto 742), 845 collected with parametrisation. An earlier versionof this body said "656 baseline + 139 new", which was wrong — it diffed collected
counts across a moving baseline.
ruff checkandruff format --checkclean.print readable messages rather than tracebacks; the on-disk layout verified
upgrade-safe.
LRU's pinning check passed every test, because they asserted object identity rather than
the notification path that is the rule's actual purpose. Removing it now fails 4 tests.
Review round 1 (#883, @aslom) is addressed in
6ebdd80: five must-fixes, including asingle-mode regression where the ownership index was never seeded from an existing store(so
exists()reported live correlation ids as free and a mint could reissue one), anunauthenticated cross-tenant group mutation,
agentmissing fromSIGNED_ATTRSwhile itselects the tool-policy sandbox, and a
userkeyreaching a filesystem path joinunvalidated from an inbound Kafka header. Each reproduced before fixing; each has a
regression test verified by reverting its fix. Six design-doc contradictions corrected in
the same commit. Deferred, by agreement: the T22 subset check,
RequestsMirror/NtfyPublisherstores=, a handler-to-topic integration test, theforget()tombstone decision, and the four phase documents.Draft because this is the first slice of a 12-step phase — review comments here should
shape T6/T7/T8 before they are written.
Design:
eventing/agentdocs/DESIGN_PHASE3.md(#881). Reading order for review: §9 forwhy this scope, §2.3 for why the key ends in a hash, §6.1 for the per-user store trade,
§2.6 for the uniqueness guarantee.
🤖 Generated with Claude Code