Skip to content

fix(catalog): grants, rules and catalog grants follow a table rename - #764

Merged
viethqb merged 24 commits into
mainfrom
fix/grants-follow-table-rename
Oct 28, 2026
Merged

fix(catalog): grants, rules and catalog grants follow a table rename#764
viethqb merged 24 commits into
mainfrom
fix/grants-follow-table-rename

Conversation

@viethqb

@viethqb viethqb commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Summary

A table or column rename now carries the permissions that name it. Before this, a rename left the new name without its deny grants, path masks and row filters (so the table could escape its restrictions), and left allow and catalog grants on the old name (so the renamed table lost its readers).

  • Deny grants, path masks and path row filters are copied to the new name before the Iceberg commit. The originals stay on the old name, dormant, so a new table at the old name and the swap pattern stay protected. The rename waits until Trino and StarRocks report the new policy version (90 s deadline; a silent engine gets 409 and nothing is renamed).
  • Allow grants move to the new name after the commit, with the same ids.
  • Catalog grants (table securable) are re-pointed after the commit. Uncancellable; a failure is logged.
  • Refusals (409), naming the rows: a restrictive name pattern that covered the old name and would not cover the new one; a hand-written Ranger policy that names the source with a mask, row filter, deny or exclusion-list allow. A pattern that already covers the new name with an equal restriction does not refuse (G5).
  • Recursive namespace drop deletes literal allows and table catalog grants under the namespace.
  • Overview: permissions_on_missing_tables, permissions_on_missing_columns, catalog_grants_on_missing_tables, renamed_names_lost_grants, renamed_names_lost_catalog_grants. Governance > Changes gains sentences for perm_grant:move|copy|left, perm_rule:copy, ranger_policy:left, catalog_grant:move|delete.
  • D17 fix (live check): the lost-grant findings count every move, not only each grant's latest, and a name is reported unless it carries every moved allow.

Spec: docs/en/specs/2026-10-27-grants-follow-table-rename-design.md (G1–G6 user decisions, D1–D21 defaults, §9 release note and Limitation, §14 deviations and live-check notes). Plan: docs/en/plans/2026-10-27-grants-follow-table-rename.md.

Refs KI-308, KI-309. Prerequisite F-A (#758) and the vended-credentials package (#760) are merged.

Defaults the user may revisit (spec §4)

D1–D21 are recorded with their reasons in spec §4. The ones most worth a look: D1 (the hold is a copy-then-forward, not a block on reads), D4 (patterns are refused or reported, never split), D8 (catalog grants are re-pointed with no hold), D10 (grant and rule writes on a held name wait their turn), D17 (the two lost-grant findings).

Gate

  • cargo fmt --all -- --check: clean.
  • cargo clippy --workspace --all-targets -- -D warnings: clean (after merging origin/main at aed09501).
  • cargo test -p egret-controlplane -p egret-catalog-engine -p egret-perm-compile: 77 binaries pass. One failure, table_hook_db::a_rename_no_trino_plugin_confirms_is_refused_and_undone, failed with "other permission changes were being written" under parallel load and passed alone (5.8 s). It is a name-lock timing flake in the rename path, which this PR does not change.
  • Earlier, on fd0d57ae: workspace tests 8573 pass, 0 DB skips; known environment-only failures: egret-workflow --test sql_starrocks_live (needs a live StarRocks) and egret-dataapi --test serving_shared_store (needs EGRET_TEST_SHARED_STORE_URL).
  • Frontend (earlier): lint, tsc -b, build and the bundle tests pass.

Live check (plan Task 19, Trino 483 + StarRocks 4.1.6, real Ranger plugins, rename read loop every 2 s)

Two runs. Run 2 is on 139db73d and is the one to read.

Step Result
1 baseline PASS on both engines
2 rename through read loop PASS. Rename 57 s. No contractor row; analyst-u sees only eu rows; email hashed for uncleared users
3 catalog grant PASS (missing table returns 403, not 404; see Observations)
4 swap PASS. Analyst-u still reads the new name through a workspace grant (not in the test's expectation; the grant is correct)
5 G4 409, G5 partial/full cover PASS
6 G3 operator policy 409 PASS, via a manual StarRocks profile (manual Trino profiles cannot take a tag service)
7 column rename PASS. Mask follows the column
8 timing, three repeats 54.0 s, 56.8 s, 55.6 s (under 90 s). Trino's tag -1 regression seen once (E27)
9 silent Trino PASS. Deny rename 409 after 90.0 s naming <no plugin has polled>; after restart 204 in 56 s. Allow-only rename 204 in ~1 s while Trino was stopped
10 recursive drop PASS. Allow and catalog grant deleted, deny kept
11 UI PARTIAL. Overview and Changes screenshots at 1440 and 390 px, light and dark. Four of five findings seen; ranger_policy:left, catalog_grant:delete and the Who column not checked

G6 gap (commit to first allowed read), run 1: about 28.8 s on Trino, 14 s on StarRocks. Run 2: no refusal observed at 2 s granularity.

D1/D2 (lost-grant findings), fixed in this PR: both now show for orders_v2 after the swap and persist to the end.

Known limitations and observations (not fixed)

  • D3: SQL Lab's 60 s limit reports FAILED for a rename held longer than that, while the rename still commits (a 61 s hold). The Changes feed is the record.
  • N1: tagging a table whose mask is decided by an exempt returns 502 "undecidable".
  • N2: binding a manual profile with an ambiguous operator mask is refused.
  • Missing table returns 403 on the catalog-grant route, not 404.
  • StarRocks masks email to 128 hex characters; Trino to 64.
  • The catalog-grant lost sentence keeps the historical "moved to" wording after a later move.

Non-goals

Views and their renames (spec §11, item 1); case-only renames of catalog grants (F-B, §11 item 2); credential lifetime (KI-315, separate package); cross-warehouse registration (KI-316).

Docs

  • Docs box not ticked. The docs follow-up PR in egret-data/docs (KI-308, KI-309, spec §11 rows, the G6 Limitation of spec §9, D20) is not opened yet. Per the repo's CLAUDE.md, a PR that changes user-visible behaviour must include it. This PR changes the Overview findings and the Changes sentences.
  • UI screenshots of the new Overview findings exist (live check, 1440/390, light/dark) but are not attached here.

Related defects

🤖 Generated with Claude Code

viethqb and others added 24 commits October 28, 2026 04:51
@viethqb @claude
docs: design and plan for grants, rules and catalog grants following …
139fac7 
…a table rename

Spec and implementation plan for KI-308 and KI-309: restrictive grants and
path rules are copied to a renamed table before the commit and stay on the
old name, allows and catalog grants move after it, restrictive patterns and
hand-written restrictions refuse the rename, and stale references are
reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
docs: record G5 and G6 in the grants-follow-rename design and plan
ffbbf76 
G5 lets a rename through when the new name is already covered by a
restriction of the same kind and equal effect (defined per kind; partial
covers refuse and the 409 names what is missing; egret never creates the
cover). G6 accepts the allow gap and records it as a docs Limitation. The
plan gains a cover task; open questions are closed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(policy): a confirmed version stays confirmed for the rest of a wait
a3cf6eb 
Trino's tag row regresses to -1 on the first tag poll after every policy
download. A rename that waits for a Trino tag pair and a Trino policy pair
would otherwise wait one more tag poll after both were reported. Each pair
is now latched the first time every recent plugin reports it (spec D7).

Also records Step 0 of the grants-follow-rename package in the spec (§14).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(perm): select the grants and rules a table or column move carries
1815c07 
egret_perm_compile::follow sorts the catalog's explicit grants and path
rules for one coordinate move: literal rows follow (deny grants and path
rules are copied, allows moved), restrictive patterns that stop matching
refuse, allow patterns that stop matching are left, and a destination of
more than one namespace level refuses restrictions and leaves allows
(spec D3, D4, D14).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(perm): check hand-written policies and copy, move or undo a move…
5ee29f3 
…'s rows

follow::handwritten_via finds the enabled operator policies on the bound
profiles that name the source literally: a mask, a row filter, a deny or an
exclusion list on the renamed level refuses, a plain allow is reported
(spec D5, G3). copy_in_tx inserts a restrictive row's copy on the
destination, skipping one whose equivalent is already there and naming a
rule copy '<name> (<destination>)' with a suffix on a collision (D1, D13);
delete_copies_in_tx undoes exactly those; move_allows_in_tx re-points an
allow in place, keeping its id (D2). refusal_message words the 409 (§8).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
fix(perm): count dormant restrictions in a held rename's compile
9fe4abb 
A held rename's copy compiles both names with the source marked leaving,
and the profile total skipped every body on a leaving coordinate. Grants
and path rules stay on the old name after the rename (spec G2), so their
bodies are served afterwards: only clearance and tag-rule bodies, which
move with the classifications, are skipped now (D12).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(perm): refuse grant and rule writes on a name a rename is holding
ac28e8c 
A grant or rule create, patch or delete that names a literal table a held
rename is moving now answers 409 table_change_held, as a catalog-tag write
does: it tries the hold's name lock for the old and the new scope on its
own transaction (spec D10). A steward's form saved during the hold can no
longer re-point a moved allow back or remove a copy the compensation then
cannot find. Pattern scopes and catalogs without an embedded warehouse take
no lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
fix(catalog): copy a table's restrictions to its new name before a re…
fc8834f 
…name commits

A table rename whose source a deny grant, a path mask or a path row filter
names now takes the hold, whether or not it is classified. The copy
transaction re-reads what the move carries under the locks, refuses there
what a pattern or a hand-written policy would let escape, and inserts a
copy of each restriction on the new name; the originals stay on the old
name (spec G2, D1, D11). A copied restriction adds the Trino profiles'
policy versions to the wait (D6); a refused or undone rename deletes
exactly the copies. The fake Trino plugin of the tests polls its policies
too, as Trino 483 does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(catalog): move a renamed table's allow grants and record every move
b3a8c73 
After a held rename commits, its allow grants are re-pointed to the new
name in place, ids kept, in the after-commit transaction (spec G2, D2,
G6); a rename that carries only allows waits for no plugin (D11). The
same transaction records perm_grant:move per allow and perm_grant:copy /
perm_rule:copy per restriction copied, by the catalog client that renamed
(D9), so a refused or undone rename records nothing. The governance feed
words them for policy:read readers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(catalog): refuse a rename a restrictive pattern would let escape
7c8decc 
A deny grant, mask or row filter whose name pattern or schema scope
covered the renamed table and would not cover its new name refuses the
rename with 409, naming the rows, at plan time and again under the hold's
locks; a following restriction into a namespace of several levels refuses
too (spec G4, D4, D14). An allow pattern that stops covering the table is
left as it is and recorded as perm_grant:left, which the feed words.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(catalog): let a rename through when the new name is already as r…
eedfc84 
…estricted

A restrictive pattern row that would stop covering a renamed table no
longer refuses when the new name already carries the same restriction:
denies whose actions and subjects together cover it, or a mask or row
filter of the same type and expression exempting no more; the rename's own
copies count (spec G5, §5.9, D21). A partial cover still refuses and the
409 says what the new name lacks. A covered rename takes the hold without a
wait, so the cover cannot be deleted while it is checked again under the
locks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(catalog): refuse a rename a hand-written restriction names
f55616e 
An enabled operator policy on a profile bound to the catalog that names
the renamed table literally with a mask, a row filter, a deny or an
exclusion list refuses the rename with 409 naming profile/policy (id); the
policy is never rewritten (spec G3, D5). A plain hand-written allow naming
it is reported as ranger_policy:left, which the feed words.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
test(catalog): pin the swap pattern and what a rename does not carry
f91d7b3 
The swap pattern leaves the new table under the original deny while the
allows follow their tables; renaming back and forth does not stack
copies (D13); a case-only rename, a view sent to the table endpoint and a
source that carries nothing move nothing; a 500 after a committed rename
still moves the allows. A migrated restriction stays undoable after a
rename, and a moved migrated allow makes its undo refuse (spec §10).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
test(catalog): a column rename carries its column grants and rules
3d459be 
A renamed column keeps its path mask and column deny on both names, copied
before the commit and waited for on Trino; its column allow moves after
the commit; a column pattern that would stop covering it refuses; swapped
columns carry both sets of denies, and swapped columns with different
masks are refused by the overlap check (spec §5.4). The column plan landed
with the table rename's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
fix(catalog): catalog grants follow a renamed table
a7210b3 
After a rename of a live table commits, held or not, every table catalog
grant on the old name is re-pointed to the new one, names compared with
lakekeeper's collation as the guard compares them; a grant the role
already holds there merges; namespace grants stay (spec D8, KI-309). The
guard reads grants on every request, so nothing waits, and since F4 the
vended credentials follow too. Each move records catalog_grant:move by
the verified caller in a new Catalog family of the feed, read with
catalog_registry:read (D9).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
fix(catalog): purge allows and catalog grants with a recursively drop…
db60484 
…ped namespace

After a recursive namespace drop commits, the literal allow grants on its
tables go in the purge transaction, recorded as perm_grant:delete with the
trigger iceberg_drop_namespace, and then the table catalog grants in the
namespace and below it, recorded as catalog_grant:delete, both by the
caller (spec D15). Deny grants, path rules, schema and pattern scopes and
namespace catalog grants stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(governance): report grants, rules and catalog grants on tables t…
52166c5 
…hat do not exist

Three Overview findings (spec D16): permissions_on_missing_tables (literal
grants, path rules and hand-written policy tables on names with no table,
one sample per name, saying when a rename kept a restriction there on
purpose), permissions_on_missing_columns (column-level grants and rules a
live table lacks, with the rename candidate), both for policy:read, and
catalog_grants_on_missing_tables for catalog_registry:read, at any
namespace depth through the new table_states_ns_via, which the
single-level table_states_via now wraps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(governance): report names a rename left without their allow grants
089cbac 
Two Overview findings beside renamed_names_reused (spec D17):
renamed_names_lost_grants for policy:read readers, from perm_grant:move
events, and renamed_names_lost_catalog_grants for catalog_registry:read
readers, from catalog_grant:move events. A name is reported when it is a
live table again and carries none of the moved grants; the sample links
the renamed table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
feat(governance): name the new stale-permission checks
e01a5c7 
The Overview's Not checked for you lines name the five checks the
grants-follow package adds instead of their raw kinds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
chore(api): regenerate the client for the changes feed's Catalog family
fd0d57a 
The governance changes route's description now names the catalog-grant
events a catalog_registry:read reader sees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
docs: record that the grants-follow package fixes #757's related defe…
b3445b3 
…cts 1-3

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb @claude
docs(grants-follow): record the Task 18 local gate in the spec
f23cb78 
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
@viethqb @claude
fix(governance): lost-grant findings count every move and partly kept…
139db73 
… names

The "name lost its allows / catalog grants" findings read only each grant's
latest move, so a grant moved on again (table renamed twice) erased the
report for the name it left, and a new table carrying one moved allow hid
the finding for all of them. Every move now counts, and a name is reported
unless it carries every moved allow.

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
@viethqb @claude
Merge origin/main into fix/grants-follow-table-rename
5eb481c 
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
@viethqb
viethqb merged commit 79b6e42 into main Oct 28, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@viethqb