feat: retired columns, Field(retired=True) (#625)

[0x054]

Closes #625

What

A column can now leave in two releases, so the release still serving during a rolling deploy keeps working. Release B declares the field retired:

class Author(Model):
    id: int | None = Field(default=None, primary_key=True)
    name: str
    bio: str | None = Field(index=True, retired=True)   # or Annotated[str | None, FerroField(index=True, retired=True)]

ferro migrate new writes one statement, the relaxation release A needs to keep writing bio:

ALTER TABLE "author" ALTER COLUMN "bio" DROP NOT NULL;   -- a table rebuild on SQLite

From B on, author.bio raises AttributeError, Author(bio="x") raises TypeError, and a query, order_by, update() or save(only=...) naming it fails with "retired in this release". No SELECT, INSERT or UPDATE the model issues names bio, so B works whether the column is still there or C has already dropped it. The column stays in the snapshot, nullable, with its index, unique, check, foreign key and every table-level artifact that names it. Release C deletes the declaration: the destructive DROP COLUMN, and the DROP TYPE when it was a native enum type's last column. ForeignKey(retired=True) does the same for a relation's *_id column.

How

• IR: SchemaColumn.retired (serde default false, skipped when false, so every existing envelope is byte-identical). The planner sees an ordinary column, so the relaxation is the existing AlterColumnNullability op and every door agrees (I-1): the create pass, migrate new, the reconciliation pass, drift and the Alembic bridge.
• Model: the metaclass derives a retired field's column spec once, while it is still a Pydantic field, then removes it from the Pydantic model. A retired relation never becomes a field. __ferro_columns__ / __ferro_query_columns__ hold the mapped columns only. The IR compiler emits the retired columns next to them (retired_column_specs), and table checks, composites and policies compile against both.
• Runtime: RegisteredModel::new leaves retired columns out of column_names and the codec plan. The root fetch now names column_names instead of "t".*, so hydration only ever sees the row the model owns (the complete-instance invariant, ADR-0008).
• Refusals at class creation, each with the ticket's text: non-optional, primary key, renamed_from together with retired, a BackRef over a retired ForeignKey, and BackRef / ManyToMany with retired=True. Also nullable=False on a retired field or ForeignKey (the same contradiction spelled through the option): ". is retired but declared nullable=False: a retired column stays nullable; drop nullable=False".

Verify

Ran the ticket's Verify block on 57fc78b: passed (cargo: schema-ir 46, ferro-migrate 387, crate 352; vectors 14; listed pytest files 1132 passed, 5 skipped, 1 xfailed, sqlite + postgres; prek clean).
TDD: RED cargo test --no-default-features --features testing a_retired_column_is_outside failed (left: ["id", "value", "bio"], right: ["id", "value"]); RED pytest tests/test_retired_columns.py -x failed ("bio" still in Author.model_fields); GREEN on both after the change, with the full suite (sqlite + postgres) green.
Verify block corrected: no

Notes for the reviewer

• Root fetch: every model's root fetch now names its columns. Before, a column the live table had beyond the model's (one added by hand, for example) was hydrated into the instance. test_auto_migrate.py::test_manual_migrate_on_live_pool_refreshes_cached_statements cached a model fetch against a table missing a declared column; it now caches a raw SELECT *, which still exercises the pool refresh.
• Keyword update() now validates its column names, as recipe keys always have (_literal_set in query/wire.py). Before, an unknown keyword reached the database. Two SET-emission tests wrote to columns their model did not declare, so they now use a dedicated SetKinds model.
• ForeignKey(related_name=...) is optional only when retired=True. Leaving it out otherwise is a TypeError naming the fix.
• Retired field + nullable=False: refused at class creation (coordinator ruling, I-6), beside the other refusals, in both declaration styles and on ForeignKey.
• src/ferro/migrations/historical.py needed no change: the retired key is ignored there, so a historical model exposes the column as an ordinary nullable one, which a new test pins.
• Casebook: "A6-retire-a-required-enum-column" (a SQLite rebuild, added to REBUILD_CASES) and "A6-delete-a-retired-enum-column" pass pins (a)–(g) on both dialects. No FINDINGS entry.

Outside Files owned

Observations for the reviewer (behaviour, as built):

• The root fetch naming the registration's columns instead of * is the ticket's acceptance criterion; it changes every model's root SELECT.
• Keyword update() checking its column names is what the ticket's update(bio="x") scenario requires.
• ForeignKey without related_name only when retired=True is consistent with "a BackRef over a retired relation is refused".

Files, each with its reason:

• src/ferro/fields.py: Field(retired=...) lives here, not in base.py.
• src/ferro/models.py: Model.__init__ (the construction TypeError) and __getattr__ (the AttributeError text) live here; base.py has no model class.
• src/ferro/relations/__init__.py: a retired relation's target is resolved and its BackRef refused during relationship resolution.
• src/ferro/query/wire.py: keyword update() column validation.
• src/live_ir.rs, src/codec_plan.rs, src/schema.rs, crates/ferro-ddl-lowering/src/lib.rs, crates/ferro-migrate/src/{plan,tests,revision_tests}.rs, crates/ferro-migrate/src/generate/mod.rs: each SchemaColumn literal gains retired: false (a one-line edit per literal; live_ir.rs is Foreign defaults are drift: the live read carries defaults, DropDefault on every door (#622) #628's file, touched only for this).
• tests/test_auto_migrate.py, tests/test_query_wire_vectors.py, tests/test_docs_migrations.py: see above; the last lists the new example.

🤖 Generated with Claude Code

https://claude.ai/code/session_014ERLk8yFhJy7rdDyU7rGdR

[0x054]

Spec: PASS

• Minor · tests/test_retired_columns.py:346 at 57fc78b · Missing (test shape) · the happy-path scenario asks for "the captured SQL names no bio"; no statement is captured. test_a_retired_column_dropped_by_hand_changes_nothing (save, bulk_create, update, save(only), select, get against a table without bio, and without team_id on Postgres) proves the same thing behaviourally, so this does not block.
• Minor · tests/test_historical_models.py (new test at end of file) at 57fc78b · Missing (test shape) · the scenario says "a data step can read it"; the test builds the historical model and constructs an instance, but runs no data step.
• Minor · tests/test_retired_columns.py:180 at 57fc78b · Missing · "What to build" lists a projection naming the retired column among the refused surfaces; lambda, order_by, update() and save(only=) are tested, a projection is not. It goes through the same validate_query_column.
• Minor · src/ferro/metaclass.py:512 at 57fc78b · Misunderstood (narrow) · the ticket refuses retired "together with a live renamed_from"; this refuses any renamed_from, including an inert one. The class cannot know liveness without the snapshot, and the message names the fix (drop renamed_from), so I accept it.

Checked: the four ticket refusals, with the ticket's exact texts. BackRef/ManyToMany with retired=True. The ruling's nullable=False refusal, tested in both styles (tests/test_retired_columns.py:231) and on ForeignKey (:247), and named in the docs refusal sentence (migrations.md "Refused at class definition…"). I-1: no new op. The relaxation is the planner's existing AlterColumnNullability, and the IR column is ordinary apart from the flag. A6-retire-a-required-enum-column (in REBUILD_CASES) and A6-delete-a-retired-enum-column were added to the casebook, FINDINGS is untouched, and test_cross_emitter_parity is green on both dialects. I-2: only the SELECT list changed (select_root_columns). No __init__ call was added on the Rust side, and test_hydration_holds_the_mapped_columns_only asserts every Pydantic slot. I-7: both tabs are backed by migrations_retire.py and _annotated.py, and test_docs_migrations runs both and compares the migrations byte for byte. IR vector contract: schema_retired_column_v2.json is added, no existing vector file changed, and retired is skip_serializing_if false, so the existing-vector tests stay green. Outside Files owned: every declared file and observation is accepted. (1) Naming the root fetch's columns is the acceptance criterion. The test_auto_migrate.py change keeps the pool-refresh coverage, because a raw SELECT * is the statement whose result shape the DDL changes, and the model fetch is still re-run after the migration. (2) Keyword update() validation changes only an unknown keyword, which now fails as an AttributeError before reaching the database. The only other caller-visible change is in two wire tests that wrote undeclared columns. (3) related_name is optional only with retired=True; it is tested. No undeclared file.
Cannot verify: whether #627 (the metaclass seam) can consume _retire_fields/__ferro_retired_names__ as built, and whether #628 rebases cleanly over the one-line retired: false in src/live_ir.rs.

Quality: PASS

• Minor · src/ferro/columns.py:176 at 57fc78b · the function-level import of _shadow_fk_types runs inside the per-relation loop. Hoist it above the loop (it is still a lazy import).

Verify: passed on 57fc78b. maturin develop ok. cargo: schema-ir 46, ferro-migrate 387, crate 352 passed. IR vectors: 14 passed. Listed pytest files: 1132 passed, 5 skipped, 1 xfailed (sqlite and postgres). prek run --all-files: clean. The full suite was also run on both backends: 4016 passed, 60 skipped, 2 xfailed. Worker's sdd-verify note is present on HEAD.