feat(python): eject owns the emitters its generators write through

CHANGELOG.md

@@ -217,6 +217,69 @@ generator catalog and the Atlas evaluation are 1.2 work.
     calls one.
   The cross-port manifest lists `api-docs` for `csharp`.
 
+- **Python: an ejected generator owns the emitters it writes through, and `api-docs` can be
+  ejected.** `metaobjects eject` used to copy one file per generator, so an owned `entity`
+  still called the shared-enum and M:N writers, and an owned `output-parser`, `extractor`,
+  `output-prompt` or `trace-helper` still called the prompt tier's field mapping, extract
+  delegate, extract schema and format-spec writers. A defect in any of them waited for a
+  release. Eject now copies those emitters with the generator, into `codegen/runtime/`, and the
+  copy reads them from there: `fr019_shared_enum` and `m2m_codegen` with `entity`;
+  `m2m_codegen` with `filter-allowlist`; `fr010_field_mapping` and `m2m_codegen` with `routes`;
+  `extract_delegate_emitter` and `fr010_field_mapping` with `output-parser` and `extractor`;
+  `output_format_spec_emitter` and `fr010_field_mapping` with `output-prompt`;
+  `extract_schema_emitter` and `fr010_field_mapping` with `trace-helper`. A file two generators
+  share is copied once and never overwritten, and `metaobjects gen --list` marks each copy
+  `identical` or `DIFFERS`. What stays in the package is the engine: the generator base and
+  file writer, `metaobjects.apidocs.naming`, `type_map` and `value_objects` (the naming and
+  type-mapping tables), and the model walks (`find_inbound`, `tph_plan`,
+  `instance_artifacts`). `template` is not ejectable: it has no emit logic of its own. Packaged
+  output is unchanged and gated to be.
+  - **`api-docs` is a generator you can own.** `metaobjects eject api-docs` copies the page
+    generator with its page model, builder, renderer and path math. The api reference is now a
+    generator (`docs_surface = "api"`) that `metaobjects docs` runs: an owned copy is listed
+    under the new `docs.generators` block of `metaobjects.config.yaml` and runs in place of the
+    packaged one. `gen` and `verify --codegen` skip a docs generator they find under a target
+    and say so: the packaged `api-docs` name is to be removed (`metaobjects docs` already runs
+    it), an owned copy is to be listed under `docs.generators`. The reference asks `shared_predicates` which objects get a router (the new
+    `emits_writable_router` and the moved `is_read_only_routed`, beside `emits_router`), the
+    module your owned `routes` and `filter-allowlist` read, so one edit to the owned predicate
+    moves the router, the allowlist and the pages together. The cross-port manifest lists
+    `api-docs` for `python`.
+  - **Where things moved.** The emitters now live in `metaobjects.codegen.runtime` (a
+    module the helper-runtime copy already used): `extract_delegate_emitter`,
+    `fr010_field_mapping`, `output_format_spec_emitter`, `extract_schema_emitter`,
+    `fr019_shared_enum`, `m2m_codegen`, and the api reference as `api_docs_model`,
+    `api_docs_builder`, `api_docs_renderer` and `api_docs_paths`. `metaobjects.apidocs` keeps
+    `naming` and exports `PythonApiModelBuilder`, `PythonApiDocsRenderer` and the path
+    functions as before. New public API: `DocsRunFacts`, `GenContext.docs`, `DEFAULT_API_SUBDIR`
+    and `DOCS_SURFACE_API` in `metaobjects.codegen.generator`, `ProjectConfig.docs_generators`,
+    `GeneratorEntry.docs_surface`, and `metaobjects.codegen.docs_run`.
+  - **A copy you ejected before this release** keeps running: it imports the emitters from
+    their old paths, which are deprecated aliases (see **Deprecated**), so it calls the
+    packaged emitters as it always did. `metaobjects gen --list` marks it `DIFFERS: N behind`.
+    To own them, eject the same name into a scratch directory and copy the generator's
+    `runtime/` files over (see
+    [own-your-codegen](docs/features/own-your-codegen.md#the-emitters-a-generator-writes-through-come-with-it)).
+  - **A `codegen/runtime/shared_predicates.py` you already own needs two new functions.**
+    `routes` now reads `is_read_only_routed` from it and `api-docs` reads that and
+    `emits_writable_router`, and eject never overwrites a runtime copy. When eject keeps a
+    runtime copy it now says how the copy compares with the installed module
+    (`kept your copy — identical`, or `kept your copy — DIFFERS: N behind, M of your own`),
+    and for a copy that is behind it prints two commands that run from the working
+    directory, `--root` or not: the `diff -u` of your copy against the installed file, whose
+    hunks you apply, and the `cp` of the installed file over yours, for a copy you have not
+    changed.
+  - **A gate for the rule.** The engine surface is written once, in the shared manifest
+    (`eject.engineSurface.python`), and `tests/conformance/test_eject_contract.py` ejects every
+    ejectable generator alone into an empty project and fails on any import outside it. A
+    second test (`tests/codegen/test_eject_alone.py`) runs each owned copy with every packaged
+    emit module made unimportable and compares its output byte for byte with the packaged
+    generator's over a model that exercises each one (the router's own runtime imports are the
+    one documented difference). Another fails on a helper module no registry entry hands over,
+    or a generator module no registry entry names.
+    The nine Python entries of the shared eject contract's `eject.pending` list are closed, and the
+    contract test runs `api-docs` through `metaobjects docs`, the way it is run.
+
 - **The stock `entity`, `queries`, `routes` and `barrel` generators are importable again
   (TypeScript).** `entityFile`, `queriesFile`, `routesFile` and `barrel` are exported from
   `@metaobjectsdev/codegen-ts/generators` once more, beside `namesFile`, `routesFileHono`
@@ -871,6 +934,16 @@ until you regenerate.
 - **C#: `Fr010FieldMapping.CSharpStringLiteral(string)` is `[Obsolete]`.** It forwards to
   `CSharpNaming.CSharpStringLiteral(string)`, which every generator now calls (so an ejected
   `names` needs no emitter). Same signature and result; switch the call when you next touch it.
+- **Python: the old import paths of the emitters that moved into `metaobjects.codegen.runtime`.**
+  `metaobjects.codegen.extract_delegate_emitter`, `.fr010_field_mapping`,
+  `.output_format_spec_emitter`, `.extract_schema_emitter`, `metaobjects.codegen.generators.fr019_shared_enum`,
+  `.m2m_codegen`, and `metaobjects.apidocs.api_model`, `.builder`, `.renderer`, `.paths` are
+  each one-line modules that become the new module and warn (`DeprecationWarning`) on import.
+  Same objects, so nothing else changes; import from `metaobjects.codegen.runtime` (the api
+  reference modules are `api_docs_model`, `api_docs_builder`, `api_docs_renderer` and
+  `api_docs_paths` there). The old paths will be removed in a later release.
+  `metaobjects.codegen.generators.router_generator.is_read_only_routed` moved to
+  `metaobjects.codegen.runtime.shared_predicates`; the router module still re-exports it.
 
 ### Removed
 

docs/features/cli.md

@@ -376,6 +376,9 @@ requirementTests:                # optional; options of the `requirement-tests`
   filter: codegen.requirement_hooks:include    # "module:symbol", resolved config-relative
   renderer: codegen.requirement_hooks:render   # "module:symbol", resolved config-relative
   warnUncovered: true            # the default
+docs:                            # optional; options of `metaobjects docs`
+  generators:                    # owned docs generators as "module:symbol", run in place of the packaged one
+    - codegen.generators.api_docs:api_docs_generator
 targets:
   api:
     outDir: src/generated/api

docs/features/own-your-codegen.md

@@ -695,9 +695,22 @@ targets:
 Eject prints the exact entry for each copy. It never overwrites an existing copy without
 `--force` and never edits your config. `metaobjects gen --list` marks each owned copy
 `identical` or `DIFFERS: N behind, M of your own` against the packaged reference, so you
-can see when an upgrade changed the generator you copied. A copy imports the same
-`metaobjects.codegen.*` modules the packaged one does, and those module paths are the
-surface an owned generator builds on.
+can see when an upgrade changed the generator you copied. Every generator `metaobjects
+gen --list` shows is ejectable except `template`, the Mustache primitive, which has no emit
+logic of its own to own.
+
+An ejected generator owns every line that writes its output. Eject copies the generator
+with the emitters it writes through (below), and the copy imports only the engine from the
+package: the generator base and file writer (`metaobjects.codegen.generator`), the
+formatter, the naming and type-mapping tables (`metaobjects.apidocs.naming`,
+`metaobjects.codegen.type_map`, `metaobjects.codegen.value_objects`), the model walks
+(`find_inbound`, `tph_plan`, `instance_artifacts`) and the loader. Those module paths are
+the surface an owned generator builds on. It is written once, in the shared manifest
+(`fixtures/generator-registry-conformance/registry.json`, `eject.engineSurface.python`), and
+`tests/conformance/test_eject_contract.py` ejects each generator alone into an empty project
+and rejects any import outside it. `tests/codegen/test_eject_alone.py` then runs each owned
+copy with the packaged emitters made unimportable and compares its output byte for byte with
+the packaged generator's.
 
 `requirement-tests` ejects the same way, to `codegen/generators/requirement_tests.py`,
 wired as `codegen.generators.requirement_tests:requirement_tests_generator`. The copy holds
@@ -742,9 +755,12 @@ like the rest of the package.
   --codegen` reports the stale `_runtime/` file until you run `gen`. That is the right
   answer, because the committed package is out of date.
 - **Eject never overwrites your runtime.** An existing `codegen/runtime/<module>.py` is
-  kept, even with `--force`, and eject says so. `metaobjects gen --list` marks each
-  runtime copy `identical` or `DIFFERS: N behind, M of your own` against the installed
-  one.
+  kept, even with `--force`, and eject says so, with how the copy compares with the
+  installed one (`kept your copy — identical` or `kept your copy — DIFFERS: N behind, M of
+  your own`). When the copy is behind, the next line names the generator that may need the
+  newer one and two commands, both written to run from the directory you ran eject in: the
+  `diff -u` of your copy against the installed file, and the `cp` to use only if you have
+  not changed yours. `metaobjects gen --list` marks each runtime copy the same way.
 - **A project that has not ejected `routes` is unchanged.** The packaged generator still
   imports `metaobjects.codegen.runtime.*` and emits no `_runtime/`.
 
@@ -764,34 +780,109 @@ codegen/runtime/`). The same recipe covers the generator itself: `metaobjects ej
 routes` into a scratch directory gives you the new reference to diff your
 `codegen/generators/routes.py` against.
 
-### The rules two generators share come with them
+### The emitters a generator writes through come with it
 
-Some emit rules are asked by more than one generator, and the answers have to match.
-`routes` and `filter-allowlist` both ask `emits_router` (which objects get a router; the
-generated router imports the allowlist module, so a mismatch is an import error at
-startup). `entity` and `extractor` both ask `has_literal_default`. These live in one
-module, `shared_predicates`, and ejecting any of those four generators copies it too:
+A generator that calls a packaged emitter is a shell around it: a defect in the emitter
+still waits for a release. So eject copies the emitters too, into `codegen/runtime/`, next
+to the generator. The set is each registry entry's `runtime` list:
+
+| Ejecting | also copies |
+|---|---|
+| `entity` | `fr019_shared_enum`, `m2m_codegen`, `shared_predicates` |
+| `routes` | `fr010_field_mapping`, `m2m_codegen`, `shared_predicates`, and the router's own runtime (above) |
+| `filter-allowlist` | `m2m_codegen`, `shared_predicates` |
+| `output-parser` | `extract_delegate_emitter`, `fr010_field_mapping` |
+| `extractor` | `extract_delegate_emitter`, `fr010_field_mapping`, `shared_predicates` |
+| `output-prompt` | `output_format_spec_emitter`, `fr010_field_mapping` |
+| `trace-helper` | `extract_schema_emitter`, `fr010_field_mapping` |
+| `api-docs` | `api_docs_model`, `api_docs_builder`, `api_docs_renderer`, `api_docs_paths`, `m2m_codegen`, `shared_predicates` |
+
+`names`, `render-helper` and `requirement-tests` call no packaged emitter. A file two
+generators share is copied once and an existing copy is never overwritten, even with
+`--force`; `metaobjects gen --list` marks each copy `identical` or `DIFFERS: N behind, M of
+your own`.
 
 ```text
 codegen/
-  generators/routes.py
-  generators/filter_allowlist.py
-  runtime/shared_predicates.py    # the shared rules, now your code
+  generators/output_parser.py
+  generators/extractor.py
+  runtime/extract_delegate_emitter.py   # the one copy both read
+  runtime/fr010_field_mapping.py
 ```
 
-Each generator imports it relatively (`from ..runtime.shared_predicates import ...`), so
-the verbatim copy in `codegen/generators/` reads `codegen/runtime/shared_predicates.py`
-and never the installed module. Two owned generators read the one copy: change
-`emits_router` there and your `routes` and your `filter-allowlist` change together. No
-generated code imports this module. It runs at `gen` time only, so nothing is emitted into
-`_runtime/` for it. The rest works as for the runtime above: an existing copy is kept, and
-`--list` marks it `identical` or `DIFFERS`.
+Each generator imports its helpers relatively (`from ..runtime import
+extract_delegate_emitter as rde`), so the verbatim copy in `codegen/generators/` reads
+`codegen/runtime/` and never the installed package. Two owned generators read the one copy:
+change it and both change. No generated code imports these modules; they run at `gen` time
+only, so nothing is emitted into `_runtime/` for them.
+
+**The rules two generators share** live in the same place. `routes` and `filter-allowlist`
+both ask `emits_router` (which objects get a router; the generated router imports the
+allowlist module, so a mismatch is an import error at startup). `entity` and `extractor`
+both ask `has_literal_default`. They are in `shared_predicates`, which each of those
+generators lists, so editing `emits_router` there moves your `routes` and your
+`filter-allowlist` together.
 
-A packaged generator keeps reading the installed rules. If you own `routes` and change
+A packaged generator keeps reading the installed copy. If you own `routes` and change
 `emits_router` but still run the packaged `filter-allowlist`, the two disagree. Eject
 `filter-allowlist` too before you change that rule; the same goes for `entity` and
 `extractor` with `has_literal_default`.
 
+**What stays in the package.** The engine: the generator base, writer and formatter, the
+naming and type-mapping tables, the model walks and the loader. A table that decides a name
+or a type is a question about the model, and the generators, the docs and the runtime must
+agree on it, so it stays one packaged, public copy.
+
+**A copy you ejected before this release** keeps running. It imports the emitters from the
+paths they had (`metaobjects.codegen.fr010_field_mapping`,
+`metaobjects.codegen.generators.m2m_codegen` and the others in the CHANGELOG), and each of
+those is now a deprecated alias of the module in `metaobjects.codegen.runtime`, so your copy
+calls the packaged emitter exactly as before and a `DeprecationWarning` names the new path.
+`metaobjects gen --list` marks it `DIFFERS: N behind`. To own the emitters, eject the same name
+into a scratch directory (`metaobjects eject <name> --root <scratch>`), copy the `runtime/`
+files over into `codegen/runtime/`, and change your imports to the relative form the new copy
+uses (`from ..runtime import fr010_field_mapping as fm`).
+
+**An existing `shared_predicates.py` needs two new functions.** `routes` now reads
+`is_read_only_routed` from it, and `api-docs` reads that and `emits_writable_router`. A
+`codegen/runtime/shared_predicates.py` you already own does not have them, and eject keeps
+it, so `metaobjects eject api-docs` or `metaobjects eject routes --force` leaves a generator
+that fails to import (`cannot import name 'emits_writable_router'`). Eject marks the kept
+copy `DIFFERS: N behind` and prints how to bring it up: first the `diff -u` of your copy
+against the installed file, whose hunks you apply, then the `cp` of the installed file over
+yours, for a copy you have not changed. Eject does not decide which applies (the recipe under
+[The runtime your generated code imports comes with it](#the-runtime-your-generated-code-imports-comes-with-it)).
+
+### `api-docs`: the api reference as an owned generator
+
+`metaobjects docs` writes the Python api reference (`api/python`). `metaobjects eject
+api-docs` copies the generator that writes it, with the page model (`api_docs_model.py`), the
+builder that decides what to document (`api_docs_builder.py`), the renderer
+(`api_docs_renderer.py`) and the page-path math (`api_docs_paths.py`). Change which nodes
+get a page, how a page reads, or where it lands, by editing them.
+
+A docs generator is not a code generator, so it is wired in its own block, never in a
+target's `generators`:
+
+```yaml
+docs:
+  generators: [codegen.generators.api_docs:api_docs_generator]
+```
+
+`metaobjects docs` runs the copy in place of the packaged generator. Without the block it
+runs the packaged one, and the positional `metaobjects docs <metadata_dir>` form reads no
+config, so it always does. An entry that is not a docs generator, or a second one for the
+`api` surface, is refused by name. `gen` and `verify --codegen` skip a docs generator
+they find under a target and say so (`'api-docs' writes the api reference, so gen skipped
+it.`), with what to do about the entry: the packaged `api-docs` name is removed, because
+`metaobjects docs` already runs it, and an owned copy moves to `docs.generators`.
+
+**It describes the code your generators emit.** The page builder asks `shared_predicates`
+which objects get a router, the same module your owned `routes` and `filter-allowlist`
+read. Eject them together and edit the predicate once: the router, the allowlist and the
+reference change together. The naming seam (`metaobjects.apidocs.naming`) is the engine,
+the one set of names every generator and every page uses.
+
 ## Java and Kotlin: `mvn metaobjects:eject`
 
 ```bash

docs/ports/python.md

@@ -660,6 +660,16 @@ standard-library module name (`types`, `json`) is refused there with an error na
 rename it. See
 [Own your codegen → Python](../features/own-your-codegen.md#python-metaobjects-eject).
 
+Every generator `metaobjects gen --list` shows is ejectable except `template`, the Mustache
+primitive, which has no emit logic of its own to own. An ejected generator comes with the
+emitters it writes its output through (the shared-enum and M:N writers, the prompt tier's
+field mapping, extract delegate, extract schema and format-spec writers), copied into
+`codegen/runtime/` and read from there, so the copy owns the code that writes the file and
+not a shell around the package's. The naming and type-mapping tables, the model walks and
+the generator base stay in the package as the engine. `metaobjects eject api-docs` does the
+same for the api reference `metaobjects docs` writes: the copy is wired under
+`docs.generators` in `metaobjects.config.yaml`, never in a target's `generators`.
+
 Ejecting `routes` also hands over the helper runtime the generated routers call:
 `filter_parser` and `constraint_errors` are copied into `codegen/runtime/`, and the owned
 generator emits them into the generated package as `_runtime/` and imports them from