From 415d08dade147d778f7c8c901c37cefda61b486f Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 11 Oct 2026 21:38:15 -0400 Subject: [PATCH 01/12] docs(skills): bring shipped agent skills inside the skill-authoring limits Move sections of metaobjects-audit, metaobjects-authoring and metaobjects-fit-assessment into reference files so each SKILL.md is under 500 lines, with a row and an open-when condition per file in a top-of-file reference table. Add a contents list to every reference file over 100 lines, link the five TypeScript topic files and all language files from metaobjects-codegen, and put the reference table at the top of metaobjects-prompts, metaobjects-runtime-ui and metaobjects-audit. The hosted single-file assess prompt is now built by scripts/build-assess-prompt.mjs, which inlines the new references; it is gated in ci-local and equals the previous hosted text except for one dropped phrase. Fit-assessment's scaffold marker moves under metadata (no code reads it) and its maintainer notes move beside the design spec. The releasing skill gains disable-model-invocation, folds the Phase 0 addendum into Phase 0 and points to docs/RELEASING.md for the dated outage and credential notes. An SDK test holds the 500-line, contents-list and one-hop rules. Agent-context conformance fixtures are regenerated, not hand-edited. --- .claude/skills/releasing/SKILL.md | 55 +-- AGENTS.md | 2 +- CHANGELOG.md | 15 + agent-context/README.md | 2 +- .../skills/metaobjects-audit/SKILL.md | 422 ++---------------- .../references/calibration.md | 34 ++ .../references/capability-checklist.md | 16 + .../metaobjects-audit/references/csharp.md | 8 + .../references/drift-signatures.md | 57 +++ .../metaobjects-audit/references/java.md | 8 + .../metaobjects-audit/references/kotlin.md | 8 + .../references/prompt-anti-patterns.md | 44 ++ .../metaobjects-audit/references/python.md | 8 + .../references/requirements.md | 6 + .../references/surface-review-axes.md | 165 +++++++ .../references/typescript.md | 8 + .../references/vocabulary-hygiene.md | 98 ++++ .../skills/metaobjects-authoring/SKILL.md | 339 +------------- .../references/abstracts-and-overlay.md | 56 +++ .../references/choosing-a-shape.md | 5 + .../references/declaring-reports.md | 101 +++++ .../references/identities-and-indexes.md | 83 ++++ .../references/model-first-rationale.md | 22 + .../references/read-views-and-projections.md | 4 + .../references/reporting.md | 16 +- .../references/requirements.md | 18 + .../references/sources.md | 97 ++++ .../skills/metaobjects-codegen/SKILL.md | 8 +- .../references/any-stack.md | 8 + .../metaobjects-codegen/references/csharp.md | 10 + .../metaobjects-codegen/references/java.md | 11 + .../references/owning-generators.md | 7 + .../references/physical-names.md | 7 + .../metaobjects-codegen/references/python.md | 11 + .../references/typescript-docs.md | 8 + .../references/typescript-document-store.md | 4 + .../references/typescript-mysql.md | 6 + .../references/typescript-retargeting.md | 7 + .../metaobjects-fit-assessment/SKILL.md | 213 +-------- .../references/capability-sheet.md | 87 ++++ .../references/json-contract.md | 61 +++ .../skills/metaobjects-prompts/SKILL.md | 22 +- .../skills/metaobjects-runtime-ui/SKILL.md | 19 +- .../references/csharp.md | 7 + .../metaobjects-runtime-ui/references/java.md | 9 + .../references/kotlin.md | 7 + .../references/python.md | 7 + .../references/typescript.md | 10 + .../references/migration.md | 12 + .../references/requirements.md | 12 + docs/RELEASING-docs-checklist.md | 35 +- docs/RELEASING.md | 5 +- ...objects-fit-assessment-maintainer-notes.md | 76 ++++ .../.claude/skills/metaobjects-audit/SKILL.md | 422 ++---------------- .../references/calibration.md | 34 ++ .../references/capability-checklist.md | 16 + .../references/drift-signatures.md | 57 +++ .../metaobjects-audit/references/java.md | 8 + .../metaobjects-audit/references/kotlin.md | 8 + .../references/prompt-anti-patterns.md | 44 ++ .../references/surface-review-axes.md | 165 +++++++ .../references/vocabulary-hygiene.md | 98 ++++ .../skills/metaobjects-authoring/SKILL.md | 339 +------------- .../references/abstracts-and-overlay.md | 56 +++ .../references/choosing-a-shape.md | 5 + .../references/declaring-reports.md | 101 +++++ .../references/identities-and-indexes.md | 83 ++++ .../references/model-first-rationale.md | 22 + .../references/read-views-and-projections.md | 4 + .../references/reporting.md | 16 +- .../references/sources.md | 97 ++++ .../skills/metaobjects-codegen/SKILL.md | 8 +- .../references/any-stack.md | 8 + .../metaobjects-codegen/references/java.md | 11 + .../references/owning-generators.md | 7 + .../references/physical-names.md | 7 + .../skills/metaobjects-prompts/SKILL.md | 22 +- .../skills/metaobjects-runtime-ui/SKILL.md | 19 +- .../metaobjects-runtime-ui/references/java.md | 9 + .../references/kotlin.md | 7 + .../references/migration.md | 12 + .../.claude/skills/metaobjects-audit/SKILL.md | 422 ++---------------- .../references/calibration.md | 34 ++ .../references/capability-checklist.md | 16 + .../references/drift-signatures.md | 57 +++ .../metaobjects-audit/references/java.md | 8 + .../references/prompt-anti-patterns.md | 44 ++ .../references/surface-review-axes.md | 165 +++++++ .../references/vocabulary-hygiene.md | 98 ++++ .../skills/metaobjects-authoring/SKILL.md | 339 +------------- .../references/abstracts-and-overlay.md | 56 +++ .../references/choosing-a-shape.md | 5 + .../references/declaring-reports.md | 101 +++++ .../references/identities-and-indexes.md | 83 ++++ .../references/model-first-rationale.md | 22 + .../references/read-views-and-projections.md | 4 + .../references/reporting.md | 16 +- .../references/sources.md | 97 ++++ .../skills/metaobjects-codegen/SKILL.md | 8 +- .../references/any-stack.md | 8 + .../metaobjects-codegen/references/java.md | 11 + .../references/owning-generators.md | 7 + .../references/physical-names.md | 7 + .../skills/metaobjects-prompts/SKILL.md | 22 +- .../skills/metaobjects-runtime-ui/SKILL.md | 19 +- .../metaobjects-runtime-ui/references/java.md | 9 + .../references/migration.md | 12 + .../.claude/skills/metaobjects-audit/SKILL.md | 422 ++---------------- .../references/calibration.md | 34 ++ .../references/capability-checklist.md | 16 + .../references/drift-signatures.md | 57 +++ .../references/prompt-anti-patterns.md | 44 ++ .../metaobjects-audit/references/python.md | 8 + .../references/surface-review-axes.md | 165 +++++++ .../references/vocabulary-hygiene.md | 98 ++++ .../skills/metaobjects-authoring/SKILL.md | 339 +------------- .../references/abstracts-and-overlay.md | 56 +++ .../references/choosing-a-shape.md | 5 + .../references/declaring-reports.md | 101 +++++ .../references/identities-and-indexes.md | 83 ++++ .../references/model-first-rationale.md | 22 + .../references/read-views-and-projections.md | 4 + .../references/reporting.md | 16 +- .../references/sources.md | 97 ++++ .../skills/metaobjects-codegen/SKILL.md | 8 +- .../references/any-stack.md | 8 + .../references/owning-generators.md | 7 + .../references/physical-names.md | 7 + .../metaobjects-codegen/references/python.md | 11 + .../skills/metaobjects-prompts/SKILL.md | 22 +- .../skills/metaobjects-runtime-ui/SKILL.md | 19 +- .../references/python.md | 7 + .../references/migration.md | 12 + .../.claude/skills/metaobjects-audit/SKILL.md | 422 ++---------------- .../references/calibration.md | 34 ++ .../references/capability-checklist.md | 16 + .../references/drift-signatures.md | 57 +++ .../references/prompt-anti-patterns.md | 44 ++ .../references/surface-review-axes.md | 165 +++++++ .../references/typescript.md | 8 + .../references/vocabulary-hygiene.md | 98 ++++ .../skills/metaobjects-authoring/SKILL.md | 339 +------------- .../references/abstracts-and-overlay.md | 56 +++ .../references/choosing-a-shape.md | 5 + .../references/declaring-reports.md | 101 +++++ .../references/identities-and-indexes.md | 83 ++++ .../references/model-first-rationale.md | 22 + .../references/read-views-and-projections.md | 4 + .../references/reporting.md | 16 +- .../references/sources.md | 97 ++++ .../skills/metaobjects-codegen/SKILL.md | 8 +- .../references/any-stack.md | 8 + .../references/owning-generators.md | 7 + .../references/physical-names.md | 7 + .../references/typescript-docs.md | 8 + .../references/typescript-document-store.md | 4 + .../references/typescript-mysql.md | 6 + .../references/typescript-retargeting.md | 7 + .../skills/metaobjects-prompts/SKILL.md | 22 +- .../skills/metaobjects-runtime-ui/SKILL.md | 19 +- .../references/typescript.md | 10 + .../references/migration.md | 12 + .../.claude/skills/metaobjects-audit/SKILL.md | 422 ++---------------- .../references/calibration.md | 34 ++ .../references/capability-checklist.md | 16 + .../references/drift-signatures.md | 57 +++ .../references/prompt-anti-patterns.md | 44 ++ .../references/requirements.md | 6 + .../references/surface-review-axes.md | 165 +++++++ .../references/typescript.md | 8 + .../references/vocabulary-hygiene.md | 98 ++++ .../skills/metaobjects-authoring/SKILL.md | 339 +------------- .../references/abstracts-and-overlay.md | 56 +++ .../references/choosing-a-shape.md | 5 + .../references/declaring-reports.md | 101 +++++ .../references/identities-and-indexes.md | 83 ++++ .../references/model-first-rationale.md | 22 + .../references/read-views-and-projections.md | 4 + .../references/reporting.md | 16 +- .../references/requirements.md | 18 + .../references/sources.md | 97 ++++ .../skills/metaobjects-codegen/SKILL.md | 8 +- .../references/any-stack.md | 8 + .../references/owning-generators.md | 7 + .../references/physical-names.md | 7 + .../references/typescript-docs.md | 8 + .../references/typescript-document-store.md | 4 + .../references/typescript-mysql.md | 6 + .../references/typescript-retargeting.md | 7 + .../skills/metaobjects-prompts/SKILL.md | 22 +- .../skills/metaobjects-runtime-ui/SKILL.md | 19 +- .../references/typescript.md | 10 + .../references/migration.md | 12 + .../references/requirements.md | 12 + scripts/build-assess-prompt.mjs | 117 +++++ scripts/ci-local.sh | 9 + scripts/test-build-assess-prompt.mjs | 74 +++ .../agent-context/skill-structure.test.ts | 53 +++ 198 files changed, 6445 insertions(+), 4603 deletions(-) create mode 100644 agent-context/skills/metaobjects-audit/references/calibration.md create mode 100644 agent-context/skills/metaobjects-audit/references/drift-signatures.md create mode 100644 agent-context/skills/metaobjects-audit/references/prompt-anti-patterns.md create mode 100644 agent-context/skills/metaobjects-audit/references/surface-review-axes.md create mode 100644 agent-context/skills/metaobjects-audit/references/vocabulary-hygiene.md create mode 100644 agent-context/skills/metaobjects-authoring/references/abstracts-and-overlay.md create mode 100644 agent-context/skills/metaobjects-authoring/references/declaring-reports.md create mode 100644 agent-context/skills/metaobjects-authoring/references/identities-and-indexes.md create mode 100644 agent-context/skills/metaobjects-authoring/references/model-first-rationale.md create mode 100644 agent-context/skills/metaobjects-authoring/references/sources.md create mode 100644 agent-context/skills/metaobjects-fit-assessment/references/capability-sheet.md create mode 100644 agent-context/skills/metaobjects-fit-assessment/references/json-contract.md create mode 100644 docs/superpowers/specs/2026-07-12-metaobjects-fit-assessment-maintainer-notes.md create mode 100644 fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/calibration.md create mode 100644 fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md create mode 100644 fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md create mode 100644 fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md create mode 100644 fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md create mode 100644 fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/abstracts-and-overlay.md create mode 100644 fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/declaring-reports.md create mode 100644 fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/identities-and-indexes.md create mode 100644 fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/model-first-rationale.md create mode 100644 fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/sources.md create mode 100644 fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/calibration.md create mode 100644 fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md create mode 100644 fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md create mode 100644 fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md create mode 100644 fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md create mode 100644 fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/abstracts-and-overlay.md create mode 100644 fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/declaring-reports.md create mode 100644 fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/identities-and-indexes.md create mode 100644 fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/model-first-rationale.md create mode 100644 fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/sources.md create mode 100644 fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/calibration.md create mode 100644 fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md create mode 100644 fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md create mode 100644 fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md create mode 100644 fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md create mode 100644 fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/abstracts-and-overlay.md create mode 100644 fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/declaring-reports.md create mode 100644 fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/identities-and-indexes.md create mode 100644 fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/model-first-rationale.md create mode 100644 fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/sources.md create mode 100644 fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/calibration.md create mode 100644 fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md create mode 100644 fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md create mode 100644 fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md create mode 100644 fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md create mode 100644 fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/abstracts-and-overlay.md create mode 100644 fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/declaring-reports.md create mode 100644 fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/identities-and-indexes.md create mode 100644 fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/model-first-rationale.md create mode 100644 fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/sources.md create mode 100644 fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/calibration.md create mode 100644 fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md create mode 100644 fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md create mode 100644 fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md create mode 100644 fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md create mode 100644 fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/abstracts-and-overlay.md create mode 100644 fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/declaring-reports.md create mode 100644 fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/identities-and-indexes.md create mode 100644 fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/model-first-rationale.md create mode 100644 fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/sources.md create mode 100644 scripts/build-assess-prompt.mjs create mode 100644 scripts/test-build-assess-prompt.mjs create mode 100644 server/typescript/packages/sdk/test/agent-context/skill-structure.test.ts diff --git a/.claude/skills/releasing/SKILL.md b/.claude/skills/releasing/SKILL.md index eb1d69001..24a742c74 100644 --- a/.claude/skills/releasing/SKILL.md +++ b/.claude/skills/releasing/SKILL.md @@ -1,6 +1,7 @@ --- name: releasing description: Use when cutting or publishing a release of the @metaobjectsdev npm/TypeScript packages — e.g. "do a release", "release 0.x.y", bumping the published npm version, promoting merged work to npm latest, or refreshing version references after a release. +disable-model-invocation: true --- # Releasing the npm/TypeScript packages @@ -40,6 +41,15 @@ Do not bump or publish anything until every check is green. Print each result. | **Target version is free** | `npm view @metaobjectsdev/cli@ version` (a 404 = free); `git tag -l v` | the target version is NOT already published or tagged (npm versions are permanent — a taken version can never be reused) | | CHANGELOG ready | `sed -n '1,20p' CHANGELOG.md` | an entry exists or you will add one (see Phase 8) | | **`metamodelVersion` moved if the metamodel did** | `node scripts/check-metamodel-version.mjs` | exit 0. It diffs `expected-registry.json` against the last release tag and fails if the vocabulary changed without the version moving. Read its PROSE warning too — a rule can change with no machine-readable footprint (#210 changed only a `rules` string). Fix with `--set `; detail with `--explain`. | +| **You CAN publish** | `npm whoami` | prints your username (see the note under this table) | + +**Check that you CAN publish.** `scripts/release.mjs` runs `npm whoami` in preflight; do the same +by hand if you are publishing manually, and use an **Automation / bypass-2FA** token. A "Publish" +token passes `whoami`, shows `read-write`, passes `--dry-run`, and then `EOTP`s on every write, +at which point `bun publish` falls back to a browser flow that cannot complete non-interactively +(`--otp=` is ignored; a code must be piped on stdin). Verify BEFORE the version bump: the bump is +committed and pushed well before the first publish, so a dead credential strands the cut with +other registries already shipped. See `docs/RELEASING.md` → Prerequisites. If the code-review check fails (no PR, or unmerged/dirty tree): **stop** — releases ship reviewed, merged `main`, never a working tree. @@ -162,12 +172,9 @@ Add the `CHANGELOG.md` entry (Keep-a-Changelog format) if not already committed. `npm deprecate @-rc.1 "superseded; use "` for each package. -Retire the RC's `next` tag by **repointing, not deleting**: `npm dist-tag rm` 403s for every -token we hold (local and CI both — npm's bypass-2FA package-access rule, not a missing OTP). -The Actions → **npm dist-tag** dispatch that repointed the whole lockstep set from the -publish token **cannot run — GitHub Actions is disabled on this repository (2026-09-16), -and the replacement path is being decided**. Repoint per package from your machine instead: -`npm dist-tag add @ next` (`add` is not restricted). See `docs/RELEASING.md` §4. +Retire the RC's `next` tag by **repointing, not deleting**: `npm dist-tag rm` is refused +for every token we hold. Repoint per package with `npm dist-tag add @ next`, +or dispatch the npm dist-tag workflow for the whole set. See `docs/RELEASING.md` §4. ## Phase 10 — Propagate versions (docs + websites) @@ -211,24 +218,14 @@ version FAILS by design — skip what's done, never force. - About to `bun publish` to `latest` without human confirmation → stop. - About to push the site repo without confirming the live deploy → stop. -## OIDC Trusted Publishing — now DATED, not optional -npm removes **direct publishing from 2FA-bypass tokens around January 2027** -([changelog](https://github.blog/changelog/2026-07-31-restricting-npm-bypass-2fa-granular-access-tokens/)); -since 2026-07-31 such tokens already cannot change package access — including -**deleting a dist-tag**. Measured 2026-09-10: the local token and the CI `NPM_TOKEN` -both 403 on `DELETE …/dist-tags/next` and `--otp` changes neither, so this is a grant -boundary and no OTP session was going to clear it. Phase 9 repoints with `dist-tag add` -instead, which is not restricted. The replacement is **OIDC Trusted Publishing** via -`publish-npm.yml` (which exists — inoperative while GitHub Actions is disabled on this -repository, 2026-09-16), the same mechanism `publish-csharp.yml` already -uses for NuGet; the remaining work is a trusted-publisher registration per package -on npmjs.com. Caveat: OIDC can't drive `npm dist-tag` yet, so RC→promote still needs -a token. - -Separately, and still genuinely optional: **Changesets** (per-PR intent files → a -reviewable "Version Packages" PR → automated bump + CHANGELOG). Adopting it would -replace Phases 1/3/7's manual steps; the preflight gate, smoke test, and -stop-and-confirm in this skill stay. +## npm credentials and Trusted Publishing +The npm token type, the read-only auth preflight, and the dated plan to move publishing to +OIDC Trusted Publishing are in `docs/RELEASING.md` → Prerequisites. Read them before the +first publish of a cut. + +Separately, and optional: **Changesets** (per-PR intent files → a reviewable "Version +Packages" PR → automated bump + CHANGELOG). Adopting it would replace Phases 1/3/7's manual +steps; the preflight gate, smoke test, and stop-and-confirm in this skill stay. ## Verify with the committed script, not ad-hoc shell `node scripts/release-verify.mjs [--preflight] [] [--smoke]` is the one place @@ -238,13 +235,3 @@ instead of improvising checks per release: the 0.24.0 cut was verified by a doze throwaway scripts, and the improvised auth check was a **write** against a published package that could not be undone. **Never probe credentials by mutating a published package.** It is read-only; extend it there when a release needs a check it lacks. - -## Phase 0 addendum — check that you CAN publish -`scripts/release.mjs` now runs `npm whoami` in preflight. Do the same by hand if you -are publishing manually, and use an **Automation / bypass-2FA** token: a "Publish" -token passes `whoami`, shows `read-write`, passes `--dry-run`, and then `EOTP`s on -every write, at which point `bun publish` falls back to a browser flow that cannot -complete non-interactively (`--otp=` is ignored; a code must be piped on stdin). -Verify BEFORE the version bump — the bump is committed and pushed well before the -first publish, so a dead credential strands the cut with other registries already -shipped. See `docs/RELEASING.md` → Prerequisites. diff --git a/AGENTS.md b/AGENTS.md index a12005242..d1c578cb3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -16,7 +16,7 @@ The first four ship per-language today across the five ports (TS / C# / Java / P 2. **Runtime metadata** — load metadata at runtime, drive behavior dynamically (CRUD, validation, relationships, dynamic admin UIs, LLM tool registration). On Kysely (TS), a DB-API 2 driver via ObjectManager (Python), modernized JDBC + Spring-tx via OMDB (Java), Exposed (Kotlin). **C# runtime metadata is on the roadmap** — its EF Core output is generated code, not a metadata-driven runtime; the runtime `MetaObjects` package depends only on YamlDotNet and ships no data access. 3. **Drift detection** — `meta verify` catches divergence between code and metadata (covers entity codegen, prompt templates, output parsers, schema). Quality-of-life on top of codegen + runtime. 4. **Prompt construction** — a prompt is code, not a string scattered across services. Declare a prompt's payload as a typed projection (payload bloat becomes a diff), keep its text external and provider-resolved, and render it deterministically: snapshot-testable, cache-stable (no whitespace change silently breaking exact-prefix prompt-cache hits), and drift-checked at build time so a renamed field can't degrade a prompt. Conformance-gated, so the guarantee holds in every language port. **Render + payload-VO codegen + `verify` + parser-on-receipt for a *responding* `template.prompt` — one carrying `@responseRef` (FR-006) — + the output-format prompt fragment & tolerant `extract` parser (FR-010) ship in all five ports today** (since 0.24.0 the whole inbound tier keys off `@responseRef`; a `template.output` is outbound-only and emits no parser — ADR-0052) — the library-side building blocks of the pillar are complete. The one remaining library-side piece is MCP exposure of declared prompts/tools (see `spec/roadmap.md`); the application-level consolidation (eval harness, end-to-end declared-prompt orchestration) and consumer adoption are exercised in adopter projects, not in this library repo. Designed in `docs/superpowers/specs/2026-05-22-fr-004-cross-language-prompt-construction-design.md`. -5. **Requirements and testing** — declare what the software is supposed to *do* in the same model as the entities, so a capability claim is checkable instead of prose. The other four pillars keep the code honest about the *model*; this one asks whether the thing you said the software does is actually built — an absence no test can fail on, because a test exercises code that exists. `requirement.functional` (existence: `meta verify` WARNS when nothing implements it) and `requirement.architectural` (universality: `meta verify` FAILS a live policy applied to nothing, and does not check that each claimed node complies) are registered vocabulary in all five ports, with the loader enforcing the closed `@status` enum. `@implementedBy` is **resolved, not trusted** — it names a real member of the real model, so a claim whose implementation was renamed or deleted fails the build instead of going quietly stale. `meta verify` reports the ledger on every run (unresolved links, entities no claim covers, gaps recorded versus gaps nobody has ruled on) plus an authoring lint whose findings can never fail a build; `meta docs` renders it for humans and for agents. **The port split, stated exactly: the vocabulary loads and validates in all five ports; the requirement gate runs in every port's `verify` (ADR-0057 — `docs/CONFORMANCE.md` "Split coverage"); every port ships an ejectable `requirement-tests` generator — which scaffolds a test per claim; the seven authoring-lint advisories are TypeScript-only.** A green run proves referential integrity, never that a status is true or an implementation correct (`docs/features/requirements.md`, "What a green run does not prove"). A project that declares no `requirement.*` nodes sees no change at all. Note the standing carve-out: `agent-context/skills/metaobjects-fit-assessment/SKILL.md` deliberately does NOT treat `requirement.*` as an assessment axis — see the ruling in that file before "finishing the job" there. +5. **Requirements and testing** — declare what the software is supposed to *do* in the same model as the entities, so a capability claim is checkable instead of prose. The other four pillars keep the code honest about the *model*; this one asks whether the thing you said the software does is actually built — an absence no test can fail on, because a test exercises code that exists. `requirement.functional` (existence: `meta verify` WARNS when nothing implements it) and `requirement.architectural` (universality: `meta verify` FAILS a live policy applied to nothing, and does not check that each claimed node complies) are registered vocabulary in all five ports, with the loader enforcing the closed `@status` enum. `@implementedBy` is **resolved, not trusted** — it names a real member of the real model, so a claim whose implementation was renamed or deleted fails the build instead of going quietly stale. `meta verify` reports the ledger on every run (unresolved links, entities no claim covers, gaps recorded versus gaps nobody has ruled on) plus an authoring lint whose findings can never fail a build; `meta docs` renders it for humans and for agents. **The port split, stated exactly: the vocabulary loads and validates in all five ports; the requirement gate runs in every port's `verify` (ADR-0057 — `docs/CONFORMANCE.md` "Split coverage"); every port ships an ejectable `requirement-tests` generator — which scaffolds a test per claim; the seven authoring-lint advisories are TypeScript-only.** A green run proves referential integrity, never that a status is true or an implementation correct (`docs/features/requirements.md`, "What a green run does not prove"). A project that declares no `requirement.*` nodes sees no change at all. Note the standing carve-out: `agent-context/skills/metaobjects-fit-assessment/SKILL.md` deliberately does NOT treat `requirement.*` as an assessment axis — the ruling is in `docs/superpowers/specs/2026-07-12-metaobjects-fit-assessment-maintainer-notes.md`; read it before "finishing the job" there. 6. **Libraries** — reusable declared design, shipped as metadata and opted into by name (`"libraries": ["iam"]` in `.metaobjects/config.json`). Not a sixth verb: a library is the **reuse unit that composes the other five** — entities, requirements, the generators its design implies, its runtime packages — as one named, opt-in, drift-gated artifact. What makes it a pillar rather than a folder of YAML is the fifth: without requirements a library is a schema snippet; with them it is design an adopter's build is held to, which is the same test the requirements pillar passes (*does it change what an agent can be checked against?*). **A library is LAYERED and its core layer is INERT** — the core declares no `source.rdb`, and a sourceless object generates nothing and migrates to nothing (#248), so `["iam"]` adds zero tables and zero generated code while making the design present and resolvable; `["iam", "iam/db"]` is the separate opt-in that proposes the schema. **Copy is the expected mode** (`meta eject ` — ADR-0034's ruling applied to metadata), and an ejected copy still named in `libraries` is refused at load (`ERR_LIBRARY_PACKAGE_COLLISION`) rather than merging asymmetrically. Object coverage activates on ADOPTER-authored requirements only, so a library cannot volunteer a project for a gate it did not ask for. `iam` (preview) and `ai` (stable) ship today; rows appear in `meta gen --list` beside the generators. See [docs/features/libraries.md](docs/features/libraries.md) and `docs/superpowers/specs/2026-09-13-fr-043-feature-and-nfr-packages-design.md`. diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fea01d41..06e5d5191 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -799,6 +799,21 @@ until you regenerate. One rule is relaxed: an authored `validator.length @min: 0` on a `@required` string now admits the empty string, as the generated schema already did. +- **The shipped agent skills fit the skill-authoring limits.** `metaobjects-audit` (601 lines to + 261), `metaobjects-authoring` (804 to 479) and `metaobjects-fit-assessment` (666 to 481) moved + sections into `references/` files, each with a row and an "open it when" condition in the + skill's reference table, which now sits at the top of `metaobjects-audit`, + `metaobjects-prompts` and `metaobjects-runtime-ui` as well. The text moved and none was cut; `meta init` + and `meta init --refresh-docs` write the new files into `.claude/skills/`. Every reference + file over 100 lines opens with a `## Contents` list, and `metaobjects-codegen` links its five + TypeScript topic files and every language file from `SKILL.md`. A test in the SDK holds all + three rules. +- **The hosted fit-assessment prompt is built, not copied.** The capability sheet and the JSON + contract left `SKILL.md`, so `node scripts/build-assess-prompt.mjs --out ` writes the + single-file form served at `metaobjects.dev/assess.md`, inlining both. The output is the old + prompt, apart from one dropped phrase ("in this file"). Its maintainer notes moved to + `docs/superpowers/specs/2026-07-12-metaobjects-fit-assessment-maintainer-notes.md`, and the + skill's `scaffold: false` marker is now `metadata: { scaffold: false }` (no code reads it). ### Deprecated diff --git a/agent-context/README.md b/agent-context/README.md index 21762d702..c58135e9f 100644 --- a/agent-context/README.md +++ b/agent-context/README.md @@ -18,7 +18,7 @@ The **scaffolded** set is exactly the six skills in the SDK's `SKILL_NAMES` additionally holds **`metaobjects-fit-assessment`**, a *pre-adoption* tool that is deliberately **not** in `SKILL_NAMES` and so is never scaffolded (a not-yet-adopted target has no `.claude/skills/metaobjects-*` to receive it); it carries -`scaffold: false` in its front-matter as the human marker. Its design + retro-test +`scaffold: false` under `metadata:` in its front-matter as the human marker (no code reads it). Its design + retro-test validation live in `docs/superpowers/specs/2026-07-12-metaobjects-fit-assessment-design.md`. Design: `docs/superpowers/specs/2026-06-02-downstream-agent-context-design.md`. diff --git a/agent-context/skills/metaobjects-audit/SKILL.md b/agent-context/skills/metaobjects-audit/SKILL.md index 4d038bd0b..225276ad7 100644 --- a/agent-context/skills/metaobjects-audit/SKILL.md +++ b/agent-context/skills/metaobjects-audit/SKILL.md @@ -18,6 +18,37 @@ findings) + a rendered Markdown report. The audit **never edits code, never auth metadata** — `metadata_sketch` per finding is a read-only proposal for human review. Actual cutovers run through the existing skills mapped per finding tier (§ Bridge). +## Guardrails + +- **Adoption direction — metadata follows the code.** This is a brownfield project: existing + code and the live schema are the spec. Every `metadata_sketch` must **reproduce the code's + existing native types, names, and nullability** (model `field.uuid` where the code uses + `UUID`, carry over `@column`/`@table`/`@required`) and every cutover must **minimize churn to + code the generator is not replacing** — customize the codegen to match the existing shape + before proposing edits to working call sites. A sketch that would re-type or rename working + code the generator isn't replacing is modeling the wrong thing; when a choice is ambiguous, + flag it for the human rather than proposing the churnier option. (Full doctrine: + `metaobjects-authoring` → "Adopting onto an existing codebase".) +- **Parity-gate every cutover** — prove behavior-equivalent before deleting hand-written code; generated schemas are often looser. +- **Verify, don't assume** — read the code behind a grep hit. +- **Calibrate before flagging** — `references/calibration.md` lists what is never the adopter's fault. +- **Verify the DB artifact, not just the types** — the contract may claim a column the view DDL dropped. +- **Don't let one bespoke action block generating the entity** — generate CRUD; mount the custom action alongside. +- **Consumption ≠ a dist copy across versions** — bump + rebuild lockstep and install. + +## Reference files — open the one your task needs + +| File | Open it when | +|---|---| +| `references/calibration.md` | Before raising any finding: it lists the port gaps and non-defects that are never the adopter's fault. | +| `references/surface-review-axes.md` | Working any axis (A–I) of Phase 1b. | +| `references/capability-checklist.md` | On every axis: the exhaustive list of what the metamodel can express, one hunt per capability. | +| `references/drift-signatures.md` | Hunting hand-written second sources of truth (validators, mappers, comments, views, JSON bags). | +| `references/prompt-anti-patterns.md` | Axis D: the project builds text sent to an LLM or parses a reply. | +| `references/vocabulary-hygiene.md` | Axis I: retired or deprecated authoring forms, custom vocabulary, vocabulary opportunities. | +| `references/requirements.md` | Axis E2: the project declares `requirement.*` nodes. | +| `references/typescript.md`, `java.md`, `kotlin.md`, `csharp.md`, `python.md` | The server port the project uses (only the ports in the project's stack are installed). | + --- ## Phase 0 — Triage (fast, mechanical) @@ -76,253 +107,20 @@ lopsidedness. **Surface review — 8 axes (independently runnable).** Work the full `references/capability-checklist.md` on every axis. Check calibration -guards (§ Calibration) before raising a finding. **Verify, don't assume** — read the +guards (`references/calibration.md`) before raising a finding. **Verify, don't assume** — read the code behind a grep hit; a "duplicate" validator's *divergence* is the finding. -- [ ] **A. Codegen candidates — API / server routes.** Catalog + classify every handler. -- [ ] **B. Codegen candidates — web / client.** Pages, data layer (hooks, central fetch), - grids/forms/filters vs `layout.dataGrid` / form generators / filter-allowlist. -- [ ] **C. Drift hotspot — validators, mappers, runtime models.** Hand validators / DTO-mappers / - dataclasses shadowing a generated shape. Diff field-by-field; the divergence is the bug. -- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see § Prompt anti-patterns). -- [ ] **E. Owned generators & scaffold-and-own** (see § Owned-codegen assessment). -- [ ] **E2. Capability ledger — `requirement.*`** (see `references/requirements.md` and the - Requirement axis of the capability checklist). The one axis whose hunt is hand-written - PROSE rather than hand-written code: a `CAPABILITIES.md`, a README features table, a - conventions doc, a known-and-tolerated gap recorded as a TODO, or a rule that exists only - as a comment above the field it governs. Prose goes stale in silence; `@implementedBy` is - a reference the loader resolves, so a rename that invalidates the claim is caught. - **Entirely opt-in — an absent ledger is NOT a defect** (object coverage is a warning by - design, and a real estate reports most of itself unclaimed on day one). Score the prose - it would replace, and prefer ONE concrete claim an author can check over a - whole-estate migration. - **First establish the estate LOADS.** An older ledger fails the load outright, so - `meta verify` never runs and everything in `references/requirements.md` is unavailable: - `@violation` and `@verifiedBy` were retired in **0.24.0** (no deprecation shim — ADR-0023 - seals the registry); an index key declaring both `@fields` and `@expr` is refused since - **0.24.1**; and **0.24.2** turned `@status: abandoned | superseded` into **`retired`**. Run - **`meta upgrade`** (previews by default; `--apply` writes) before auditing, and audit the - upgraded tree. It rewrites only what has one correct answer and **refuses the rest, exiting - non-zero** — a refusal is itself a finding, naming the entries whose disposition nobody - recorded. - **Then audit the `retired` entries specifically.** They are where the ledger earns its - keep — agents proposing a retired capability's rebuild is the one failure this mechanism - has controlled evidence against — and they are also where it decays quietly. Check that - each `@statement` reads as a PROHIBITION rather than a diary entry ("X is never done", - not "we used to do X"), and that `@counterexample` describes the REVIVAL. A retired entry - whose statement narrates history is invisible to the reader it exists to stop. -- [ ] **F. Drift-gate adoption.** Is `meta verify` wired into CI / pre-commit? Which - subverbs (`--codegen` / `--templates` / `--db`)? Committed-codegen freshness gate? - Advisories heeded? Routine `--no-verify` bypass? Loader `ERR_*` / warnings addressed? - Parse the stable `code` field, not message text (ADR-0009). - **Is any advisory half switched OFF?** A gate can be wired, green, and muted. The - requirements authoring lint is advisory and mutable — `--no-requirement-lint` or - `META_NO_REQUIREMENT_LINT=1` silences it while the gate above it still runs and can still - exit 1, so a green pipeline says nothing about the seven `WARN_REQUIREMENT_*` authoring - checks. Grep CI config and scripts for both spellings. Muted is not automatically a defect — - a project that made a deliberate call is fine — but it moves those checks back onto the - auditor (see `references/requirements.md`), so **say which it is** rather than reporting the - ledger as lint-clean. - **Coverage completeness — "wired" is not "covers it all."** A gate can be present - yet blind to a whole artifact class. Confirm each subverb actually covers what the - project HAS: (a) **templates** — `verify --templates` on a CLI before the #193 fix - SKIPS `template.output @kind=email` and does not body-check a document - `template.output`'s mustache, so a `{{field}}` that drifted from the payload sails - through; a project with email/output templates that relies on `verify` alone (no - `meta gen` + `git diff --exit-code`, and no CLI past the fix) can ship broken — - **finding**; (b) **codegen** — committed generated code needs `--codegen` (or - gen+no-diff), else hand-edits/regeneration drift silently; (c) **schema** — `--db` - (Node `meta` only) for DB drift. Flag a gate that is WIRED BUT INCOMPLETE for the - artifacts present, and recommend the specific missing coverage. -- [ ] **F2. Over-generation — unnecessary generated files (codegen WASTE).** - Generated code is disposable, but that is NOT license to generate what nothing - consumes. Every generated file is a merge/review/maintenance surface and inflates - the leverage ratio falsely. Hunt OVER-generation (the inverse of A/B's - under-generation): - - **Dead generated files** — an `@generated` file that NOTHING imports or - references repo-wide (grep the emitted symbol / module path for importers): - delete it and stop generating it. - - **Artifacts an entity doesn't need** — REST routes / TanStack grids / forms / - hooks emitted for an entity that has no such surface. The remedy is the - generator's own `filter` option (`routesFile({ filter: (e) => e.name !== "Audit" })`, - `tanstackQuery({ filter })`), which is ANDed with the generator's built-in gates - and so can only NARROW what emits; a grid additionally needs a `layout.dataGrid` - on the entity at all. Flag the unused artifact and name the filter that would - suppress it. **Never recommend an `@emit*` attribute** — `@emitRoutes`, - `@emitTanstack`, `@emitForm`, `@emitGrid` and `@emitAngular` were never - registered vocabulary, so they passed `meta gen` (open load) and FAILED - `meta verify` (strict). A project carrying one is a **finding**, not an opt-out: - report it and route to `meta upgrade --apply` plus the generator config above. - - **Generators wired but unconsumed** — a generator in `metaobjects.config.ts` - `generators: [...]` (or the per-port equivalent) whose whole output class no - code imports: drop the generator rather than generate into the void. - - **Wrong target / duplicate output** — the same logical artifact emitted to two - places (a mis-set per-target `outDir`), one of which is orphaned. - Recommend generating ONLY what is consumed — decide per generator: drop the ones - whose whole output class nothing imports, and narrow the ones that over-emit with - their `filter`. A smaller, fully-consumed generated surface beats a large one with - dead files (which also make the leverage ratio lie; discount them from the census). -- [ ] **G. Runtime-contract anti-patterns.** Module-global `db` vs context-as-parameter - (ADR-0008); wire-canonicalization in the query path vs native in-process return types - (ADR-0019); runtime reflection to resolve a type from FQN vs generated static imports / - FQN registry (ADR-0001 / 0017); process-global registry vs per-loader (ADR-0014); code - that **mutates the loaded metadata tree** (read-only after load); JVM/Kotlin missing - startup validator; writes not routed to the `@role: primary` source; **`own*()` - accessor reads of effective properties / own-only member iteration (ADR-0039 — see - the active check below).** - -- [ ] **G2. `own*()` accessor discipline (ADR-0039 — CORRECTNESS DEFECT, not advisory).** - In any custom generator, metamodel provider, or runtime path (NOT the sanctioned cases - below), **flag every read of a field/node's effective property, or own-only member - iteration, done through an own-only accessor** — it silently drops everything inherited - via `extends` (a super-reference, not a flatten), corrupting codegen and runtime. This - is exactly the class of bug that broke Kotlin's array-type derivation (a concrete field - inheriting an array flag from an abstract parent generated a scalar) and, per the audit, - is latent cross-port. Grep for the own-only accessors and verify each hit: - - **TS:** `ownAttr(`, `ownChildren(`, `ownFields(`, a raw `isArray` field flag read → - should be `attr(` / `children()` / `fields()` unless emitting a subclass's own members. - - **Python:** `own_children(`, `own_fields(`, and the **inverted** bare `attr(` (Python - `attr()` is OWN; the resolving form is `attrs().get(`) → flag `attr(` used to read an - effective value. - - **Java / Kotlin:** `getMetaAttr(name, false)` (the `,false` own overload), own-only - child walks (e.g. an own-only `filterIsInstance<…>()` source lookup that emits nothing - for an entity inheriting its source). - - **C#:** a native `IsArray` flag read, `OwnChildren()`, own attr reads. - - **Sanctioned (do NOT flag):** (a) a generator emitting a generated **subclass** that - iterates `ownFields()` so inherited members aren't re-emitted (the generated base - declares them — the `class Sub extends Base` / TPH pattern); (b) the own-mode canonical - serializer + overlay-merge + super-resolution walks (library-internal); (c) the single - deliberately-own attribute `@dbColumnType` (a physical column-type override, never - inherited). Any own read that carries a comment naming one of these cases is fine; an - uncommented own read of an effective property is the defect. -- [ ] **H. Authoring-correctness / ADR-conformance (deep).** Invented/unregistered - `@`-attrs or post-bootstrap registration (ADR-0023 — custom attrs belong in a registered - provider or `attr.properties`); retired source-v2 forms (`source.dbTable` / `@name` / - `@dbColumn` → use `source.rdb` + `@kind` + `@table` / `@column` + `@role`, ADR-0007/0018); - taxonomy impurity (entity over read-only primary source; read model that should be - `object.projection`; `value` carrying identity/source, ADR-0028); copy-pasted base-field - blocks — or the same-named field re-declared across objects (drift signature 10) — instead of - abstract + `extends`; `@`-prefixed YAML keys (ADR-0006); relative refs - in committed canonical JSON (ADR-0032); DB-type-as-logical-subtype (ADR-0013); per-port - migration engine where schema is Node-`meta`-owned (ADR-0015). - -- [ ] **H2. Wrong native type — `field.` + `@dbColumnType` that hides the real type - (CORRECTNESS-ADJACENT finding, NOT advisory axis-I).** The headline instance is a **UUID - column modeled `field.string` + `@dbColumnType: uuid`**: the DB column is uuid but the - generated property is a **`String`**, so the code coerces `String↔UUID` at every boundary - and the native type is wrong everywhere the field is used. `verify --db` **cannot** catch it - (the column type matches), so it hides in plain sight. When it sits in a shared - `BaseEntity`/`BaseAuditedEntity`, **every inheriting `id`/`tenantId`/FK is wrong** — count the - blast radius (grep every `field.string` paired with `@dbColumnType: uuid`; it is often - hundreds of fields). This is a **real finding**, not a modernization nudge: recommend - `field.uuid` and flag it as a **staged migration** (re-typing `id`/FK ripples through - repositories, finders, and call sites) — tier by blast radius, not buried as advisory. The - ONLY non-finding is a field the code genuinely handles as a *string* over a uuid column - (explicitly justified). Report the total pair count so the migration has a completion - criterion (see the CI ratchet gate in `metaobjects-verify`). - -- [ ] **I. Vocabulary hygiene / modernization (ADVISORY).** Flag already-retired or - deprecated authoring patterns and recommend the canonical form (see § Vocabulary - hygiene). Advisory severity — scored as modernization opportunities, **never a - failing finding**. +Axes (the full hunt for each is in `references/surface-review-axes.md` — open it before working an axis): -**Phase 4 — Synthesize** into the tiered roadmap and populate both artifacts. - ---- +- **A.** Codegen candidates — API / server routes · **B.** Codegen candidates — web / client +- **C.** Drift hotspot — validators, mappers, runtime models · **D.** Prompt pillar +- **E.** Owned generators and scaffold-and-own · **E2.** Capability ledger (`requirement.*`, opt-in) +- **F.** Drift-gate adoption · **F2.** Over-generation (codegen waste) +- **G.** Runtime-contract anti-patterns · **G2.** `own*()` accessor discipline (a correctness defect, not advisory) +- **H.** Authoring correctness / ADR conformance · **H2.** Wrong native type (`field.string` + `@dbColumnType: uuid`) +- **I.** Vocabulary hygiene / modernization (advisory, never a failing finding) -## Vocabulary hygiene / modernization (axis I — ADVISORY) - -Per ADR-0037, vocabulary expansion follows ONE ordered test (derivable → derive; -physical-only → `@dbColumnType`; logical: different native type → subtype, same kind + -modifier → attribute). This axis surfaces authoring that predates or contradicts that -framework. **All findings here are advisory** — modernization opportunities scored as -such, surfaced in the roadmap, but **non-failing** (the code works; the form is dated). - -**Already-retired / deprecated forms → recommend the canonical form:** - -- `@dbColumnType: uuid_array` / `@dbColumnType: text_array` (a physical array column - type) → **`isArray: true`** on the base subtype. Array-ness is logical and derivable; - the array column type is retired. -- The `@kind: text` hack (forcing text via a kind override) → **bare `field.string`** - (text is the default; no override needed). -- `@dbColumnType: uuid_array` was covered above. **`field.string` + `@dbColumnType: uuid` - is NOT advisory — it is a real mismodeling finding (see axis H).** It generates a `String` - where the code uses/wants a native `UUID`, forcing `String↔UUID` coercions at every - boundary; `verify --db` passes (the column really is uuid), so the schema gate can't see it. - The genuine string-over-uuid-column case (code truly handles the value as text) is the ONE - legitimate use and must be explicitly justified — otherwise recommend **`field.uuid`**. -- `@dbColumnType: timestamp_with_tz` (ADR-0036 Wave 2) → **drop it.** `field.timestamp` is - instant / timezone-aware **by default** now; the `timestamp_with_tz` column-type override - is **retired**. Timezone-awareness lives in `field.timestamp` + the `@localTime` opt-out. -- A bare `field.timestamp` that is **semantically a wall-clock value** (a store-open time, a - birthday-with-time, a recurring local schedule) → recommend **`@localTime: true`** (naive - `timestamp without time zone`), or confirm it is genuinely meant to be an instant. Default - `field.timestamp` is an instant; only flag when the field's meaning is clearly wall-clock. -- A `validator.regex` (or a plain `field.string`) validating an **email** shape → recommend - **`@stringFormat: email`** on the `field.string` (ADR-0036 Wave 3). The native type stays - `string`; the per-port codegen emits the idiomatic email check — don't hand-roll the regex. -- A `field.string` that **holds a URL / URI** → recommend **`field.uri`** (native `URI`/`Uri`, - URL validation) — a distinct native type + behavior is a subtype, not a validated string. -- A `field.string` that **holds an IP address** → recommend **`field.inet`** (native IP type; - Postgres `inet` column). - -**Custom-provider vocabulary (adopters who register their own types/attrs):** check -new/custom vocab against the ADR-0037 procedure (advisory) — e.g. *a custom subtype -that differs from an existing one only by a property should be an attribute, not a -subtype*; an email/hostname string-validation is an **attribute** (`@stringFormat`, -native type unchanged), while a URL or IP is a **native type** (`field.uri` / -`field.inet`, a subtype). Recommend re-shaping against the ordered test. - -**Hand-rolled reverse-query repository methods (ADR-0038) → recommend the generated reverse -FK finder.** Reverse navigation — *"find all the rows that reference this one"* — is now -codegen. Flag a hand-written reverse-query method (a `findByParentId` / `findBy` -repository finder, or a manual `WHERE fk = ?` query helper — exactly what a JVM adopter -hand-writes in its `SceneRepository`) and recommend the **generated reverse FK finder** -instead: codegen now emits `findBy` plus a batched `…In(ids)` variant from -the FK metadata, idiomatic per port (Spring repository finder / EF query method / Python or -TS query function). It is **performant** (one indexed query, no N+1) and **framework-free** -(no lazy collections / proxies). When an entity has two FKs to the same target, the codegen -emits two distinct finders (named by the FK field) automatically — no annotation needed; the -reverse finder is a **codegen feature, not an attribute** (there is no reverse-nav `@`-attr -to author). Advisory severity — a modernization opportunity, not a failing finding. - -**New-vocabulary OPPORTUNITY (the inverse hunt — advisory).** The checks above audit -custom vocabulary the adopter *already* registered; this one hunts where the app -**should** register vocabulary but hand-coded the pattern instead. Smells -(grep-then-verify; require a *recurring, closed* set — never flag a one-off): -- **N parallel hand-written integration modules** sharing a payload shape and a config - pattern (a closed set of channels / providers / export targets, each a near-copy with - a different transport) → candidate for ONE project-registered subtype whose variants - sit behind a closed structural-variant discriminator (the `source.rdb` `@kind` - pattern), plus a small owned generator emitting the per-variant wiring. -- **An ad-hoc string discriminator steering code switches** — a string column whose - values select per-variant behavior/config scattered across the codebase. If the values - are just a closed symbol set, that is `field.enum` (`@values`), NOT new vocabulary. It - earns a subtype only when the discriminated concept **owns behavior or attributes of - its own** (ADR-0037 step 2a). -- **A provider-shaped pattern re-implemented per instance** — repeated registration / - config blocks restating a shape one declared node per instance could carry, with codegen - emitting the repetition. -- **A downstream integration modeled entirely in code** (an outbound webhook / notifier / - queue publisher / tool wiring beyond `template.toolcall`) where registered vocabulary + - a small generator would own the payload wiring, the names-only fail-closed config check, - and drift detection. - -Run the **ADR-0037 ordered test** before proposing: derivable → derive; differs from an -existing subtype only by a property → an attribute; a plain validated value → an -attribute; one-off author-supplied properties → the `attr.properties` bag. Only a concept -with its own behavior/attributes earns a subtype, and transport/protocol is never the -subtype axis (keep the node protocol- and address-free; variants behind the discriminator). -**Converge before inventing** — check shipped vocabulary AND the planned/chartered names -(see CALIBRATION "planned, not shipped"); never claim a chartered name for a project-local -type. **Recommendation:** propose a project-registered provider (explicit loader wiring, -ADR-0023 — never a loosened `strict` free-ride), a `metadata_sketch` of the node, and the -owned generator that retires the duplication; a second independent consumer needing the -same concept is a consumer→core promotion candidate (ADR-0011 — file an upstream issue). -**Verdict: VOCAB CANDIDATE (advisory)** — a modeling opportunity, never a failing finding, -never a tier gate; bias to under-flagging (the >15% false-positive kill criterion applies -with full force). Full guidance: `docs/features/downstream-metadata-decisions.md`. +**Phase 4 — Synthesize** into the tiered roadmap and populate both artifacts. --- @@ -346,52 +144,6 @@ hidden candidate. **Stub trap:** demo-data routes have nothing to replace — cl --- -## Drift signatures (highest-value; grep-then-verify) - -Per finding: `file:line` → what → generated-equivalent exists? → recommendation. - -1. **Hand validators shadowing a generated schema** — diff field-by-field; divergence is the bug. -2. **Field-by-field serialize / deserialize / DTO↔model / row mappers** — silently drops a field when metadata grows one. -3. **camelCase↔snake_case / body↔column maps** maintained beside a generated view that already renames. -4. **Drift-admitting comments** — grep: `"keep in sync with"` / `"mirrors the"` / `"matching the"`. -5. **Runtime schema patching** (`ALTER TABLE … ADD COLUMN IF NOT EXISTS`, `_ensure_schema()`) — N schema owners. -6. **N declarations of one shape** — same entity as Drizzle table + Zod schema + Pydantic model + hand dataclass; target is 1 + N generated. -7. **`own*()` accessor read of an effective property** (ADR-0039) — `ownAttr` / `ownFields` / `own_children` / bare Python `attr(` / `getMetaAttr(name, false)` / native `IsArray` used to read a value or iterate members outside the sanctioned subclass-emit / own-serializer / `@dbColumnType` cases → silently drops `extends`-inherited values. A **correctness defect** (axis G2), not advisory. -8. **Hand-written `CREATE VIEW` / read-only SQL standing in for a projection or entity read-view (view-necessity test).** Grep migrations, checked-in `.sql`, and repository/query code for `CREATE [OR REPLACE] [MATERIALIZED] VIEW`, and for hand-rolled read-only queries that mirror a read model — a pure-`SELECT` repository/service method with joins or `GROUP BY` feeding a DTO, or a raw-SQL escape (`db.execute(sql…)`, `FromSqlRaw`, a JPA @Query with hand-written SQL). For each, run the **necessity test** — can origins express this shape? A column is derivable when it is (a) a base-entity or relationship-joined column → `origin.passthrough` (`@from` / `@via`), (b) a count/sum/avg/min/max over related rows → `origin.aggregate` (`@agg` / `@of` / `@via`), or an `EXISTS` / `array_agg` → `origin.aggregate` `any` / `all` / `collect` — any of them optionally row-scoped with `@filter`, (d) a computed scalar / **non-aggregate expression column** → `origin.computed` (`@expr`, #195), (e) one related row's column picked by an ordering — an **argmax / `DISTINCT ON … ORDER BY` / correlated `ORDER BY … LIMIT 1`** → `origin.first` (`@of` / `@via` / `@orderBy`, #195), or (f) a column borrowed via `extends` — and the joins follow declared relationships / `identity.reference` FKs. - - **Entity-shaped (`SELECT own.* + derived`) → an entity read-view, NOT a projection (#214).** The single most common legacy view is an entity's OWN columns plus a joined/derived extra (`SELECT o.*, c.name AS customer_name`). This is the `Order` **entity with a read route**, not an exposure contract — route it to an **entity read-view**: keep the writable `@role: primary` `@kind: table` source and add a **non-primary `@role: replica` `@kind: view`** source, declaring only the *extra* as a derived `origin.*` field on the entity (the own field set already covers `o.*`). Reads route to the replica view, writes to the table. Reach for a **projection** instead only when the view renames base columns or **row-filters** (`WHERE status='active'`, soft-delete) — the latter is a projection with an object-level `@filter` (#207) that lowers to the outer `WHERE`. - - **Exposure contract, expressible → CODEGEN CANDIDATE (high):** a subset / renamed / versioned / multi-base read model → convert to an `object.projection` with a read-only `source.rdb` `@kind: view` child, let `meta migrate` emit the `CREATE VIEW`, and consume the generated read-only query — the hand-written view is a second source of truth for a derivable shape. Parity-gate: the generated view returns row-identical results before the hand-written SQL is deleted. - - **Not expressible → carry it in `@sql` or `@unmanaged`, never a hand-edited migration (#208, ADR-0043).** When a NAMED irreducible construct blocks origin authoring — recursive CTE, window function / `OVER`, `UNION` / `INTERSECT` / `EXCEPT`, lateral join — the body still belongs in the metadata: carry the hand-written SQL in the `source.rdb` **`@sql`** escape — a read-only-`@kind` body the tool REGISTERS, fingerprints, and drift-checks (adopt a pre-existing view with `meta migrate --allow adopt-view`); `@sql` forbids `origin.*` children (two sources of truth). A DB object whose DDL is owned **entirely elsewhere** (Flyway / a hand-migration) → mark its source **`@unmanaged: true`** (legal on any `@kind` incl. `table`); `meta migrate` never creates/drops/drift-checks it and `verify --db` reports it as external. `@sql` and `@unmanaged` are mutually exclusive. **Only a view left *undeclared*** — neither modeled, nor `@sql`, nor `@unmanaged` — is truly *unmanaged*, invisible to `meta verify --db`, so this audit is the only gate that sees it. "It's an aggregation" is NOT an irreducibility justification (plain count/sum/avg/min/max rollups are `origin.aggregate`); nor is a `DISTINCT ON` pick-one-row (`origin.first`) or a non-aggregate expression column (`origin.computed`). -9. **A closed variant-set hand-modeled per instance** — N sibling modules / classes / config blocks, one per channel / provider / target, sharing a payload + config shape and diverging only by transport. Grep for sibling-file families and switch-on-a-string dispatch; verify the set is closed and recurring (never a one-off). → axis I "New-vocabulary OPPORTUNITY" (VOCAB CANDIDATE, advisory). -10. **N declarations of one FIELD across objects (same-name-field census)** — the field-level sibling of signature 6. Census field names across `object.*` nodes (`grep -rn 'name: '` the metadata dir); a name recurring in ≥2 objects where a canonical owner exists — one whose name the field embeds (``: `wizardId` → `Wizard.id`, `orderTotal` → `Order.total`) or whose type+constraints it matches — is provenance loss → `extends: Owner.field` (dotted child targets, ADR-0029). **VERIFY by diffing the copies' attrs: a `@maxLength` / `@required` / validator divergence across them is drift already shipping — cite it.** Evidence multiplier: `extends` already used elsewhere in the repo raises confidence. Do NOT flag: generic names on unrelated concepts (`id` / `name` / `status` with no owner-embedding name and no matching constraints); required per-node attrs a loader forces (e.g. `payloadRef` / `format` on sibling `template.prompt` nodes — a product constraint, not a copy). -11. **A metadata-derived value spelled a second time by hand.** A declaration is the single source of truth for every value the loader reads from it and every value a generator derives from it — the logical name, the physical table / schema / column / index names after the naming strategy, the resource path, an enum's member symbols, a label from `@title`, a limit from a validator child. Anywhere hand-written code or config carries a literal that must EQUAL one of those values, that literal is a second spelling, and a second spelling is drift waiting for the first one to move. **The three shapes this takes most often are raw SQL (a physical name), an API call or route mount (the resource path), and a hand-built form (a label, a limit, a message) — illustrative, never the set.** An adopter's other libraries carry the same defect in shapes this skill cannot list — a GraphQL resolver map, a search-index field, a cache or feature-flag key, a queue topic, a CSV `COPY` header, a form library's registration string, an LLM tool-parameter name — and the hunt has to reach those from the rule, because no menu will. Two generated surfaces hold the constants a site should reference instead: the per-object physical-names artifact (`.names.ts` / `Names` / `_names.py` — table, schema, `fields[].column`, identity and index names) on all five ports; and on TypeScript the entity module's constants with their browser-safe `.meta.ts` twin (`$entity`, `$table`, `$path`, and per field `name` / `label` / `view` / `htmlType` / `rules` / `options`), whose own docblock says to use them instead of magic strings. The other four ports answer that second surface with a generated typed handle — a Pydantic attribute, an EF property, an Exposed `Column` — which is already the gold standard. - - **The discriminator is what happens when the declaration changes, not whether the string looks like a name.** Rename the thing in metadata and regenerate. A site is this signature only when it SURVIVES the regeneration and goes wrong later — on the request the router matches, the query the database parses, the row a reader mis-keys, the export a human mis-reads — or never goes wrong and simply lies. A site the regeneration itself breaks before anything runs is already gated and is NOT a finding. The mechanism behind the two outcomes is whether the hand-written spelling is RESOLVED at the site or COMPARED by a consumer the value travels to: an identifier on a typed handle, a value narrowed by a generated union, a key of a typed filter object are resolved — by the compiler where there is one, by import or attribute resolution of that line where there is not — and the toolchain names the site; a string a database parses, a router matches, a JSON consumer looks up, a broker routes or a human reads is compared, and the failure surfaces in the consumer, on the data, or nowhere. On TypeScript "resolved" means the compiler checks it, and that one test turned an estate's 332 bare enum-member strings (`status === "edited_approved"`) into 0 findings — each is narrowed by the generated `export type = "a" | "b"`, so renaming the member breaks the build at every site. The physical-name case is the opposite pole and the reason "fails loudly" cannot be the line: a renamed column DOES fail loudly — in production, on that query, from the database — which is exactly the failure this signature exists to prevent. The gate that counts fires from the regeneration alone. - - **Hunt from the declaration outward, never from code shapes inward.** (1) Build the value list from the generated artifacts, because they already carry every derived value after every transform — naming strategy, pluralized path, humanized label: read `Names` and, on TypeScript, the `.meta.ts` twin. Where the port emits no names artifact yet, derive the list from the metadata directly (`@table` / `@schema` / `@column`, else the naming strategy applied to the field name; `@values`; `@title`) and record the un-wired generator as the first finding (below). (2) Grep the non-generated tree once — code AND checked-in config, fixtures, YAML, JSON, `.env`, OpenAPI fragments — with an alternation of those VALUES as literals; distinctive values (`"email_address"`, `"/subscribers"`, `"edited_approved"`) score on the hit, generic ones (`"id"`, `"name"`, `255`) only by reading the consumer, never by count. This step is what finds the topic name, the index field, the `COPY` header and the flag key without knowing they exist. (3) Trace each hit to its CONSUMER and ask what that consumer matches it against. Matched against the schema, the generated route table, a generated wire shape or a generated label → the metadata owns it, apply the discriminator. Matched against something the app chose — its own page-route table, its own copy deck — → not owned, not this finding. (4) Only then run the shape greps — raw SQL, `fetch(` / `app.get(` / a `queryKey`, a hand validator — as the second pass, for what a value grep cannot see: a literal that has ALREADY drifted from the declared value, a concatenated or interpolated name, a case-transformed one. - - **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a flattened value-object composite column belongs to no single field of either object; a relationship-synthesized FK column is derived, never declared; a write-through entity's replica view name is not in the artifact, which holds the PRIMARY source's. A hand-written repository that imports `Names` is the gold standard — call it out as good. - - **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. - - **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on ALL FIVE ports an existing project emits none until it asks**. ADR-0034 Amendment 2 made codegen opt-in everywhere: C# and Python require `--generators`, TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, and `meta init` scaffolds `generators: []` — so no port begins emitting this artifact merely because the package was upgraded. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. - - **Report TWO numbers for this signature, always, and never just the first: (a) is the artifact EMITTED, and (b) how many of the literal sites actually IMPORT it — the adoption ratio, as `/ sites`.** Emission is the cheap half and it is the half that gets done: measured on an adopter whose audit proved the artifact emitted correctly, ran the generator to show it, and then adopted it at **0 of 53 sites** (37 table + 16 column). Wiring the generator changes the scorecard; it changes nothing a reader of the code experiences, because the second spelling is still the one every query uses. An audit that reports only (a) makes the estate look upgraded while every literal it found is still there — so a project that emits the artifact and imports it nowhere scores WORSE than one that has not started, not better, because it now carries a third spelling that claims to be the source of truth. - - **A deferral must carry its measurement forward.** "Wire the names tier" is a real thing to postpone — on the observed estate it meant a new generated tier, a new dev-loop step, a new `verify --codegen` gate and edits to 4 service and 6 test modules that could only be validated against a live database, and the branch in hand deliberately changed nothing in the running application. That is a legitimate decision and belongs in the report. But record it with **the site count, the exact command that emits the artifact, and the modules that would change** — never as intent alone — and **re-raise it in every later audit until the ratio moves**. A deferral with no number attached reads as a nice-to-have on the next reading and silently becomes permanent; the one carrying `0/53` does not. - - -12. **An "open JSON bag" whose readers all agree on a shape.** `field.string` + `@dbColumnType: jsonb` is SANCTIONED vocabulary, not a defect — it is the deliberate escape hatch for a genuinely untyped JSON column with no value object to reference, it emits `unknown` / `z.unknown()` by design, and a cross-port corpus (`fixtures/api-contract-conformance/jsonb/`) gates that contract. **Never flag the pattern on sight.** Flag it only when the bag is not actually open: when the code demonstrably knows the shape. **Look at the WRITER FIRST — it is the strongest and cheapest evidence, and a reader-only hunt badly under-counts.** The function that writes the column declares the value's type in its own signature: `related_memory_ids: list[str]`, `vector_scores: list[float]`, `tools_offered: list[dict[str, Any]]`, `preferences: dict[str, Preference]`, a serialized dataclass / DTO / `@Serializable` class. If the writer's type is anything but a genuinely open map, the metadata is claiming less than the code already knows. Reader-side tells come second: a FIXED KEY SET in raw-SQL JSON-path reads (`->>'decision'`, `#>>'{a,b}'`), casts to a concrete type (`row.output as { draftText?: string }`), destructuring, typed parsing (`model_validate`, `readValue(…, X.class)`, `decodeFromString`), or a hand-written validator over the parsed value. Measured: hunting reader-side raw SQL ALONE found 2 columns on an estate; adding the writer signatures found many more on the same estate, because raw-SQL JSON paths are the rarest way to consume a jsonb column — most consumers parse it into a type. - - **The `isArray` sub-case is the commonest and is mechanical.** A bag whose writer types it `list[str]` / `string[]` / `List` is not an object and not an open bag: it is the BASE SUBTYPE plus `isArray: true` (`field.string @isArray`, `field.double @isArray`). Declaring it as an open jsonb bag loses the element type entirely, so nothing stops a writer putting a map where every reader expects a list. A plural field name (`scopes`, `relatedMemoryIds`, `participantEmails`, `capabilities`) plus a typed-collection writer is the whole test. A `*Json` SUFFIX is a second free tell — a field named `conditionJson` / `configJson` / `definitionJson` is admitting it carries a structured shape. At that point the shape exists; it is simply declared in N readers instead of once in the metadata, which is drift signature 6 with zero declarations rather than several — and it is invisible to every gate, because `verify --db` sees a jsonb column that matches and `--codegen` sees generated files that are correct for an open bag. - - **Measured on one estate:** 6 such columns, and one of them (`AuditEntry.output`) carried **12 distinct keys across 31 raw-SQL path reads plus 7 casts** in 10 files. Nothing in that is open. Estate-wide the pattern is common — 74 `field.string` + jsonb fields across five of nine projects — which is exactly why the discriminator matters more than the count: report the ones whose readers pin a shape, not the ones whose readers treat the value as opaque. - - **Do NOT flag:** a bag whose readers genuinely pass it through (logged, echoed, stored, forwarded to a third party) without reading a key; a third-party or LLM raw response whose shape is not the adopter's to declare; an **array of scalars**, which is `isArray: true` on the base subtype and never an object; a bag the metadata's own comment justifies as open with a reason. - - **Remedy — a ladder, chosen by the WRITER's declared type, and every rung gives the consumer a TYPED HANDLE rather than a string to get and set.** `list[X]` ⇒ the base subtype + `isArray: true`. `dict[str, X]` with a known `X` ⇒ **`field.map` + `@objectRef`** (a value object) or `@valueType` (a scalar) — registered vocabulary that emits `jsonb("col").$type>()` and `z.record(z.string(), X)`. **Check the CONSUMER before recommending this rung: `field.map` codegen now completes on all five ports** (Java types it `java.util.Map`; C# emits the `Dictionary` property and its EF jsonb storage mapping), **but no persistence- or api-contract-conformance fixture exercises `field.map` on ANY port, and the runtime persistence tier is uneven** — only Python's `ObjectManager` encodes a map, while `runtime-ts`, Java's OMDB and the Kotlin Exposed lane carry none. So where GENERATED CODE is the consumer, check the port before recommending this rung — nested map values are validated on every write path on every port as of 1.0.1 ([#362](https://github.com/metaobjectsdev/metaobjects/issues/362) — before that, Java skipped the cascade on TPH write paths and C# skipped it everywhere, silently accepting invalid values rather than erroring). **Check the adopter's pinned version before recommending this rung to a Java-TPH or C# consumer: on 1.0.0 the hole is open and they need their own boundary check.** Prefer a value object where a PORT RUNTIME must read the column back; a genuinely dynamic key set stays a bag. A serialized DTO / dataclass / `@Serializable` class ⇒ declare it as an `object.value` and point the field at it with `field.object` + `@objectRef` + `@storage: jsonb`. Only `dict[str, Any]` whose readers pin no key stays an open bag. **What the adopter GETS, per port** — this is the point of the conversion and worth stating in the finding: TypeScript emits the Drizzle column as `.$type()` plus the VO's own Zod schema and inferred type; Python annotates the field with the referenced VO's Pydantic model (and its `Create` on the wire, so a nested violation still validates); Kotlin emits typed Exposed jsonb codecs backed by a shared Jackson `MetaJsonbMapper`. In none of them does the consumer touch a JSON string — that is what the open bag costs them today, since `unknown` forces every reader to cast. The column stays jsonb, so this is a codegen/contract change and not necessarily a migration; what changes is that `unknown` becomes a generated type, the casts delete, and the keys become metadata-owned — which also promotes their literals into drift signature 11's scope, where the `->>'key'` sites were previously exempt because the metadata owned nothing. **Verify the column type is genuinely unchanged before promising no migration**, and parity-gate each converted read against the raw-SQL one before deleting it. Convert one column at a time: the shape is a judgment per bag, and a wrong guess declares a contract the writers do not honor. - ---- - ## Owned-codegen & scaffold-and-own assessment - A config may import `entityFile` / `queriesFile` / `routesFile` / `barrel` from @@ -423,54 +175,13 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend neither modeled nor carried in the `source.rdb` `@sql` / `@unmanaged` escapes is *unmanaged* (informational only — never actionable drift, never auto-dropped), so a hand-written view standing in for an expressible `object.projection` / entity read-view can never be outsourced - to the drift gate; hunt it here (drift signature 8, below). Once carried in `@sql` (#208) it IS + to the drift gate; hunt it here (drift signature 8 in `references/drift-signatures.md`). Once carried in `@sql` (#208) it IS registered, fingerprinted, and drift-checked — no longer audit-only. - **Version skew:** check *actually-resolved* package versions, not declared; consuming a fix requires a coordinated lockstep bump, not a source-file copy. --- -## Prompt anti-patterns (hunt per site; classify: fully-modeled / partial / fully-inline) - -**The census covers EVERY module**, not only the application: test harnesses, simulators, -evaluation and seeding scripts, and tooling. A census scoped to "the app" is how hand-built -prompts, and the regex that parses their replies, survive a migration that reports itself -complete. A site is anything that builds text sent to an LLM (format string, f-string, text -block, template literal, string builder, concatenation) or reads its reply (regex, string -search, hand-rolled JSON/XML parsing). - -**Say what keeps the result in place.** An audit is a one-off; nothing in `meta verify` flags -a NEW hand-built prompt. When the census comes back clean, check that the project's always-on -context carries the prompt rule (regenerated by `meta init --refresh-docs`) and recommend a -project-local guard where agents keep adding prompts — a check that flags a literal or -format-built argument at the project's own LLM request call sites. - -- Inline prompt strings (triple-quoted / template-literal constants in service code). -- Untyped payloads (`str.format(**dict)` / f-strings / ad-hoc dicts) — payload should be a declared - shape: an `object.value` (caller-supplied fields; `origin.passthrough` only — FR-015 parameter - lineage) or, when fields derive by `aggregate` / `computed` / `first`, a - **sourceless `object.projection`** carrying those origins (#210 — assembly origins on an - `object.value` fail load with `ERR_SUBTYPE_RULE_VIOLATION`; `@payloadRef` accepts the sourceless - projection). -- Silent-degradation hack (`try/except KeyError` or `?? ''` around formatting) — flag every instance. -- Hand-rolled output parsing (regex / XML / ad-hoc JSON) vs a declared **responding - `template.prompt`** (one carrying `@responseRef`) + generated `parse*` / `safeParse*` / - `extract*` parser — **generated in all five ports** (Java's generated `Parser` owns - the Jackson `readValue`); flag a hand-rolled parser in a **non-generated** file where a - responding `template.prompt` exists. ADR-0052: a `template.output` is outbound only and - generates no parser, so it is not the node to look for here. -- Engine-side formatting breaking byte-identical render (prompt-cache exact-prefix hits - depend on byte-stability). -- `template.toolcall` candidates: LLM tool schemas hand-defined per call vs modeled - `toolcall @toolName/@payloadRef`. -- `@responseRef` + AI-trace: hand-parsed responses with no typed response shape; note that - `voRequest` / `voResponse` jsonb columns must be authored `field.object` — the loader - must not mutate the tree; vendor SDK client + pricing are BYO (ADR-0024). -- No `meta verify --templates` gate; no declared `@maxChars` / `@maxTokens` budget. -- **Template-text duplication.** Hash every external template file (`md5sum` the template dir) and diff within groups sharing a `payloadRef`. A byte-identical cluster, or N≥3 templates sharing an identical skeleton with only slot lines differing, is ONE prompt maintained N times — a wording fix must be applied N times and WILL be missed. Recommend a shared `{{> group/partial }}` include (the render engine inlines partials recursively) or one template + payload-carried variant fields. Do NOT flag: the per-node `payloadRef` / `@format` repetition across `template.prompt` nodes (loader-forced, not a copy); templates meant to diverge independently — require N≥3 and structural identity before flagging. - ---- - ## Semantic-constraint ratification (prevents over-modeling) When folding hand validators into metadata, apply human judgment per constraint. @@ -550,52 +261,3 @@ The audit never edits code. Pattern: **dry-run → review the diff → apply**. - Cut over **one surface at a time, one commit each**. - A separate guided-cutover skill (not this one) reads `adoption-audit.json` and walks findings one tier/surface at a time with human approval at each step. - ---- - -## Guardrails - -- **Adoption direction — metadata follows the code.** This is a brownfield project: existing - code and the live schema are the spec. Every `metadata_sketch` must **reproduce the code's - existing native types, names, and nullability** (model `field.uuid` where the code uses - `UUID`, carry over `@column`/`@table`/`@required`) and every cutover must **minimize churn to - code the generator is not replacing** — customize the codegen to match the existing shape - before proposing edits to working call sites. A sketch that would re-type or rename working - code the generator isn't replacing is modeling the wrong thing; when a choice is ambiguous, - flag it for the human rather than proposing the churnier option. (Full doctrine: - `metaobjects-authoring` → "Adopting onto an existing codebase".) -- **Parity-gate every cutover** — prove behavior-equivalent before deleting hand-written code; generated schemas are often looser. -- **Verify, don't assume** — read the code behind a grep hit. -- **Verify the DB artifact, not just the types** — the contract may claim a column the view DDL dropped. -- **Don't let one bespoke action block generating the entity** — generate CRUD; mount the custom action alongside. -- **Consumption ≠ a dist copy across versions** — bump + rebuild lockstep and install. - ---- - -## Calibration — port gaps & non-defects (do NOT flag these as adopter fault) - -- **Filter-operator route codegen — the baseline grammar ships in all five ports.** The - `?filter[field][op]=value` grammar (all 9 operators `eq/ne/gt/gte/lt/lte/in/like/isNull`, - implicit-AND across params, the generated `FilterAllowlist` + `invalid-field` / - `invalid-op` / `in`-over-cap 400s) is **generated in every port** (Java `SpringControllerGenerator`, - C# `RoutesGenerator`, Python `router_generator`, Kotlin `KotlinSpringControllerGenerator`, TS) — - gated by the api-contract corpus in BOTH lanes. **Flag hand-rolled filter parsing anywhere.** - Only the *richer* surface is genuinely **TS-only**: free-text `?search=`, the explicit - `filter[or][N]` / `filter[and][N]` nested boolean combinators (+ their nesting-depth cap), and - leading-wildcard gating — do NOT flag the absence of those in a non-TS port. -- **Output-parser codegen** ships in **all five ports** — Java's `SpringOutputParserGenerator` - *generates* the `Parser` (the Jackson `readValue` lives inside that generated file). A - hand-rolled parser in a **non-generated** file where a **responding `template.prompt`** - (`@responseRef`) exists IS a finding. Per ADR-0052 a `template.output` emits no parser. -- **Python** still hand-wires the FastAPI router + repository impl around a generated - `APIRouter`; relationship / non-`table` source-kind / `field.object flattened` codegen is partial. -- **C#** has no ObjectManager runtime tier (EF Core is the runtime) — hand services over the generated `DbContext` are expected. -- **Cut subtypes** — `field.byte` / `field.short` / `field.class` are removed; never recommend them. -- **TS/web-only** — `view.*` widget subtypes load in every port but are consumed only by TS/web; only `view.base` / `view.currency` are cross-port-gated. -- **Planned, not shipped** — `api.*` / `operation.*` / `binding.*` (FR-024) and MCP exposure of declared prompts/tools are not yet in the registry; their absence is not an adopter defect. -- **Cross-port version-NUMBER skew is by design** — TS/C#/Python `0.x` vs Java/Kotlin `7.x` Maven is correct; never flag the *number lines* differing. But that is exactly why you can't eyeball cross-language drift: compare **`metamodelVersion`** (Phase 0 cross-language consistency item), not the package numbers. A `metamodelVersion` MISMATCH across ports *is* a finding; so is a port lagging its ecosystem's latest release. Also flag *intra-port* drift (mixed versions within one port, or a runtime package in `devDependencies`). -- **Stale upstream prose** — "hand-write the Spring controller" (Java/Kotlin) is out of date; trust `meta gen --list`, not stale prose. - ---- - -For this project's port specifics and the exhaustive capability checklist, read every `references/*.md` in this skill's directory. diff --git a/agent-context/skills/metaobjects-audit/references/calibration.md b/agent-context/skills/metaobjects-audit/references/calibration.md new file mode 100644 index 000000000..4271e57ee --- /dev/null +++ b/agent-context/skills/metaobjects-audit/references/calibration.md @@ -0,0 +1,34 @@ +# Calibration — port gaps and non-defects (do NOT flag these as adopter fault) + +## Contents +- Filter-operator route codegen +- Output-parser codegen +- Python +- C# +- Cut subtypes +- TypeScript / web-only vocabulary +- Planned, not shipped +- Cross-port version-number skew +- Stale upstream prose + +- **Filter-operator route codegen — the baseline grammar ships in all five ports.** The + `?filter[field][op]=value` grammar (all 9 operators `eq/ne/gt/gte/lt/lte/in/like/isNull`, + implicit-AND across params, the generated `FilterAllowlist` + `invalid-field` / + `invalid-op` / `in`-over-cap 400s) is **generated in every port** (Java `SpringControllerGenerator`, + C# `RoutesGenerator`, Python `router_generator`, Kotlin `KotlinSpringControllerGenerator`, TS) — + gated by the api-contract corpus in BOTH lanes. **Flag hand-rolled filter parsing anywhere.** + Only the *richer* surface is genuinely **TS-only**: free-text `?search=`, the explicit + `filter[or][N]` / `filter[and][N]` nested boolean combinators (+ their nesting-depth cap), and + leading-wildcard gating — do NOT flag the absence of those in a non-TS port. +- **Output-parser codegen** ships in **all five ports** — Java's `SpringOutputParserGenerator` + *generates* the `Parser` (the Jackson `readValue` lives inside that generated file). A + hand-rolled parser in a **non-generated** file where a **responding `template.prompt`** + (`@responseRef`) exists IS a finding. Per ADR-0052 a `template.output` emits no parser. +- **Python** still hand-wires the FastAPI router + repository impl around a generated + `APIRouter`; relationship / non-`table` source-kind / `field.object flattened` codegen is partial. +- **C#** has no ObjectManager runtime tier (EF Core is the runtime) — hand services over the generated `DbContext` are expected. +- **Cut subtypes** — `field.byte` / `field.short` / `field.class` are removed; never recommend them. +- **TS/web-only** — `view.*` widget subtypes load in every port but are consumed only by TS/web; only `view.base` / `view.currency` are cross-port-gated. +- **Planned, not shipped** — `api.*` / `operation.*` / `binding.*` (FR-024) and MCP exposure of declared prompts/tools are not yet in the registry; their absence is not an adopter defect. +- **Cross-port version-NUMBER skew is by design** — TS/C#/Python `0.x` vs Java/Kotlin `7.x` Maven is correct; never flag the *number lines* differing. But that is exactly why you can't eyeball cross-language drift: compare **`metamodelVersion`** (Phase 0 cross-language consistency item), not the package numbers. A `metamodelVersion` MISMATCH across ports *is* a finding; so is a port lagging its ecosystem's latest release. Also flag *intra-port* drift (mixed versions within one port, or a runtime package in `devDependencies`). +- **Stale upstream prose** — "hand-write the Spring controller" (Java/Kotlin) is out of date; trust `meta gen --list`, not stale prose. diff --git a/agent-context/skills/metaobjects-audit/references/capability-checklist.md b/agent-context/skills/metaobjects-audit/references/capability-checklist.md index 61c563d4a..9ba3439d7 100644 --- a/agent-context/skills/metaobjects-audit/references/capability-checklist.md +++ b/agent-context/skills/metaobjects-audit/references/capability-checklist.md @@ -14,6 +14,22 @@ classify it (using the classification scheme in `SKILL.md`) and route the cutove --- +## Contents +- Object — `object.*` +- Field — `field.*` +- Source — `source.rdb` +- Relationship — `relationship.*` +- Identity — `identity.*` +- Index — `index.*` (non-unique retrieval) +- Origin — `origin.*` (derived fields — on projections AND entity read-views) +- Validator — `validator.*` +- View / Layout — `view.*`, `layout.*` +- Template — `template.*` (prompt pillar) +- Attr — `attr.*` +- Requirement — `requirement.*` (capability ledger) +- Common documentation attrs (any node) +- Cross-cutting + ## Object — `object.*` - **`object.entity`** (`@discriminator` / `@discriminatorValue` for STI/TPH) — hunt diff --git a/agent-context/skills/metaobjects-audit/references/csharp.md b/agent-context/skills/metaobjects-audit/references/csharp.md index 3a9099661..e569f647e 100644 --- a/agent-context/skills/metaobjects-audit/references/csharp.md +++ b/agent-context/skills/metaobjects-audit/references/csharp.md @@ -7,6 +7,14 @@ subcommand and `dotnet meta verify --db` is rejected with exit 2. --- +## Contents +- Finding generated files +- Run codegen + verify +- Drift signatures (what to grep for) +- Owned generators +- Version-skew check +- Calibration gaps (do NOT flag these) + ## Finding generated files Generated files are named `.g.cs` and carry an `@generated` guard comment: diff --git a/agent-context/skills/metaobjects-audit/references/drift-signatures.md b/agent-context/skills/metaobjects-audit/references/drift-signatures.md new file mode 100644 index 000000000..edab253ff --- /dev/null +++ b/agent-context/skills/metaobjects-audit/references/drift-signatures.md @@ -0,0 +1,57 @@ +# Drift signatures (highest-value; grep-then-verify) + +## Contents +- 1. Hand validators shadowing a generated schema +- 2. Field-by-field serializers and mappers +- 3. camelCase / snake_case maps +- 4. Drift-admitting comments +- 5. Runtime schema patching +- 6. N declarations of one shape +- 7. `own*()` reads of an effective property +- 8. Hand-written `CREATE VIEW` / read-only SQL +- 9. A closed variant set hand-modeled per instance +- 10. N declarations of one field across objects +- 11. A metadata-derived value spelled a second time by hand +- 12. An open JSON bag whose readers agree on a shape + +Per finding: `file:line` → what → generated-equivalent exists? → recommendation. + +1. **Hand validators shadowing a generated schema** — diff field-by-field; divergence is the bug. +2. **Field-by-field serialize / deserialize / DTO↔model / row mappers** — silently drops a field when metadata grows one. +3. **camelCase↔snake_case / body↔column maps** maintained beside a generated view that already renames. +4. **Drift-admitting comments** — grep: `"keep in sync with"` / `"mirrors the"` / `"matching the"`. +5. **Runtime schema patching** (`ALTER TABLE … ADD COLUMN IF NOT EXISTS`, `_ensure_schema()`) — N schema owners. +6. **N declarations of one shape** — same entity as Drizzle table + Zod schema + Pydantic model + hand dataclass; target is 1 + N generated. +7. **`own*()` accessor read of an effective property** (ADR-0039) — `ownAttr` / `ownFields` / `own_children` / bare Python `attr(` / `getMetaAttr(name, false)` / native `IsArray` used to read a value or iterate members outside the sanctioned subclass-emit / own-serializer / `@dbColumnType` cases → silently drops `extends`-inherited values. A **correctness defect** (axis G2), not advisory. +8. **Hand-written `CREATE VIEW` / read-only SQL standing in for a projection or entity read-view (view-necessity test).** Grep migrations, checked-in `.sql`, and repository/query code for `CREATE [OR REPLACE] [MATERIALIZED] VIEW`, and for hand-rolled read-only queries that mirror a read model — a pure-`SELECT` repository/service method with joins or `GROUP BY` feeding a DTO, or a raw-SQL escape (`db.execute(sql…)`, `FromSqlRaw`, a JPA @Query with hand-written SQL). For each, run the **necessity test** — can origins express this shape? A column is derivable when it is (a) a base-entity or relationship-joined column → `origin.passthrough` (`@from` / `@via`), (b) a count/sum/avg/min/max over related rows → `origin.aggregate` (`@agg` / `@of` / `@via`), or an `EXISTS` / `array_agg` → `origin.aggregate` `any` / `all` / `collect` — any of them optionally row-scoped with `@filter`, (d) a computed scalar / **non-aggregate expression column** → `origin.computed` (`@expr`, #195), (e) one related row's column picked by an ordering — an **argmax / `DISTINCT ON … ORDER BY` / correlated `ORDER BY … LIMIT 1`** → `origin.first` (`@of` / `@via` / `@orderBy`, #195), or (f) a column borrowed via `extends` — and the joins follow declared relationships / `identity.reference` FKs. + - **Entity-shaped (`SELECT own.* + derived`) → an entity read-view, NOT a projection (#214).** The single most common legacy view is an entity's OWN columns plus a joined/derived extra (`SELECT o.*, c.name AS customer_name`). This is the `Order` **entity with a read route**, not an exposure contract — route it to an **entity read-view**: keep the writable `@role: primary` `@kind: table` source and add a **non-primary `@role: replica` `@kind: view`** source, declaring only the *extra* as a derived `origin.*` field on the entity (the own field set already covers `o.*`). Reads route to the replica view, writes to the table. Reach for a **projection** instead only when the view renames base columns or **row-filters** (`WHERE status='active'`, soft-delete) — the latter is a projection with an object-level `@filter` (#207) that lowers to the outer `WHERE`. + - **Exposure contract, expressible → CODEGEN CANDIDATE (high):** a subset / renamed / versioned / multi-base read model → convert to an `object.projection` with a read-only `source.rdb` `@kind: view` child, let `meta migrate` emit the `CREATE VIEW`, and consume the generated read-only query — the hand-written view is a second source of truth for a derivable shape. Parity-gate: the generated view returns row-identical results before the hand-written SQL is deleted. + - **Not expressible → carry it in `@sql` or `@unmanaged`, never a hand-edited migration (#208, ADR-0043).** When a NAMED irreducible construct blocks origin authoring — recursive CTE, window function / `OVER`, `UNION` / `INTERSECT` / `EXCEPT`, lateral join — the body still belongs in the metadata: carry the hand-written SQL in the `source.rdb` **`@sql`** escape — a read-only-`@kind` body the tool REGISTERS, fingerprints, and drift-checks (adopt a pre-existing view with `meta migrate --allow adopt-view`); `@sql` forbids `origin.*` children (two sources of truth). A DB object whose DDL is owned **entirely elsewhere** (Flyway / a hand-migration) → mark its source **`@unmanaged: true`** (legal on any `@kind` incl. `table`); `meta migrate` never creates/drops/drift-checks it and `verify --db` reports it as external. `@sql` and `@unmanaged` are mutually exclusive. **Only a view left *undeclared*** — neither modeled, nor `@sql`, nor `@unmanaged` — is truly *unmanaged*, invisible to `meta verify --db`, so this audit is the only gate that sees it. "It's an aggregation" is NOT an irreducibility justification (plain count/sum/avg/min/max rollups are `origin.aggregate`); nor is a `DISTINCT ON` pick-one-row (`origin.first`) or a non-aggregate expression column (`origin.computed`). +9. **A closed variant-set hand-modeled per instance** — N sibling modules / classes / config blocks, one per channel / provider / target, sharing a payload + config shape and diverging only by transport. Grep for sibling-file families and switch-on-a-string dispatch; verify the set is closed and recurring (never a one-off). → axis I "New-vocabulary OPPORTUNITY" (VOCAB CANDIDATE, advisory). +10. **N declarations of one FIELD across objects (same-name-field census)** — the field-level sibling of signature 6. Census field names across `object.*` nodes (`grep -rn 'name: '` the metadata dir); a name recurring in ≥2 objects where a canonical owner exists — one whose name the field embeds (``: `wizardId` → `Wizard.id`, `orderTotal` → `Order.total`) or whose type+constraints it matches — is provenance loss → `extends: Owner.field` (dotted child targets, ADR-0029). **VERIFY by diffing the copies' attrs: a `@maxLength` / `@required` / validator divergence across them is drift already shipping — cite it.** Evidence multiplier: `extends` already used elsewhere in the repo raises confidence. Do NOT flag: generic names on unrelated concepts (`id` / `name` / `status` with no owner-embedding name and no matching constraints); required per-node attrs a loader forces (e.g. `payloadRef` / `format` on sibling `template.prompt` nodes — a product constraint, not a copy). +11. **A metadata-derived value spelled a second time by hand.** A declaration is the single source of truth for every value the loader reads from it and every value a generator derives from it — the logical name, the physical table / schema / column / index names after the naming strategy, the resource path, an enum's member symbols, a label from `@title`, a limit from a validator child. Anywhere hand-written code or config carries a literal that must EQUAL one of those values, that literal is a second spelling, and a second spelling is drift waiting for the first one to move. **The three shapes this takes most often are raw SQL (a physical name), an API call or route mount (the resource path), and a hand-built form (a label, a limit, a message) — illustrative, never the set.** An adopter's other libraries carry the same defect in shapes this skill cannot list — a GraphQL resolver map, a search-index field, a cache or feature-flag key, a queue topic, a CSV `COPY` header, a form library's registration string, an LLM tool-parameter name — and the hunt has to reach those from the rule, because no menu will. Two generated surfaces hold the constants a site should reference instead: the per-object physical-names artifact (`.names.ts` / `Names` / `_names.py` — table, schema, `fields[].column`, identity and index names) on all five ports; and on TypeScript the entity module's constants with their browser-safe `.meta.ts` twin (`$entity`, `$table`, `$path`, and per field `name` / `label` / `view` / `htmlType` / `rules` / `options`), whose own docblock says to use them instead of magic strings. The other four ports answer that second surface with a generated typed handle — a Pydantic attribute, an EF property, an Exposed `Column` — which is already the gold standard. + + **The discriminator is what happens when the declaration changes, not whether the string looks like a name.** Rename the thing in metadata and regenerate. A site is this signature only when it SURVIVES the regeneration and goes wrong later — on the request the router matches, the query the database parses, the row a reader mis-keys, the export a human mis-reads — or never goes wrong and simply lies. A site the regeneration itself breaks before anything runs is already gated and is NOT a finding. The mechanism behind the two outcomes is whether the hand-written spelling is RESOLVED at the site or COMPARED by a consumer the value travels to: an identifier on a typed handle, a value narrowed by a generated union, a key of a typed filter object are resolved — by the compiler where there is one, by import or attribute resolution of that line where there is not — and the toolchain names the site; a string a database parses, a router matches, a JSON consumer looks up, a broker routes or a human reads is compared, and the failure surfaces in the consumer, on the data, or nowhere. On TypeScript "resolved" means the compiler checks it, and that one test turned an estate's 332 bare enum-member strings (`status === "edited_approved"`) into 0 findings — each is narrowed by the generated `export type = "a" | "b"`, so renaming the member breaks the build at every site. The physical-name case is the opposite pole and the reason "fails loudly" cannot be the line: a renamed column DOES fail loudly — in production, on that query, from the database — which is exactly the failure this signature exists to prevent. The gate that counts fires from the regeneration alone. + + **Hunt from the declaration outward, never from code shapes inward.** (1) Build the value list from the generated artifacts, because they already carry every derived value after every transform — naming strategy, pluralized path, humanized label: read `Names` and, on TypeScript, the `.meta.ts` twin. Where the port emits no names artifact yet, derive the list from the metadata directly (`@table` / `@schema` / `@column`, else the naming strategy applied to the field name; `@values`; `@title`) and record the un-wired generator as the first finding (below). (2) Grep the non-generated tree once — code AND checked-in config, fixtures, YAML, JSON, `.env`, OpenAPI fragments — with an alternation of those VALUES as literals; distinctive values (`"email_address"`, `"/subscribers"`, `"edited_approved"`) score on the hit, generic ones (`"id"`, `"name"`, `255`) only by reading the consumer, never by count. This step is what finds the topic name, the index field, the `COPY` header and the flag key without knowing they exist. (3) Trace each hit to its CONSUMER and ask what that consumer matches it against. Matched against the schema, the generated route table, a generated wire shape or a generated label → the metadata owns it, apply the discriminator. Matched against something the app chose — its own page-route table, its own copy deck — → not owned, not this finding. (4) Only then run the shape greps — raw SQL, `fetch(` / `app.get(` / a `queryKey`, a hand validator — as the second pass, for what a value grep cannot see: a literal that has ALREADY drifted from the declared value, a concatenated or interpolated name, a case-transformed one. + + **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a flattened value-object composite column belongs to no single field of either object; a relationship-synthesized FK column is derived, never declared; a write-through entity's replica view name is not in the artifact, which holds the PRIMARY source's. A hand-written repository that imports `Names` is the gold standard — call it out as good. + + **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. + + **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on ALL FIVE ports an existing project emits none until it asks**. ADR-0034 Amendment 2 made codegen opt-in everywhere: C# and Python require `--generators`, TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, and `meta init` scaffolds `generators: []` — so no port begins emitting this artifact merely because the package was upgraded. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. + + **Report TWO numbers for this signature, always, and never just the first: (a) is the artifact EMITTED, and (b) how many of the literal sites actually IMPORT it — the adoption ratio, as `/ sites`.** Emission is the cheap half and it is the half that gets done: measured on an adopter whose audit proved the artifact emitted correctly, ran the generator to show it, and then adopted it at **0 of 53 sites** (37 table + 16 column). Wiring the generator changes the scorecard; it changes nothing a reader of the code experiences, because the second spelling is still the one every query uses. An audit that reports only (a) makes the estate look upgraded while every literal it found is still there — so a project that emits the artifact and imports it nowhere scores WORSE than one that has not started, not better, because it now carries a third spelling that claims to be the source of truth. + + **A deferral must carry its measurement forward.** "Wire the names tier" is a real thing to postpone — on the observed estate it meant a new generated tier, a new dev-loop step, a new `verify --codegen` gate and edits to 4 service and 6 test modules that could only be validated against a live database, and the branch in hand deliberately changed nothing in the running application. That is a legitimate decision and belongs in the report. But record it with **the site count, the exact command that emits the artifact, and the modules that would change** — never as intent alone — and **re-raise it in every later audit until the ratio moves**. A deferral with no number attached reads as a nice-to-have on the next reading and silently becomes permanent; the one carrying `0/53` does not. + + +12. **An "open JSON bag" whose readers all agree on a shape.** `field.string` + `@dbColumnType: jsonb` is SANCTIONED vocabulary, not a defect — it is the deliberate escape hatch for a genuinely untyped JSON column with no value object to reference, it emits `unknown` / `z.unknown()` by design, and a cross-port corpus (`fixtures/api-contract-conformance/jsonb/`) gates that contract. **Never flag the pattern on sight.** Flag it only when the bag is not actually open: when the code demonstrably knows the shape. **Look at the WRITER FIRST — it is the strongest and cheapest evidence, and a reader-only hunt badly under-counts.** The function that writes the column declares the value's type in its own signature: `related_memory_ids: list[str]`, `vector_scores: list[float]`, `tools_offered: list[dict[str, Any]]`, `preferences: dict[str, Preference]`, a serialized dataclass / DTO / `@Serializable` class. If the writer's type is anything but a genuinely open map, the metadata is claiming less than the code already knows. Reader-side tells come second: a FIXED KEY SET in raw-SQL JSON-path reads (`->>'decision'`, `#>>'{a,b}'`), casts to a concrete type (`row.output as { draftText?: string }`), destructuring, typed parsing (`model_validate`, `readValue(…, X.class)`, `decodeFromString`), or a hand-written validator over the parsed value. Measured: hunting reader-side raw SQL ALONE found 2 columns on an estate; adding the writer signatures found many more on the same estate, because raw-SQL JSON paths are the rarest way to consume a jsonb column — most consumers parse it into a type. + + **The `isArray` sub-case is the commonest and is mechanical.** A bag whose writer types it `list[str]` / `string[]` / `List` is not an object and not an open bag: it is the BASE SUBTYPE plus `isArray: true` (`field.string @isArray`, `field.double @isArray`). Declaring it as an open jsonb bag loses the element type entirely, so nothing stops a writer putting a map where every reader expects a list. A plural field name (`scopes`, `relatedMemoryIds`, `participantEmails`, `capabilities`) plus a typed-collection writer is the whole test. A `*Json` SUFFIX is a second free tell — a field named `conditionJson` / `configJson` / `definitionJson` is admitting it carries a structured shape. At that point the shape exists; it is simply declared in N readers instead of once in the metadata, which is drift signature 6 with zero declarations rather than several — and it is invisible to every gate, because `verify --db` sees a jsonb column that matches and `--codegen` sees generated files that are correct for an open bag. + + **Measured on one estate:** 6 such columns, and one of them (`AuditEntry.output`) carried **12 distinct keys across 31 raw-SQL path reads plus 7 casts** in 10 files. Nothing in that is open. Estate-wide the pattern is common — 74 `field.string` + jsonb fields across five of nine projects — which is exactly why the discriminator matters more than the count: report the ones whose readers pin a shape, not the ones whose readers treat the value as opaque. + + **Do NOT flag:** a bag whose readers genuinely pass it through (logged, echoed, stored, forwarded to a third party) without reading a key; a third-party or LLM raw response whose shape is not the adopter's to declare; an **array of scalars**, which is `isArray: true` on the base subtype and never an object; a bag the metadata's own comment justifies as open with a reason. + + **Remedy — a ladder, chosen by the WRITER's declared type, and every rung gives the consumer a TYPED HANDLE rather than a string to get and set.** `list[X]` ⇒ the base subtype + `isArray: true`. `dict[str, X]` with a known `X` ⇒ **`field.map` + `@objectRef`** (a value object) or `@valueType` (a scalar) — registered vocabulary that emits `jsonb("col").$type>()` and `z.record(z.string(), X)`. **Check the CONSUMER before recommending this rung: `field.map` codegen now completes on all five ports** (Java types it `java.util.Map`; C# emits the `Dictionary` property and its EF jsonb storage mapping), **but no persistence- or api-contract-conformance fixture exercises `field.map` on ANY port, and the runtime persistence tier is uneven** — only Python's `ObjectManager` encodes a map, while `runtime-ts`, Java's OMDB and the Kotlin Exposed lane carry none. So where GENERATED CODE is the consumer, check the port before recommending this rung — nested map values are validated on every write path on every port as of 1.0.1 ([#362](https://github.com/metaobjectsdev/metaobjects/issues/362) — before that, Java skipped the cascade on TPH write paths and C# skipped it everywhere, silently accepting invalid values rather than erroring). **Check the adopter's pinned version before recommending this rung to a Java-TPH or C# consumer: on 1.0.0 the hole is open and they need their own boundary check.** Prefer a value object where a PORT RUNTIME must read the column back; a genuinely dynamic key set stays a bag. A serialized DTO / dataclass / `@Serializable` class ⇒ declare it as an `object.value` and point the field at it with `field.object` + `@objectRef` + `@storage: jsonb`. Only `dict[str, Any]` whose readers pin no key stays an open bag. **What the adopter GETS, per port** — this is the point of the conversion and worth stating in the finding: TypeScript emits the Drizzle column as `.$type()` plus the VO's own Zod schema and inferred type; Python annotates the field with the referenced VO's Pydantic model (and its `Create` on the wire, so a nested violation still validates); Kotlin emits typed Exposed jsonb codecs backed by a shared Jackson `MetaJsonbMapper`. In none of them does the consumer touch a JSON string — that is what the open bag costs them today, since `unknown` forces every reader to cast. The column stays jsonb, so this is a codegen/contract change and not necessarily a migration; what changes is that `unknown` becomes a generated type, the casts delete, and the keys become metadata-owned — which also promotes their literals into drift signature 11's scope, where the `->>'key'` sites were previously exempt because the metadata owned nothing. **Verify the column type is genuinely unchanged before promising no migration**, and parity-gate each converted read against the raw-SQL one before deleting it. Convert one column at a time: the shape is a judgment per bag, and a wrong guess declares a contract the writers do not honor. diff --git a/agent-context/skills/metaobjects-audit/references/java.md b/agent-context/skills/metaobjects-audit/references/java.md index 376e128ff..fcd381860 100644 --- a/agent-context/skills/metaobjects-audit/references/java.md +++ b/agent-context/skills/metaobjects-audit/references/java.md @@ -8,6 +8,14 @@ The runtime persistence tier is **OMDB** (`metaobjects-omdb`, --- +## Contents +- Finding generated files +- Run codegen + verify +- Drift signatures (what to grep for) +- Owned generators +- Version-skew check +- Calibration gaps (do NOT flag these) + ## Finding generated files Generated sources land under: diff --git a/agent-context/skills/metaobjects-audit/references/kotlin.md b/agent-context/skills/metaobjects-audit/references/kotlin.md index c94534f50..1928e3687 100644 --- a/agent-context/skills/metaobjects-audit/references/kotlin.md +++ b/agent-context/skills/metaobjects-audit/references/kotlin.md @@ -12,6 +12,14 @@ uses (`mvn metaobjects:generate`). Schema migration and live-DB drift are --- +## Contents +- Finding generated files +- Run codegen + verify +- Drift signatures (what to grep for) +- Owned generators +- Version-skew check +- Calibration gaps (do NOT flag these) + ## Finding generated files Generated sources land under: diff --git a/agent-context/skills/metaobjects-audit/references/prompt-anti-patterns.md b/agent-context/skills/metaobjects-audit/references/prompt-anti-patterns.md new file mode 100644 index 000000000..92cfb4748 --- /dev/null +++ b/agent-context/skills/metaobjects-audit/references/prompt-anti-patterns.md @@ -0,0 +1,44 @@ +# Prompt anti-patterns (hunt per site; classify: fully-modeled / partial / fully-inline) + +## Contents +- Census scope +- Keeping the result in place +- Anti-pattern list +- Template-text duplication + +**The census covers EVERY module**, not only the application: test harnesses, simulators, +evaluation and seeding scripts, and tooling. A census scoped to "the app" is how hand-built +prompts, and the regex that parses their replies, survive a migration that reports itself +complete. A site is anything that builds text sent to an LLM (format string, f-string, text +block, template literal, string builder, concatenation) or reads its reply (regex, string +search, hand-rolled JSON/XML parsing). + +**Say what keeps the result in place.** An audit is a one-off; nothing in `meta verify` flags +a NEW hand-built prompt. When the census comes back clean, check that the project's always-on +context carries the prompt rule (regenerated by `meta init --refresh-docs`) and recommend a +project-local guard where agents keep adding prompts — a check that flags a literal or +format-built argument at the project's own LLM request call sites. + +- Inline prompt strings (triple-quoted / template-literal constants in service code). +- Untyped payloads (`str.format(**dict)` / f-strings / ad-hoc dicts) — payload should be a declared + shape: an `object.value` (caller-supplied fields; `origin.passthrough` only — FR-015 parameter + lineage) or, when fields derive by `aggregate` / `computed` / `first`, a + **sourceless `object.projection`** carrying those origins (#210 — assembly origins on an + `object.value` fail load with `ERR_SUBTYPE_RULE_VIOLATION`; `@payloadRef` accepts the sourceless + projection). +- Silent-degradation hack (`try/except KeyError` or `?? ''` around formatting) — flag every instance. +- Hand-rolled output parsing (regex / XML / ad-hoc JSON) vs a declared **responding + `template.prompt`** (one carrying `@responseRef`) + generated `parse*` / `safeParse*` / + `extract*` parser — **generated in all five ports** (Java's generated `Parser` owns + the Jackson `readValue`); flag a hand-rolled parser in a **non-generated** file where a + responding `template.prompt` exists. ADR-0052: a `template.output` is outbound only and + generates no parser, so it is not the node to look for here. +- Engine-side formatting breaking byte-identical render (prompt-cache exact-prefix hits + depend on byte-stability). +- `template.toolcall` candidates: LLM tool schemas hand-defined per call vs modeled + `toolcall @toolName/@payloadRef`. +- `@responseRef` + AI-trace: hand-parsed responses with no typed response shape; note that + `voRequest` / `voResponse` jsonb columns must be authored `field.object` — the loader + must not mutate the tree; vendor SDK client + pricing are BYO (ADR-0024). +- No `meta verify --templates` gate; no declared `@maxChars` / `@maxTokens` budget. +- **Template-text duplication.** Hash every external template file (`md5sum` the template dir) and diff within groups sharing a `payloadRef`. A byte-identical cluster, or N≥3 templates sharing an identical skeleton with only slot lines differing, is ONE prompt maintained N times — a wording fix must be applied N times and WILL be missed. Recommend a shared `{{> group/partial }}` include (the render engine inlines partials recursively) or one template + payload-carried variant fields. Do NOT flag: the per-node `payloadRef` / `@format` repetition across `template.prompt` nodes (loader-forced, not a copy); templates meant to diverge independently — require N≥3 and structural identity before flagging. diff --git a/agent-context/skills/metaobjects-audit/references/python.md b/agent-context/skills/metaobjects-audit/references/python.md index 07fac5ae4..57368c30a 100644 --- a/agent-context/skills/metaobjects-audit/references/python.md +++ b/agent-context/skills/metaobjects-audit/references/python.md @@ -9,6 +9,14 @@ SQLAlchemy Core or asyncpg), or a hand-written repository implementation. --- +## Contents +- Finding generated files +- Run codegen + verify +- Drift signatures (what to grep for) +- Owned generators +- Version-skew check +- Calibration gaps (do NOT flag these) + ## Finding generated files Generated files land under the `--out` directory passed to `metaobjects gen` and diff --git a/agent-context/skills/metaobjects-audit/references/requirements.md b/agent-context/skills/metaobjects-audit/references/requirements.md index a81dfbafd..f7e2c9f66 100644 --- a/agent-context/skills/metaobjects-audit/references/requirements.md +++ b/agent-context/skills/metaobjects-audit/references/requirements.md @@ -9,6 +9,12 @@ Run `verify` first — the project's own: `meta verify`, `metaobjects verify`, gate. It settles referential integrity mechanically — do not spend audit effort re-deriving what a green run already proves. +## Contents +- What verify has already proven (do not re-check by hand) +- What only a human or an agent reading the code can catch +- Scope — do NOT flag these as defects +- Reporting + ## What verify has already proven (do not re-check by hand) A run answers two different questions in two separate sections. Both are already-proven diff --git a/agent-context/skills/metaobjects-audit/references/surface-review-axes.md b/agent-context/skills/metaobjects-audit/references/surface-review-axes.md new file mode 100644 index 000000000..1012d6fc5 --- /dev/null +++ b/agent-context/skills/metaobjects-audit/references/surface-review-axes.md @@ -0,0 +1,165 @@ +# Surface review — the axes + +The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs independently. Work the capability checklist (`capability-checklist.md`) alongside it, and read `calibration.md` before raising a finding. + +## Contents +- A. Codegen candidates — API / server routes +- B. Codegen candidates — web / client +- C. Drift hotspot — validators, mappers, runtime models +- D. Prompt pillar +- E. Owned generators and scaffold-and-own +- E2. Capability ledger — `requirement.*` +- F. Drift-gate adoption +- F2. Over-generation — unnecessary generated files +- G. Runtime-contract anti-patterns +- G2. `own*()` accessor discipline +- H. Authoring correctness / ADR conformance +- H2. Wrong native type +- I. Vocabulary hygiene / modernization (advisory) + +- [ ] **A. Codegen candidates — API / server routes.** Catalog + classify every handler. +- [ ] **B. Codegen candidates — web / client.** Pages, data layer (hooks, central fetch), + grids/forms/filters vs `layout.dataGrid` / form generators / filter-allowlist. +- [ ] **C. Drift hotspot — validators, mappers, runtime models.** Hand validators / DTO-mappers / + dataclasses shadowing a generated shape. Diff field-by-field; the divergence is the bug. +- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`). +- [ ] **E. Owned generators & scaffold-and-own** (see "Owned-codegen & scaffold-and-own assessment" in `SKILL.md`). +- [ ] **E2. Capability ledger — `requirement.*`** (see `references/requirements.md` and the + Requirement axis of the capability checklist). The one axis whose hunt is hand-written + PROSE rather than hand-written code: a `CAPABILITIES.md`, a README features table, a + conventions doc, a known-and-tolerated gap recorded as a TODO, or a rule that exists only + as a comment above the field it governs. Prose goes stale in silence; `@implementedBy` is + a reference the loader resolves, so a rename that invalidates the claim is caught. + **Entirely opt-in — an absent ledger is NOT a defect** (object coverage is a warning by + design, and a real estate reports most of itself unclaimed on day one). Score the prose + it would replace, and prefer ONE concrete claim an author can check over a + whole-estate migration. + **First establish the estate LOADS.** An older ledger fails the load outright, so + `meta verify` never runs and everything in `references/requirements.md` is unavailable: + `@violation` and `@verifiedBy` were retired in **0.24.0** (no deprecation shim — ADR-0023 + seals the registry); an index key declaring both `@fields` and `@expr` is refused since + **0.24.1**; and **0.24.2** turned `@status: abandoned | superseded` into **`retired`**. Run + **`meta upgrade`** (previews by default; `--apply` writes) before auditing, and audit the + upgraded tree. It rewrites only what has one correct answer and **refuses the rest, exiting + non-zero** — a refusal is itself a finding, naming the entries whose disposition nobody + recorded. + **Then audit the `retired` entries specifically.** They are where the ledger earns its + keep — agents proposing a retired capability's rebuild is the one failure this mechanism + has controlled evidence against — and they are also where it decays quietly. Check that + each `@statement` reads as a PROHIBITION rather than a diary entry ("X is never done", + not "we used to do X"), and that `@counterexample` describes the REVIVAL. A retired entry + whose statement narrates history is invisible to the reader it exists to stop. +- [ ] **F. Drift-gate adoption.** Is `meta verify` wired into CI / pre-commit? Which + subverbs (`--codegen` / `--templates` / `--db`)? Committed-codegen freshness gate? + Advisories heeded? Routine `--no-verify` bypass? Loader `ERR_*` / warnings addressed? + Parse the stable `code` field, not message text (ADR-0009). + **Is any advisory half switched OFF?** A gate can be wired, green, and muted. The + requirements authoring lint is advisory and mutable — `--no-requirement-lint` or + `META_NO_REQUIREMENT_LINT=1` silences it while the gate above it still runs and can still + exit 1, so a green pipeline says nothing about the seven `WARN_REQUIREMENT_*` authoring + checks. Grep CI config and scripts for both spellings. Muted is not automatically a defect — + a project that made a deliberate call is fine — but it moves those checks back onto the + auditor (see `references/requirements.md`), so **say which it is** rather than reporting the + ledger as lint-clean. + **Coverage completeness — "wired" is not "covers it all."** A gate can be present + yet blind to a whole artifact class. Confirm each subverb actually covers what the + project HAS: (a) **templates** — `verify --templates` on a CLI before the #193 fix + SKIPS `template.output @kind=email` and does not body-check a document + `template.output`'s mustache, so a `{{field}}` that drifted from the payload sails + through; a project with email/output templates that relies on `verify` alone (no + `meta gen` + `git diff --exit-code`, and no CLI past the fix) can ship broken — + **finding**; (b) **codegen** — committed generated code needs `--codegen` (or + gen+no-diff), else hand-edits/regeneration drift silently; (c) **schema** — `--db` + (Node `meta` only) for DB drift. Flag a gate that is WIRED BUT INCOMPLETE for the + artifacts present, and recommend the specific missing coverage. +- [ ] **F2. Over-generation — unnecessary generated files (codegen WASTE).** + Generated code is disposable, but that is NOT license to generate what nothing + consumes. Every generated file is a merge/review/maintenance surface and inflates + the leverage ratio falsely. Hunt OVER-generation (the inverse of A/B's + under-generation): + - **Dead generated files** — an `@generated` file that NOTHING imports or + references repo-wide (grep the emitted symbol / module path for importers): + delete it and stop generating it. + - **Artifacts an entity doesn't need** — REST routes / TanStack grids / forms / + hooks emitted for an entity that has no such surface. The remedy is the + generator's own `filter` option (`routesFile({ filter: (e) => e.name !== "Audit" })`, + `tanstackQuery({ filter })`), which is ANDed with the generator's built-in gates + and so can only NARROW what emits; a grid additionally needs a `layout.dataGrid` + on the entity at all. Flag the unused artifact and name the filter that would + suppress it. **Never recommend an `@emit*` attribute** — `@emitRoutes`, + `@emitTanstack`, `@emitForm`, `@emitGrid` and `@emitAngular` were never + registered vocabulary, so they passed `meta gen` (open load) and FAILED + `meta verify` (strict). A project carrying one is a **finding**, not an opt-out: + report it and route to `meta upgrade --apply` plus the generator config above. + - **Generators wired but unconsumed** — a generator in `metaobjects.config.ts` + `generators: [...]` (or the per-port equivalent) whose whole output class no + code imports: drop the generator rather than generate into the void. + - **Wrong target / duplicate output** — the same logical artifact emitted to two + places (a mis-set per-target `outDir`), one of which is orphaned. + Recommend generating ONLY what is consumed — decide per generator: drop the ones + whose whole output class nothing imports, and narrow the ones that over-emit with + their `filter`. A smaller, fully-consumed generated surface beats a large one with + dead files (which also make the leverage ratio lie; discount them from the census). +- [ ] **G. Runtime-contract anti-patterns.** Module-global `db` vs context-as-parameter + (ADR-0008); wire-canonicalization in the query path vs native in-process return types + (ADR-0019); runtime reflection to resolve a type from FQN vs generated static imports / + FQN registry (ADR-0001 / 0017); process-global registry vs per-loader (ADR-0014); code + that **mutates the loaded metadata tree** (read-only after load); JVM/Kotlin missing + startup validator; writes not routed to the `@role: primary` source; **`own*()` + accessor reads of effective properties / own-only member iteration (ADR-0039 — see + the active check in G2).** + +- [ ] **G2. `own*()` accessor discipline (ADR-0039 — CORRECTNESS DEFECT, not advisory).** + In any custom generator, metamodel provider, or runtime path (NOT the sanctioned cases + below), **flag every read of a field/node's effective property, or own-only member + iteration, done through an own-only accessor** — it silently drops everything inherited + via `extends` (a super-reference, not a flatten), corrupting codegen and runtime. This + is exactly the class of bug that broke Kotlin's array-type derivation (a concrete field + inheriting an array flag from an abstract parent generated a scalar) and, per the audit, + is latent cross-port. Grep for the own-only accessors and verify each hit: + - **TS:** `ownAttr(`, `ownChildren(`, `ownFields(`, a raw `isArray` field flag read → + should be `attr(` / `children()` / `fields()` unless emitting a subclass's own members. + - **Python:** `own_children(`, `own_fields(`, and the **inverted** bare `attr(` (Python + `attr()` is OWN; the resolving form is `attrs().get(`) → flag `attr(` used to read an + effective value. + - **Java / Kotlin:** `getMetaAttr(name, false)` (the `,false` own overload), own-only + child walks (e.g. an own-only `filterIsInstance<…>()` source lookup that emits nothing + for an entity inheriting its source). + - **C#:** a native `IsArray` flag read, `OwnChildren()`, own attr reads. + + **Sanctioned (do NOT flag):** (a) a generator emitting a generated **subclass** that + iterates `ownFields()` so inherited members aren't re-emitted (the generated base + declares them — the `class Sub extends Base` / TPH pattern); (b) the own-mode canonical + serializer + overlay-merge + super-resolution walks (library-internal); (c) the single + deliberately-own attribute `@dbColumnType` (a physical column-type override, never + inherited). Any own read that carries a comment naming one of these cases is fine; an + uncommented own read of an effective property is the defect. +- [ ] **H. Authoring-correctness / ADR-conformance (deep).** Invented/unregistered + `@`-attrs or post-bootstrap registration (ADR-0023 — custom attrs belong in a registered + provider or `attr.properties`); retired source-v2 forms (`source.dbTable` / `@name` / + `@dbColumn` → use `source.rdb` + `@kind` + `@table` / `@column` + `@role`, ADR-0007/0018); + taxonomy impurity (entity over read-only primary source; read model that should be + `object.projection`; `value` carrying identity/source, ADR-0028); copy-pasted base-field + blocks — or the same-named field re-declared across objects (drift signature 10) — instead of + abstract + `extends`; `@`-prefixed YAML keys (ADR-0006); relative refs + in committed canonical JSON (ADR-0032); DB-type-as-logical-subtype (ADR-0013); per-port + migration engine where schema is Node-`meta`-owned (ADR-0015). + +- [ ] **H2. Wrong native type — `field.` + `@dbColumnType` that hides the real type + (CORRECTNESS-ADJACENT finding, NOT advisory axis-I).** The headline instance is a **UUID + column modeled `field.string` + `@dbColumnType: uuid`**: the DB column is uuid but the + generated property is a **`String`**, so the code coerces `String↔UUID` at every boundary + and the native type is wrong everywhere the field is used. `verify --db` **cannot** catch it + (the column type matches), so it hides in plain sight. When it sits in a shared + `BaseEntity`/`BaseAuditedEntity`, **every inheriting `id`/`tenantId`/FK is wrong** — count the + blast radius (grep every `field.string` paired with `@dbColumnType: uuid`; it is often + hundreds of fields). This is a **real finding**, not a modernization nudge: recommend + `field.uuid` and flag it as a **staged migration** (re-typing `id`/FK ripples through + repositories, finders, and call sites) — tier by blast radius, not buried as advisory. The + ONLY non-finding is a field the code genuinely handles as a *string* over a uuid column + (explicitly justified). Report the total pair count so the migration has a completion + criterion (see the CI ratchet gate in `metaobjects-verify`). + +- [ ] **I. Vocabulary hygiene / modernization (ADVISORY).** Flag already-retired or + deprecated authoring patterns and recommend the canonical form (see `vocabulary-hygiene.md`). + Advisory severity — scored as modernization opportunities, **never a failing finding**. diff --git a/agent-context/skills/metaobjects-audit/references/typescript.md b/agent-context/skills/metaobjects-audit/references/typescript.md index 6981e1b4e..8c8df4696 100644 --- a/agent-context/skills/metaobjects-audit/references/typescript.md +++ b/agent-context/skills/metaobjects-audit/references/typescript.md @@ -6,6 +6,14 @@ binary `meta`). The full feature set ships here; other ports are measured agains --- +## Contents +- Finding generated files +- Run codegen + verify +- Drift signatures (what to grep for) +- Owned generators — scaffold-and-own (ADR-0034) +- Version-skew check +- Calibration gaps (do NOT flag these) + ## Finding generated files Generated files carry the header: diff --git a/agent-context/skills/metaobjects-audit/references/vocabulary-hygiene.md b/agent-context/skills/metaobjects-audit/references/vocabulary-hygiene.md new file mode 100644 index 000000000..0457d764e --- /dev/null +++ b/agent-context/skills/metaobjects-audit/references/vocabulary-hygiene.md @@ -0,0 +1,98 @@ +# Vocabulary hygiene / modernization (axis I — advisory) + +## Contents +- Already-retired / deprecated forms +- Custom-provider vocabulary +- Hand-rolled reverse-query repository methods +- New-vocabulary opportunity (the inverse hunt) + +Per ADR-0037, vocabulary expansion follows ONE ordered test (derivable → derive; +physical-only → `@dbColumnType`; logical: different native type → subtype, same kind + +modifier → attribute). This axis surfaces authoring that predates or contradicts that +framework. **All findings here are advisory** — modernization opportunities scored as +such, surfaced in the roadmap, but **non-failing** (the code works; the form is dated). + +**Already-retired / deprecated forms → recommend the canonical form:** + +- `@dbColumnType: uuid_array` / `@dbColumnType: text_array` (a physical array column + type) → **`isArray: true`** on the base subtype. Array-ness is logical and derivable; + the array column type is retired. +- The `@kind: text` hack (forcing text via a kind override) → **bare `field.string`** + (text is the default; no override needed). +- `@dbColumnType: uuid_array` was covered above. **`field.string` + `@dbColumnType: uuid` + is NOT advisory — it is a real mismodeling finding (see axis H).** It generates a `String` + where the code uses/wants a native `UUID`, forcing `String↔UUID` coercions at every + boundary; `verify --db` passes (the column really is uuid), so the schema gate can't see it. + The genuine string-over-uuid-column case (code truly handles the value as text) is the ONE + legitimate use and must be explicitly justified — otherwise recommend **`field.uuid`**. +- `@dbColumnType: timestamp_with_tz` (ADR-0036 Wave 2) → **drop it.** `field.timestamp` is + instant / timezone-aware **by default** now; the `timestamp_with_tz` column-type override + is **retired**. Timezone-awareness lives in `field.timestamp` + the `@localTime` opt-out. +- A bare `field.timestamp` that is **semantically a wall-clock value** (a store-open time, a + birthday-with-time, a recurring local schedule) → recommend **`@localTime: true`** (naive + `timestamp without time zone`), or confirm it is genuinely meant to be an instant. Default + `field.timestamp` is an instant; only flag when the field's meaning is clearly wall-clock. +- A `validator.regex` (or a plain `field.string`) validating an **email** shape → recommend + **`@stringFormat: email`** on the `field.string` (ADR-0036 Wave 3). The native type stays + `string`; the per-port codegen emits the idiomatic email check — don't hand-roll the regex. +- A `field.string` that **holds a URL / URI** → recommend **`field.uri`** (native `URI`/`Uri`, + URL validation) — a distinct native type + behavior is a subtype, not a validated string. +- A `field.string` that **holds an IP address** → recommend **`field.inet`** (native IP type; + Postgres `inet` column). + +**Custom-provider vocabulary (adopters who register their own types/attrs):** check +new/custom vocab against the ADR-0037 procedure (advisory) — e.g. *a custom subtype +that differs from an existing one only by a property should be an attribute, not a +subtype*; an email/hostname string-validation is an **attribute** (`@stringFormat`, +native type unchanged), while a URL or IP is a **native type** (`field.uri` / +`field.inet`, a subtype). Recommend re-shaping against the ordered test. + +**Hand-rolled reverse-query repository methods (ADR-0038) → recommend the generated reverse +FK finder.** Reverse navigation — *"find all the rows that reference this one"* — is now +codegen. Flag a hand-written reverse-query method (a `findByParentId` / `findBy` +repository finder, or a manual `WHERE fk = ?` query helper — exactly what a JVM adopter +hand-writes in its `SceneRepository`) and recommend the **generated reverse FK finder** +instead: codegen now emits `findBy` plus a batched `…In(ids)` variant from +the FK metadata, idiomatic per port (Spring repository finder / EF query method / Python or +TS query function). It is **performant** (one indexed query, no N+1) and **framework-free** +(no lazy collections / proxies). When an entity has two FKs to the same target, the codegen +emits two distinct finders (named by the FK field) automatically — no annotation needed; the +reverse finder is a **codegen feature, not an attribute** (there is no reverse-nav `@`-attr +to author). Advisory severity — a modernization opportunity, not a failing finding. + +**New-vocabulary OPPORTUNITY (the inverse hunt — advisory).** The checks above audit +custom vocabulary the adopter *already* registered; this one hunts where the app +**should** register vocabulary but hand-coded the pattern instead. Smells +(grep-then-verify; require a *recurring, closed* set — never flag a one-off): +- **N parallel hand-written integration modules** sharing a payload shape and a config + pattern (a closed set of channels / providers / export targets, each a near-copy with + a different transport) → candidate for ONE project-registered subtype whose variants + sit behind a closed structural-variant discriminator (the `source.rdb` `@kind` + pattern), plus a small owned generator emitting the per-variant wiring. +- **An ad-hoc string discriminator steering code switches** — a string column whose + values select per-variant behavior/config scattered across the codebase. If the values + are just a closed symbol set, that is `field.enum` (`@values`), NOT new vocabulary. It + earns a subtype only when the discriminated concept **owns behavior or attributes of + its own** (ADR-0037 step 2a). +- **A provider-shaped pattern re-implemented per instance** — repeated registration / + config blocks restating a shape one declared node per instance could carry, with codegen + emitting the repetition. +- **A downstream integration modeled entirely in code** (an outbound webhook / notifier / + queue publisher / tool wiring beyond `template.toolcall`) where registered vocabulary + + a small generator would own the payload wiring, the names-only fail-closed config check, + and drift detection. + +Run the **ADR-0037 ordered test** before proposing: derivable → derive; differs from an +existing subtype only by a property → an attribute; a plain validated value → an +attribute; one-off author-supplied properties → the `attr.properties` bag. Only a concept +with its own behavior/attributes earns a subtype, and transport/protocol is never the +subtype axis (keep the node protocol- and address-free; variants behind the discriminator). +**Converge before inventing** — check shipped vocabulary AND the planned/chartered names +(see `calibration.md`, "Planned, not shipped"); never claim a chartered name for a project-local +type. **Recommendation:** propose a project-registered provider (explicit loader wiring, +ADR-0023 — never a loosened `strict` free-ride), a `metadata_sketch` of the node, and the +owned generator that retires the duplication; a second independent consumer needing the +same concept is a consumer→core promotion candidate (ADR-0011 — file an upstream issue). +**Verdict: VOCAB CANDIDATE (advisory)** — a modeling opportunity, never a failing finding, +never a tier gate; bias to under-flagging (the >15% false-positive kill criterion applies +with full force). Full guidance: `docs/features/downstream-metadata-decisions.md`. diff --git a/agent-context/skills/metaobjects-authoring/SKILL.md b/agent-context/skills/metaobjects-authoring/SKILL.md index 304bae183..3b164338e 100644 --- a/agent-context/skills/metaobjects-authoring/SKILL.md +++ b/agent-context/skills/metaobjects-authoring/SKILL.md @@ -25,11 +25,16 @@ This file covers what almost every model needs. The topics below live in | `references/adopting-existing-code.md` | working code or a populated schema already exists for what you are modeling | | `references/choosing-a-shape.md` | no row of "Canonical form for common field needs" fits; you need the full UUID / timestamp / URI / `@autoSet` rule; or you are registering your own vocabulary | | `references/json-columns.md` | a JSON column beyond the ladder below: coverage per port, `field.map` at runtime, version caveats, bag → native array | +| `references/identities-and-indexes.md` | declaring a primary, secondary or reference identity, an FK, or a non-unique `index.lookup` | +| `references/sources.md` | the `source.rdb` shape and `@kind`; a store MetaObjects does not manage; a derived or computed column (reach for an entity read-view first) | +| `references/abstracts-and-overlay.md` | a shared `abstract` base, `extends`, or a layered `overlay` file | +| `references/declaring-reports.md` | declaring `dimension.*`, `measure.*`, `segment.filter` or an `object.report` | +| `references/model-first-rationale.md` | someone argues that hand-writing a layer is cheaper this once | | `references/relationships.md` | a many-to-many link or self-join, or a "rows that reference this one" query | | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | -| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, rows with no facts (`@spine`), defaults (`@default`), engine differences | +| `references/reporting.md` | what a declared report gives you: what columns a report gets, time grains, null rules, rows with no facts (`@spine`), defaults (`@default`), engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -46,20 +51,6 @@ it?"* — and lets codegen own the *how*. **Describe WHAT, not HOW.** The metada the source of truth; generated code is a disposable, regenerable artifact — delete it and `meta gen` restores it identically. -**Why model-first wins even when hand-writing is cheaper this once — and it often -is, this once:** -- **Hand-writing a layer the metadata could own creates a second source of truth for - one fact.** A field's type, validation, column, route, and form then change in N - places and must stay consistent forever — not drift *risk*, but two sources of - truth for one fact, broken by construction. -- **The hand-roll saving is paid once; the consistency tax is paid on every future - change.** Assume the system will grow — it always does. The metadata amortizes - toward zero as the model is reused across layers and time; the hand-rolled - liability compounds with every field, refactor, and language port. -- **One metadata change regenerates persistence + DAO + API + UI consistently** — - and inherits every future generator improvement. Hand-writing opts out of all of - it, permanently. - **Before you hand-write anything data-shaped, STOP and find the model.** The moment you reach for a hand-written query, route, validator, form, relationship, or aggregate — that is almost always **metadata you have not declared yet.** In order: @@ -373,88 +364,10 @@ keys, bare attributes. Two house-style rules: The `[]` key-suffix declares an array field: `field.long[]: weekIds` lowers to `{ "field.long": { "name": "weekIds", "isArray": true } }`. -## Identities - -| Subtype | Purpose | Key attrs | -|---|---|---| -| `identity.primary` | the PK field(s) | `@fields`, `@generation` | -| `identity.secondary` | a unique alternate key (always enforces uniqueness — uniqueness is the type, not a `@unique` attr) | `@fields` (or `@expr` for a functional index) | -| `identity.reference` | an inbound FK from this entity to another | `@fields`, `@references`, `@enforce` | - -`@generation` on a primary controls value generation (e.g. `increment`). -`@fields` accepts a single string in authoring; it normalizes to an array in -canonical JSON. `@enforce` on a reference (default `true`) controls whether the -backend physically enforces it (a SQL FK constraint); set `false` for a logical -reference for navigation/typing/codegen only. Referential actions -(`@onDelete`/`@onUpdate`) normally live on the `relationship.*` node (see -Relationships below — the subtype carries the default), but `identity.reference` -also registers them as the **explicit per-FK override** (ADR-0047): use them for -a reference-only FK with no relationship, an M:N junction's FK sides (no -relationship ever correlates with a junction FK), or a single FK that must -deviate from its relationship's action. A reference-level action always wins -over the correlated relationship's. - -`@references` resolves cross-package by **fully-qualified name** -(`@references: "shared::billing::Account"`), the same rule as `extends`; a bare -name resolves within the current package. The FK target must be an entity with a -single-column primary key (the FK points at that PK); a target with a composite -PK needs the explicit dotted form `@references: "pkg::Target.fieldA,fieldB"`. - -**A dangling reference fails the load (0.11.0+).** An unresolved -`identity.reference.@references` raises `ERR_INVALID_REFERENCE` and an unresolved -`relationship.@objectRef` raises `ERR_INVALID_RELATIONSHIP` — the target entity must -exist (previously such references loaded silently). So every `@references` / -`@objectRef` you author must name a real entity. - -An `identity.secondary` can index an **expression** instead of plain columns: use -`@expr` (e.g. `"lower(email)"`) in place of `@fields`, optionally with `@using` (the -index method — `gin` / `gist` / `hash`; default `btree`) and `@where` (a partial-index -predicate). - -```json -{ "identity.primary": { "name": "id", "@fields": ["id"], "@generation": "increment" } } -{ "identity.secondary": { "name": "byEmail", "@fields": ["email"] } } -{ "identity.secondary": { "name": "byEmailCI", "@expr": "lower(email)" } } -{ "identity.reference": { "name": "fkAuthor", "@fields": ["authorId"], "@references": "Author", "@enforce": true } } -``` - -## Indexes (non-unique) - -Use `index.lookup` for a **non-unique** DB index added purely for query performance — it -does NOT enforce uniqueness. Choose the right construct by what the constraint IS: - -| Need | Construct | -|---|---| -| Unique alternate key (e.g. email, slug) | `identity.secondary` — uniqueness is the type | -| Query-performance index, no uniqueness | `index.lookup` | - -**An index keys off EXACTLY ONE of `@fields` or `@expr`.** `@fields` names the indexed -columns; `@expr` is a raw key expression used **instead of** `@fields` (a functional index). -Declaring **neither** — or **both** — is `ERR_INVALID_INDEX`. This applies to -`identity.secondary` as well as `index.lookup`: uniqueness lives in the type, so a unique -index keys itself the same way. - -A legacy index declaring both is fixed by `meta upgrade --apply`, which keeps `@expr` (the -index the database actually has). Do not hand-pick the survivor. - -The db provider contributes physical-tuning attrs alongside either form: `@orders` -(per-column sort direction), `@using` (access method — `gin`/`gist`/`hash`; default -`btree`), and `@where` (partial-index predicate). - -```json -{ "index.lookup": { "name": "byCreatedAt", "@fields": ["createdAt"], "@orders": ["desc"] } } -{ "index.lookup": { "name": "byStatusCreatedAt", "@fields": ["status", "createdAt"] } } -{ "index.lookup": { "name": "byEmailCI", "@expr": "lower(email)" } } -{ "identity.secondary": { "name": "uniqLowerEmail", "@expr": "lower(email)" } } -``` - -`index.lookup` is a sibling of `identity.*` — declare it as a direct child of an `object.entity`, -at the same level as fields and identities. - ## Relationships A `relationship.*` child is the navigation / ownership side of a link to another -entity; `identity.reference` (above) is the FK-column side. They are the two halves +entity; `identity.reference` (see `references/identities-and-indexes.md`) is the FK-column side. They are the two halves of one FK. **Choose the subtype by ownership semantics — it decides the default referential action**, so modeling every FK as `composition` silently arms unintended `CASCADE` deletes: @@ -528,149 +441,6 @@ enforcement (a CHECK constraint / cross-field assertion) — no raw expression i These are children of `object.entity`, alongside its fields and identities. -## Sources — `source.rdb` + `@kind` - -`source.rdb` declares where an entity's data lives. Read-only-ness derives from -`@kind` (it is NOT a separate subtype): - -| `@kind` | Read-only | Default? | -|---|---|---| -| `table` | no | yes (when `@kind` omitted) | -| `view` | yes | – | -| `materializedView` | yes | – | -| `storedProc` | yes | – | -| `tableFunction` | yes | – | - -The physical name attr is **kind-matched** — never `@name`: `@table` for a table, -`@view` for a view, `@materializedView`, `@proc` (storedProc), `@function` -(tableFunction). (`@table` on a non-table kind is a pre-1.0 legacy spelling the -canonical serializer rewrites to the kind's alias; author the kind-matched attr -directly.) The physical column name on a field is `@column`. `@schema` namespaces -the DB schema (Postgres default `public`; SQLite rejects non-default values). -Multi-source: multiple `source.rdb` children, each with a `@role`, exactly one -`primary`. - -```json -{ "source.rdb": { "@kind": "view", "@view": "v_author", "@schema": "blog" } } -``` - -**This declaration is the only place a physical name is ever spelled.** Codegen emits a -per-object names artifact from it (`Names` — `ProgramNames.fields.createdAt.column`, -`ProgramNames.CreatedAtColumn`, `PROGRAM_CREATED_AT_COLUMN`, per port), the generated table -binding reads that artifact, and hand-written SQL, repositories and migration scripts -reference it — so a consumer never restates `v_author` or `created_at`, and a rename here -propagates. Declare `@column` explicitly whenever the physical name is not the naming -strategy's answer (`callPurpose` → `purpose_code`): nothing downstream can recover that -mapping by derivation, and the constant is what carries it. (See `metaobjects-codegen` → -"Never hand-write a physical name".) - -**An entity's PRIMARY source must be writable** (`table`) — read-only kinds are -legal only in non-primary roles. - -### A store MetaObjects does not manage — MongoDB, Cassandra, Neo4j, an API - -`source.rdb` is the only registered source. A record kept in a document, wide-column or -graph store, or behind a remote API, is an `object.entity` with an `identity.primary` and -**no `source.*` child**: a *sourceless entity*. Do not invent a document or graph source -subtype, or a `@collection` attr: ADR-0007 designs them but registers only `rdb`, so the -loader refuses them. Put -store settings in your own property bag instead: any `@name` whose value is a JSON object is -a registered `attr.properties` bag, for example `"@mongo": { "collection": "orders" }`. - -A sourceless entity gets no table, queries, routes or migration. It still gets its wire -contract in every port: the create and PATCH schemas and the filter allowlist. Keep the -`identity.primary`, because it is what makes the record addressable. Without it the object -is a shape only. The data access layer is a generator you own: the procedure is in the -`metaobjects-codegen` skill, `references/any-stack.md`, and on TypeScript -`references/typescript-document-store.md` carries a complete, tested MongoDB repository -generator to copy. - -**The database dialect is codegen config, not metadata.** MySQL, Postgres, SQLite and D1 -entities are all modeled with `source.rdb`; `dialect` in the codegen config picks the target. -Do not encode the database in the model. - -### Derived/computed columns: reach for an ENTITY READ-VIEW first - -**Do not default to `object.projection` for a list screen, grid, or "one extra -column" read.** The usual need — an entity's own rows plus a joined display name or a -per-row count, filterable and sortable like any other column — is an **entity -read-view**, not a projection: - -```jsonc -{ "object.entity": { "name": "Order", "children": [ - { "source.rdb": { "@kind": "table", "@table": "orders" } }, // writes - { "source.rdb": { "@kind": "view", "@view": "v_order", "@role": "replica" } }, // reads - // …the entity's own fields stay as they are — you re-state NOTHING… - { "field.string": { "name": "customerName", "@filterable": true, "children": [ - { "origin.passthrough": { "@from": "Customer.name" } } ]}}, - { "field.int": { "name": "itemCount", "@filterable": true, "children": [ - { "origin.aggregate": { "@agg": "count", "@of": "OrderItem.id", "@via": "Order.items" } } ]}} -]}} -``` - -That is the whole change. Writes route to the table (derived fields are excluded from -the write codecs); reads route to the view; `meta migrate` emits the `CREATE VIEW` from -the **same assembly logic** a projection view uses — one emitter, two hosts (FR-024 §7, -#213/#214). Filtering and sorting on `customerName` / `itemCount` work like any other -column because the filter tier runs against the view. - -**When you need an independent exposure** — a subset, renamed columns, a row-filtered -view, a versioned or external shape — that is an `object.projection`. It, the `origin.*` -vocabulary, the structured `@filter` / `@expr` shapes and the `@sql` / `@unmanaged` escapes -are in `references/read-views-and-projections.md`. - -## Abstracts + `extends` (deferred resolution) + `overlay` - -An **abstract** node (`abstract: true`) describes a shape but is never emitted as -a concrete entity. A concrete node references it via `extends:` to inherit its -children + attrs. This is the lightest reuse mechanism — pure data, no codegen -change. - -```yaml -- object.entity: - name: BaseEntity - abstract: true - children: - - field.long: { name: id } - - field.timestamp: { name: createdAt, required: true } - -- object.entity: - name: Author - extends: BaseEntity - children: - - source.rdb: { table: authors } - - field.string: { name: name, required: true } - - identity.primary: { name: id, fields: id } -``` - -Resolution facts: - -- **Deferred.** `extends:` resolves *after all files load* — abstracts can live in - any file, forward references are fine. -- **Multi-level chains resolve through the whole chain** (`Author extends BaseEntity - extends Auditable`) — each `extends` is a super-*reference*, so resolution walks the - full chain (it does not flatten inherited members onto the child; see ADR-0039). -- **Cross-package** refs use the fully-qualified name (`extends: "shared::auditable"`); - same-package refs use the bare name. -- An unresolved reference fails with `ERR_UNRESOLVED_SUPER`. - -`abstract` and `extends` are **structural keys** (bare, no `@`). - -**Inherited properties live on the parent** (`extends` is a super-reference, not a flatten). -A generator or provider must read through the resolving accessors, never `own*()` (ADR-0039; -the `metaobjects-codegen` skill has the per-port table). - -**`overlay` is a different concept.** `extends:` is an IS-A relationship between -two distinct nodes. `overlay: true` re-opens the *same* named node to amend it -across files (same `package` + same `name` → merged; last-writer-wins on attr -conflicts, structural children accumulate). Use `extends` to share shape between -distinct entities; use `overlay` to split one entity's declaration across files. - -**Building on a dependency's metadata** (`dependencies` in `.metaobjects/config.json`): -its nodes load before yours, so `extends` and `overlay: true` work across the boundary. Always -say `overlay: true` on an amendment to a node you do not own. The rules are in -`references/metadata-dependencies.md`. - ## Discriminator inheritance (TPH) Several variants of one thing sharing **one table**: the base `object.entity` declares @@ -678,101 +448,6 @@ Several variants of one thing sharing **one table**: the base `object.entity` de `@discriminatorValue`. Codegen emits per-subtype routes with the discriminator injected and immutable. Supported in all five ports; the worked example is in `references/inheritance-tph.md`. -## Reporting — dimensions, measures and reports - -Reach for it when a dashboard number would otherwise be a hand-written `GROUP BY`: revenue per -day, buyers per program, a total. You name the pieces once, on the entity that owns the rows, -and an `object.report` combines them by name. Four node kinds: - -- `dimension.attribute` / `dimension.time` — what to group by (`@of: Entity.field`; a time - dimension lists the `@grains` it supports: `hour`, `day`, `week`, `month`, `quarter`, `year`); -- `measure.aggregate` (`@agg`: `count`, `sum`, `avg`, `min`, `max`) and `measure.ratio` - (`@numerator` / `@denominator`, both measures of the entity); -- `segment.filter` — a named, reusable `@filter` ("active purchase"); -- `object.report` — a top-level object: `@from` an entity, `@dimensions` (`name` or - `name:grain`), `@measures`. - -```json -{ "metadata.root": { - "package": "acme::shop", - "children": [ - { "object.entity": { - "name": "Purchase", - "children": [ - { "source.rdb": { "@table": "purchases" } }, - { "field.long": { "name": "id" } }, - { "field.string": { "name": "status" } }, - { "field.currency": { "name": "amountCents" } }, - { "field.timestamp": { "name": "purchasedAt" } }, - { "identity.primary": { "name": "id", "@fields": ["id"] } }, - { "segment.filter": { "name": "active", "@filter": { "status": "active" } } }, - { "dimension.time": { "name": "purchasedAt", "@of": "Purchase.purchasedAt", - "@grains": ["day", "month"] } }, - { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id", - "@segment": "active" } }, - { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents" } } - ] - }}, - { "object.report": { - "name": "DailyRevenue", - "@from": "Purchase", - "@dimensions": ["purchasedAt:day"], - "@measures": ["purchases", "revenue"], - "children": [ - { "source.rdb": { "@kind": "view", "@view": "v_daily_revenue" } } - ] - }} - ] -}} -``` - -Three rules an author trips on: - -1. **Every measure belongs to `@from`.** A report cannot mix measures of two entities (joining - two fact tables multiplies each side's rows); two fact tables are two reports. -2. **`@via` is to-one only, and each hop needs a declared foreign key.** A dimension reaches a - related entity's column through a `relationship.*` with `@cardinality: one` (or an - `identity.reference`), never through a to-many, which would repeat fact rows and double-count - a `sum`. The view joins the hop through an `identity.reference` between the two entities - (`{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": - "Program" } }`); a relationship with none behind it loads and then fails `meta migrate`. -3. **A report declares no fields.** Its columns are derived: one per dimension, then one per - measure (a time dimension at a grain is ``, so `purchasedAt:day` is - `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. - -Two questions to answer for every report: - -- **Do I want the rows that have no facts?** A report's rows come from the facts, so a - program nobody bought has no row. If the page should list it, add `"@spine": - "Purchase.program"` to the report (a to-one path from `@from`, written like `@via`), and - **reach every listed dimension through it**: each needs an `@via` that begins with the - spine's hops (`programId` as `Program.id` via `Purchase.program`). A dimension on the fact - row is a load error under `@spine`. List the spine entity's key to get one row per program. -- **Is zero true here, or is null?** A `sum`, `avg`, `min`, `max` or ratio with nothing to - aggregate is null. If zero is the true answer (revenue of a program nobody bought), declare - `"@default": 0` on the measure; if the value is unknown (an average rating with no - ratings), leave it null. **Never on a `count`**: it is already `0`, and `@default` there is a - load error. Write `0`, never `0.0`. - -**A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration -is what makes `meta migrate` create the view (Postgres, SQLite, D1), what every port's -runtime reads, and what makes every port's generators serve it: one read-only list route at -the pluralized snake_case name (`InvoicesByMonth` is `/invoices_by_months`), with paging, -and filter and sort on every derived field whose type has filter operators (an `object` or -`map` derived field is neither filterable nor sortable), `405` on `POST` and no `/{id}` -route. A report with -no `source.*` is checked at load and generates nothing. - -A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its -base's table): declare it `@from` the base with an `@filter` on the discriminator field. - -What does not exist: no grid or form for a report (TypeScript generates a list hook; no other port has a client tier), no way to narrow which -derived fields are filterable, no `measure.derived` -(arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or -measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and -relative dates are UTC). Column types, the null rules, Monday weeks and per-engine differences -are in `references/reporting.md`. - ## Requirements — capability ledger (opt-in) **This capability exists whether or not the project uses it yet.** `requirement.functional` and `requirement.architectural` are registered metadata types, declared in `metaobjects/` beside the entities they describe and loaded by the same loader — no side file, no bespoke parser. They record *why* each part of the model exists, so a field with no reason to exist becomes visible as one. diff --git a/agent-context/skills/metaobjects-authoring/references/abstracts-and-overlay.md b/agent-context/skills/metaobjects-authoring/references/abstracts-and-overlay.md new file mode 100644 index 000000000..6de9b0113 --- /dev/null +++ b/agent-context/skills/metaobjects-authoring/references/abstracts-and-overlay.md @@ -0,0 +1,56 @@ +# Abstracts, `extends` and `overlay` + +## Contents +- Abstracts + `extends` (deferred resolution) + `overlay` + +## Abstracts + `extends` (deferred resolution) + `overlay` + +An **abstract** node (`abstract: true`) describes a shape but is never emitted as +a concrete entity. A concrete node references it via `extends:` to inherit its +children + attrs. This is the lightest reuse mechanism — pure data, no codegen +change. + +```yaml +- object.entity: + name: BaseEntity + abstract: true + children: + - field.long: { name: id } + - field.timestamp: { name: createdAt, required: true } + +- object.entity: + name: Author + extends: BaseEntity + children: + - source.rdb: { table: authors } + - field.string: { name: name, required: true } + - identity.primary: { name: id, fields: id } +``` + +Resolution facts: + +- **Deferred.** `extends:` resolves *after all files load* — abstracts can live in + any file, forward references are fine. +- **Multi-level chains resolve through the whole chain** (`Author extends BaseEntity + extends Auditable`) — each `extends` is a super-*reference*, so resolution walks the + full chain (it does not flatten inherited members onto the child; see ADR-0039). +- **Cross-package** refs use the fully-qualified name (`extends: "shared::auditable"`); + same-package refs use the bare name. +- An unresolved reference fails with `ERR_UNRESOLVED_SUPER`. + +`abstract` and `extends` are **structural keys** (bare, no `@`). + +**Inherited properties live on the parent** (`extends` is a super-reference, not a flatten). +A generator or provider must read through the resolving accessors, never `own*()` (ADR-0039; +the `metaobjects-codegen` skill has the per-port table). + +**`overlay` is a different concept.** `extends:` is an IS-A relationship between +two distinct nodes. `overlay: true` re-opens the *same* named node to amend it +across files (same `package` + same `name` → merged; last-writer-wins on attr +conflicts, structural children accumulate). Use `extends` to share shape between +distinct entities; use `overlay` to split one entity's declaration across files. + +**Building on a dependency's metadata** (`dependencies` in `.metaobjects/config.json`): +its nodes load before yours, so `extends` and `overlay: true` work across the boundary. Always +say `overlay: true` on an amendment to a node you do not own. The rules are in +`references/metadata-dependencies.md`. diff --git a/agent-context/skills/metaobjects-authoring/references/choosing-a-shape.md b/agent-context/skills/metaobjects-authoring/references/choosing-a-shape.md index 4fbb34974..7ae89655d 100644 --- a/agent-context/skills/metaobjects-authoring/references/choosing-a-shape.md +++ b/agent-context/skills/metaobjects-authoring/references/choosing-a-shape.md @@ -2,6 +2,11 @@ > Part of the `metaobjects-authoring` skill. Read it when no row of the skill's "Canonical form for common field needs" table fits, when you need the full rule behind a row (UUID, timestamps, URIs/IPs, `@autoSet`, validated strings), or when you are registering vocabulary of your own. +## Contents +- The decision procedure (ADR-0037) +- Subtype rules in detail +- Extending the metamodel — custom providers, and the downstream lifecycle + ## The decision procedure (ADR-0037) This procedure decides the shape of **any** concept entering the metamodel — a diff --git a/agent-context/skills/metaobjects-authoring/references/declaring-reports.md b/agent-context/skills/metaobjects-authoring/references/declaring-reports.md new file mode 100644 index 000000000..f1457ed7d --- /dev/null +++ b/agent-context/skills/metaobjects-authoring/references/declaring-reports.md @@ -0,0 +1,101 @@ +# Declaring dimensions, measures and reports + +How to declare a report. `reporting.md` covers the columns a report gets and how it behaves. + +## Contents +- Reporting — dimensions, measures and reports + +## Reporting — dimensions, measures and reports + +Reach for it when a dashboard number would otherwise be a hand-written `GROUP BY`: revenue per +day, buyers per program, a total. You name the pieces once, on the entity that owns the rows, +and an `object.report` combines them by name. Four node kinds: + +- `dimension.attribute` / `dimension.time` — what to group by (`@of: Entity.field`; a time + dimension lists the `@grains` it supports: `hour`, `day`, `week`, `month`, `quarter`, `year`); +- `measure.aggregate` (`@agg`: `count`, `sum`, `avg`, `min`, `max`) and `measure.ratio` + (`@numerator` / `@denominator`, both measures of the entity); +- `segment.filter` — a named, reusable `@filter` ("active purchase"); +- `object.report` — a top-level object: `@from` an entity, `@dimensions` (`name` or + `name:grain`), `@measures`. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Purchase", + "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "status" } }, + { "field.currency": { "name": "amountCents" } }, + { "field.timestamp": { "name": "purchasedAt" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "segment.filter": { "name": "active", "@filter": { "status": "active" } } }, + { "dimension.time": { "name": "purchasedAt", "@of": "Purchase.purchasedAt", + "@grains": ["day", "month"] } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id", + "@segment": "active" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents" } } + ] + }}, + { "object.report": { + "name": "DailyRevenue", + "@from": "Purchase", + "@dimensions": ["purchasedAt:day"], + "@measures": ["purchases", "revenue"], + "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_daily_revenue" } } + ] + }} + ] +}} +``` + +Three rules an author trips on: + +1. **Every measure belongs to `@from`.** A report cannot mix measures of two entities (joining + two fact tables multiplies each side's rows); two fact tables are two reports. +2. **`@via` is to-one only, and each hop needs a declared foreign key.** A dimension reaches a + related entity's column through a `relationship.*` with `@cardinality: one` (or an + `identity.reference`), never through a to-many, which would repeat fact rows and double-count + a `sum`. The view joins the hop through an `identity.reference` between the two entities + (`{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": + "Program" } }`); a relationship with none behind it loads and then fails `meta migrate`. +3. **A report declares no fields.** Its columns are derived: one per dimension, then one per + measure (a time dimension at a grain is ``, so `purchasedAt:day` is + `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. + +Two questions to answer for every report: + +- **Do I want the rows that have no facts?** A report's rows come from the facts, so a + program nobody bought has no row. If the page should list it, add `"@spine": + "Purchase.program"` to the report (a to-one path from `@from`, written like `@via`), and + **reach every listed dimension through it**: each needs an `@via` that begins with the + spine's hops (`programId` as `Program.id` via `Purchase.program`). A dimension on the fact + row is a load error under `@spine`. List the spine entity's key to get one row per program. +- **Is zero true here, or is null?** A `sum`, `avg`, `min`, `max` or ratio with nothing to + aggregate is null. If zero is the true answer (revenue of a program nobody bought), declare + `"@default": 0` on the measure; if the value is unknown (an average rating with no + ratings), leave it null. **Never on a `count`**: it is already `0`, and `@default` there is a + load error. Write `0`, never `0.0`. + +**A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration +is what makes `meta migrate` create the view (Postgres, SQLite, D1), what every port's +runtime reads, and what makes every port's generators serve it: one read-only list route at +the pluralized snake_case name (`InvoicesByMonth` is `/invoices_by_months`), with paging, +and filter and sort on every derived field whose type has filter operators (an `object` or +`map` derived field is neither filterable nor sortable), `405` on `POST` and no `/{id}` +route. A report with +no `source.*` is checked at load and generates nothing. + +A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its +base's table): declare it `@from` the base with an `@filter` on the discriminator field. + +What does not exist: no grid or form for a report (TypeScript generates a list hook; no other port has a client tier), no way to narrow which +derived fields are filterable, no `measure.derived` +(arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or +measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and +relative dates are UTC). Column types, the null rules, Monday weeks and per-engine differences +are in `references/reporting.md`. diff --git a/agent-context/skills/metaobjects-authoring/references/identities-and-indexes.md b/agent-context/skills/metaobjects-authoring/references/identities-and-indexes.md new file mode 100644 index 000000000..f21a5d007 --- /dev/null +++ b/agent-context/skills/metaobjects-authoring/references/identities-and-indexes.md @@ -0,0 +1,83 @@ +# Identities and indexes + +## Contents +- Identities +- Indexes (non-unique) + +## Identities + +| Subtype | Purpose | Key attrs | +|---|---|---| +| `identity.primary` | the PK field(s) | `@fields`, `@generation` | +| `identity.secondary` | a unique alternate key (always enforces uniqueness — uniqueness is the type, not a `@unique` attr) | `@fields` (or `@expr` for a functional index) | +| `identity.reference` | an inbound FK from this entity to another | `@fields`, `@references`, `@enforce` | + +`@generation` on a primary controls value generation (e.g. `increment`). +`@fields` accepts a single string in authoring; it normalizes to an array in +canonical JSON. `@enforce` on a reference (default `true`) controls whether the +backend physically enforces it (a SQL FK constraint); set `false` for a logical +reference for navigation/typing/codegen only. Referential actions +(`@onDelete`/`@onUpdate`) normally live on the `relationship.*` node (see +the Relationships section of `SKILL.md` — the subtype carries the default), but `identity.reference` +also registers them as the **explicit per-FK override** (ADR-0047): use them for +a reference-only FK with no relationship, an M:N junction's FK sides (no +relationship ever correlates with a junction FK), or a single FK that must +deviate from its relationship's action. A reference-level action always wins +over the correlated relationship's. + +`@references` resolves cross-package by **fully-qualified name** +(`@references: "shared::billing::Account"`), the same rule as `extends`; a bare +name resolves within the current package. The FK target must be an entity with a +single-column primary key (the FK points at that PK); a target with a composite +PK needs the explicit dotted form `@references: "pkg::Target.fieldA,fieldB"`. + +**A dangling reference fails the load (0.11.0+).** An unresolved +`identity.reference.@references` raises `ERR_INVALID_REFERENCE` and an unresolved +`relationship.@objectRef` raises `ERR_INVALID_RELATIONSHIP` — the target entity must +exist (previously such references loaded silently). So every `@references` / +`@objectRef` you author must name a real entity. + +An `identity.secondary` can index an **expression** instead of plain columns: use +`@expr` (e.g. `"lower(email)"`) in place of `@fields`, optionally with `@using` (the +index method — `gin` / `gist` / `hash`; default `btree`) and `@where` (a partial-index +predicate). + +```json +{ "identity.primary": { "name": "id", "@fields": ["id"], "@generation": "increment" } } +{ "identity.secondary": { "name": "byEmail", "@fields": ["email"] } } +{ "identity.secondary": { "name": "byEmailCI", "@expr": "lower(email)" } } +{ "identity.reference": { "name": "fkAuthor", "@fields": ["authorId"], "@references": "Author", "@enforce": true } } +``` + +## Indexes (non-unique) + +Use `index.lookup` for a **non-unique** DB index added purely for query performance — it +does NOT enforce uniqueness. Choose the right construct by what the constraint IS: + +| Need | Construct | +|---|---| +| Unique alternate key (e.g. email, slug) | `identity.secondary` — uniqueness is the type | +| Query-performance index, no uniqueness | `index.lookup` | + +**An index keys off EXACTLY ONE of `@fields` or `@expr`.** `@fields` names the indexed +columns; `@expr` is a raw key expression used **instead of** `@fields` (a functional index). +Declaring **neither** — or **both** — is `ERR_INVALID_INDEX`. This applies to +`identity.secondary` as well as `index.lookup`: uniqueness lives in the type, so a unique +index keys itself the same way. + +A legacy index declaring both is fixed by `meta upgrade --apply`, which keeps `@expr` (the +index the database actually has). Do not hand-pick the survivor. + +The db provider contributes physical-tuning attrs alongside either form: `@orders` +(per-column sort direction), `@using` (access method — `gin`/`gist`/`hash`; default +`btree`), and `@where` (partial-index predicate). + +```json +{ "index.lookup": { "name": "byCreatedAt", "@fields": ["createdAt"], "@orders": ["desc"] } } +{ "index.lookup": { "name": "byStatusCreatedAt", "@fields": ["status", "createdAt"] } } +{ "index.lookup": { "name": "byEmailCI", "@expr": "lower(email)" } } +{ "identity.secondary": { "name": "uniqLowerEmail", "@expr": "lower(email)" } } +``` + +`index.lookup` is a sibling of `identity.*` — declare it as a direct child of an `object.entity`, +at the same level as fields and identities. diff --git a/agent-context/skills/metaobjects-authoring/references/model-first-rationale.md b/agent-context/skills/metaobjects-authoring/references/model-first-rationale.md new file mode 100644 index 000000000..7937be392 --- /dev/null +++ b/agent-context/skills/metaobjects-authoring/references/model-first-rationale.md @@ -0,0 +1,22 @@ +# Why model-first wins + +The case behind the operating principle in `SKILL.md`: why declaring the model beats hand-writing a layer, even when the hand-written version is cheaper this once. + +## Contents +- Second source of truth +- The consistency tax +- One change, every layer + +**Why model-first wins even when hand-writing is cheaper this once — and it often +is, this once:** +- **Hand-writing a layer the metadata could own creates a second source of truth for + one fact.** A field's type, validation, column, route, and form then change in N + places and must stay consistent forever — not drift *risk*, but two sources of + truth for one fact, broken by construction. +- **The hand-roll saving is paid once; the consistency tax is paid on every future + change.** Assume the system will grow — it always does. The metadata amortizes + toward zero as the model is reused across layers and time; the hand-rolled + liability compounds with every field, refactor, and language port. +- **One metadata change regenerates persistence + DAO + API + UI consistently** — + and inherits every future generator improvement. Hand-writing opts out of all of + it, permanently. diff --git a/agent-context/skills/metaobjects-authoring/references/read-views-and-projections.md b/agent-context/skills/metaobjects-authoring/references/read-views-and-projections.md index 37cbd1b0c..c58b0e5ad 100644 --- a/agent-context/skills/metaobjects-authoring/references/read-views-and-projections.md +++ b/agent-context/skills/metaobjects-authoring/references/read-views-and-projections.md @@ -2,6 +2,10 @@ > Part of the `metaobjects-authoring` skill. The skill covers the entity read-view, which is the right answer for most grids. Read this when you need an independent exposure (`object.projection`), the `origin.*` vocabulary, `@filter` / `@expr` shapes, or an `@sql` / `@unmanaged` view. +## Contents +- Read-view or projection? +- Projections + ## Read-view or projection? **Reach for a projection only when one of these is true** (from diff --git a/agent-context/skills/metaobjects-authoring/references/reporting.md b/agent-context/skills/metaobjects-authoring/references/reporting.md index 6db3b1472..782404e8e 100644 --- a/agent-context/skills/metaobjects-authoring/references/reporting.md +++ b/agent-context/skills/metaobjects-authoring/references/reporting.md @@ -1,6 +1,20 @@ # Reporting: the columns a report gets, and how it behaves -> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, rows with no facts (`@spine`), defaults for empty measures (`@default`), what differs between databases, and what it leaves out. +> Part of the `metaobjects-authoring` skill. `declaring-reports.md` covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, rows with no facts (`@spine`), defaults for empty measures (`@default`), what differs between databases, and what it leaves out. + +## Contents +- A report is served only with a view source +- The columns you get +- Time grains +- Relative dates +- Nulls and zeros +- Rows with no facts: `@spine` +- Joins: a dimension through a required reference drops rows +- Engine differences +- Known limits +- How a report is served +- Exporting to Cube +- What a report does not have ## A report is served only with a view source diff --git a/agent-context/skills/metaobjects-authoring/references/requirements.md b/agent-context/skills/metaobjects-authoring/references/requirements.md index 8ccfc08bd..9007a9c5e 100644 --- a/agent-context/skills/metaobjects-authoring/references/requirements.md +++ b/agent-context/skills/metaobjects-authoring/references/requirements.md @@ -4,6 +4,24 @@ Capabilities are **metadata**, declared in `metaobjects/` beside the entities th describe. Read the existing requirement nodes before designing anything. Two rules matter more than the rest. +## Contents +- A requirement is PRESCRIPTIVE — it states what should be true, never what happened +- Do not delete a retired capability's entry +- When you add an entity, claim it +- Every requirement states its counterexample +- Four prose slots, and `statement` is the one that means "description" +- `title` is a LABEL and is chartered here; `summary` is not +- Never put a catalogue or ticket id in `title` +- The `name` is an address, so write it as an identifier +- Hierarchy is nesting, and links live at the bottom +- Write an L5 ref as a dotted `pkg::Owner.member` path +- L1–L3 are levels of abstraction and ownership in the problem domain +- Architectural requirements are the other kind +- Record gaps rather than rounding them off +- `status: planned` locks in work you have not started +- Which `verify` says what +- A requirement's name and package end up in generated test names + **1. A requirement is PRESCRIPTIVE — it states what should be true, never what happened.** `status` is `planned | live | partial | retired`, and every one of them states something meant to be true. `retired` is not the exception: it means **"this was built, then diff --git a/agent-context/skills/metaobjects-authoring/references/sources.md b/agent-context/skills/metaobjects-authoring/references/sources.md new file mode 100644 index 000000000..5f7f94c03 --- /dev/null +++ b/agent-context/skills/metaobjects-authoring/references/sources.md @@ -0,0 +1,97 @@ +# Sources and read-views + +## Contents +- Sources — `source.rdb` + `@kind` +- A store MetaObjects does not manage — MongoDB, Cassandra, Neo4j, an API +- Derived/computed columns: reach for an ENTITY READ-VIEW first + +## Sources — `source.rdb` + `@kind` + +`source.rdb` declares where an entity's data lives. Read-only-ness derives from +`@kind` (it is NOT a separate subtype): + +| `@kind` | Read-only | Default? | +|---|---|---| +| `table` | no | yes (when `@kind` omitted) | +| `view` | yes | – | +| `materializedView` | yes | – | +| `storedProc` | yes | – | +| `tableFunction` | yes | – | + +The physical name attr is **kind-matched** — never `@name`: `@table` for a table, +`@view` for a view, `@materializedView`, `@proc` (storedProc), `@function` +(tableFunction). (`@table` on a non-table kind is a pre-1.0 legacy spelling the +canonical serializer rewrites to the kind's alias; author the kind-matched attr +directly.) The physical column name on a field is `@column`. `@schema` namespaces +the DB schema (Postgres default `public`; SQLite rejects non-default values). +Multi-source: multiple `source.rdb` children, each with a `@role`, exactly one +`primary`. + +```json +{ "source.rdb": { "@kind": "view", "@view": "v_author", "@schema": "blog" } } +``` + +**This declaration is the only place a physical name is ever spelled.** Codegen emits a +per-object names artifact from it (`Names` — `ProgramNames.fields.createdAt.column`, +`ProgramNames.CreatedAtColumn`, `PROGRAM_CREATED_AT_COLUMN`, per port), the generated table +binding reads that artifact, and hand-written SQL, repositories and migration scripts +reference it — so a consumer never restates `v_author` or `created_at`, and a rename here +propagates. Declare `@column` explicitly whenever the physical name is not the naming +strategy's answer (`callPurpose` → `purpose_code`): nothing downstream can recover that +mapping by derivation, and the constant is what carries it. (See `metaobjects-codegen` → +"Never hand-write a physical name".) + +**An entity's PRIMARY source must be writable** (`table`) — read-only kinds are +legal only in non-primary roles. + +### A store MetaObjects does not manage — MongoDB, Cassandra, Neo4j, an API + +`source.rdb` is the only registered source. A record kept in a document, wide-column or +graph store, or behind a remote API, is an `object.entity` with an `identity.primary` and +**no `source.*` child**: a *sourceless entity*. Do not invent a document or graph source +subtype, or a `@collection` attr: ADR-0007 designs them but registers only `rdb`, so the +loader refuses them. Put +store settings in your own property bag instead: any `@name` whose value is a JSON object is +a registered `attr.properties` bag, for example `"@mongo": { "collection": "orders" }`. + +A sourceless entity gets no table, queries, routes or migration. It still gets its wire +contract in every port: the create and PATCH schemas and the filter allowlist. Keep the +`identity.primary`, because it is what makes the record addressable. Without it the object +is a shape only. The data access layer is a generator you own: the procedure is in the +`metaobjects-codegen` skill, `references/any-stack.md`, and on TypeScript +`references/typescript-document-store.md` carries a complete, tested MongoDB repository +generator to copy. + +**The database dialect is codegen config, not metadata.** MySQL, Postgres, SQLite and D1 +entities are all modeled with `source.rdb`; `dialect` in the codegen config picks the target. +Do not encode the database in the model. + +### Derived/computed columns: reach for an ENTITY READ-VIEW first + +**Do not default to `object.projection` for a list screen, grid, or "one extra +column" read.** The usual need — an entity's own rows plus a joined display name or a +per-row count, filterable and sortable like any other column — is an **entity +read-view**, not a projection: + +```jsonc +{ "object.entity": { "name": "Order", "children": [ + { "source.rdb": { "@kind": "table", "@table": "orders" } }, // writes + { "source.rdb": { "@kind": "view", "@view": "v_order", "@role": "replica" } }, // reads + // …the entity's own fields stay as they are — you re-state NOTHING… + { "field.string": { "name": "customerName", "@filterable": true, "children": [ + { "origin.passthrough": { "@from": "Customer.name" } } ]}}, + { "field.int": { "name": "itemCount", "@filterable": true, "children": [ + { "origin.aggregate": { "@agg": "count", "@of": "OrderItem.id", "@via": "Order.items" } } ]}} +]}} +``` + +That is the whole change. Writes route to the table (derived fields are excluded from +the write codecs); reads route to the view; `meta migrate` emits the `CREATE VIEW` from +the **same assembly logic** a projection view uses — one emitter, two hosts (FR-024 §7, +#213/#214). Filtering and sorting on `customerName` / `itemCount` work like any other +column because the filter tier runs against the view. + +**When you need an independent exposure** — a subset, renamed columns, a row-filtered +view, a versioned or external shape — that is an `object.projection`. It, the `origin.*` +vocabulary, the structured `@filter` / `@expr` shapes and the `@sql` / `@unmanaged` escapes +are in `references/read-views-and-projections.md`. diff --git a/agent-context/skills/metaobjects-codegen/SKILL.md b/agent-context/skills/metaobjects-codegen/SKILL.md index a92b5352a..c1f048a88 100644 --- a/agent-context/skills/metaobjects-codegen/SKILL.md +++ b/agent-context/skills/metaobjects-codegen/SKILL.md @@ -28,7 +28,8 @@ and command for *this* project's server language live in a reference fragment ## Reference files — open only the one your task needs -Your server language's file (`references/typescript.md`, `references/java.md`, …) has the +Your server language's file (`references/typescript.md`, `references/java.md`, +`references/kotlin.md`, `references/csharp.md` or `references/python.md`) has the exact config, generator names and commands for this project; read it. The other files in `references/` are topics — open one when its row matches what you are doing. @@ -40,6 +41,11 @@ exact config, generator names and commands for this project; read it. The other | `references/template-or-code.md` | choosing between a code generator and a Mustache template | | `references/physical-names.md` | before you write a table, column or index name as a string | | `references/dependencies-and-shared-models.md` | the project consumes or publishes metadata across projects | +| `references/typescript-document-store.md` | a TypeScript project on MongoDB, or any store MetaObjects does not manage, wanting a tested repository generator | +| `references/typescript-mysql.md` | a TypeScript project on a MySQL database | +| `references/typescript-retargeting.md` | a TypeScript project on a framework other than Fastify, Hono or React | +| `references/typescript-templates.md` | declarative Mustache template-codegen in a TypeScript project | +| `references/typescript-docs.md` | `meta docs` in a TypeScript project | ## Need an output? Write the generator — this is the primary path diff --git a/agent-context/skills/metaobjects-codegen/references/any-stack.md b/agent-context/skills/metaobjects-codegen/references/any-stack.md index f69c1bbdb..57f1c9651 100644 --- a/agent-context/skills/metaobjects-codegen/references/any-stack.md +++ b/agent-context/skills/metaobjects-codegen/references/any-stack.md @@ -10,6 +10,14 @@ generators cover some stacks. Every other stack is reached the same way: keep th source of truth, generate what is built in, and write or retarget a generator for the rest. Nothing below needs a new MetaObjects release, a new metamodel attribute, or permission. +## Contents +- Find out what is built in — do not guess +- Split the app into layers and decide each one +- Persistence +- An HTTP framework that is not built in +- A UI framework that is not built in +- Prove it works + ## 1. Find out what is built in — do not guess ```bash diff --git a/agent-context/skills/metaobjects-codegen/references/csharp.md b/agent-context/skills/metaobjects-codegen/references/csharp.md index 03eeb622d..8c829447b 100644 --- a/agent-context/skills/metaobjects-codegen/references/csharp.md +++ b/agent-context/skills/metaobjects-codegen/references/csharp.md @@ -5,6 +5,16 @@ The C# port targets .NET consumers (EF Core + ASP.NET). Codegen runs through the is **schema-migrations only** on the C# side (ADR-0015): `meta migrate` / `meta verify --db` are Node-`meta`-owned; everything below is `dotnet meta`. +## Contents +- Install +- Run +- `MetaObjects.Codegen` generators +- Docs — `dotnet meta docs` +- Persistence + routes are the deployed artifact +- Write your own generator +- Extending the generators (open-for-extension, ADR-0002) +- Re-scaffold this context + ## Install Per the always-on descriptor: diff --git a/agent-context/skills/metaobjects-codegen/references/java.md b/agent-context/skills/metaobjects-codegen/references/java.md index bdfd52171..d8e1855ab 100644 --- a/agent-context/skills/metaobjects-codegen/references/java.md +++ b/agent-context/skills/metaobjects-codegen/references/java.md @@ -4,6 +4,17 @@ The Java port targets Spring Boot consumers on Maven. Codegen runs as a build-ti Maven plugin goal — there is no standalone `meta` binary on the Java side (the Node `meta` is for schema migrations only; see the migration reference). +## Contents +- Maven coordinates +- Plugin config in `pom.xml` +- Run +- Write your own generator +- When a generator's output is wrong — own the generator +- Docs — `mvn metaobjects:docs` +- `codegen-spring` generators +- Discriminator inheritance (TPH) +- Serializing generated objects + ## Maven coordinates All artifacts share `groupId` `com.metaobjects` and one version property: diff --git a/agent-context/skills/metaobjects-codegen/references/owning-generators.md b/agent-context/skills/metaobjects-codegen/references/owning-generators.md index 839454b75..3de6cf6b9 100644 --- a/agent-context/skills/metaobjects-codegen/references/owning-generators.md +++ b/agent-context/skills/metaobjects-codegen/references/owning-generators.md @@ -2,6 +2,13 @@ > Part of the `metaobjects-codegen` skill. Read it when generated output is wrong, when your framework is not the one a reference emits for, or when you write or review a generator (the `own*()` accessor table is here). +## Contents +- A defect in generated code is a defect in your generator +- Run it as shipped, or own a copy +- A generator of your own beside the references +- Your framework isn't the default — the retargeting procedure +- Never read metadata through an `own*()` accessor (ADR-0039) — top bug source + ## A defect in generated code is a defect in your generator When generated output does not compile, has the wrong shape, or collides with code you diff --git a/agent-context/skills/metaobjects-codegen/references/physical-names.md b/agent-context/skills/metaobjects-codegen/references/physical-names.md index 89085e8fe..5086f31cd 100644 --- a/agent-context/skills/metaobjects-codegen/references/physical-names.md +++ b/agent-context/skills/metaobjects-codegen/references/physical-names.md @@ -9,6 +9,13 @@ free-form, so a field named `callPurpose` may map to a column named `purpose_cod neither the field name nor any transformation of it. A consumer deriving the column as `to_snake_case()` gets that case wrong and never finds out. +## Contents +- Reference the generated names artifact +- Check that it is actually wired before you reference it +- It follows `extends` +- Where generated code consumes it, and where it does not +- But prefer a typed handle where one exists — this rule has a real limit + Every port emits a per-object names artifact. Reference it: ```ts diff --git a/agent-context/skills/metaobjects-codegen/references/python.md b/agent-context/skills/metaobjects-codegen/references/python.md index 62c3510e1..e181065cb 100644 --- a/agent-context/skills/metaobjects-codegen/references/python.md +++ b/agent-context/skills/metaobjects-codegen/references/python.md @@ -6,6 +6,17 @@ are **Node-`meta`-owned** (ADR-0015): `meta migrate` / `meta verify --db` run through the Node `meta` tool — the Python CLI has **no `migrate` subcommand** and `metaobjects verify --db` is rejected. Everything below is `metaobjects`. +## Contents +- Install +- Run +- Docs — `metaobjects docs` +- Write your own generator +- Generators +- Discriminator inheritance (TPH) +- No ORM — you own persistence (unlike the C# port) +- Known gaps (current — may require a hand-edit) +- Re-scaffold this context + ## Install ```bash diff --git a/agent-context/skills/metaobjects-codegen/references/typescript-docs.md b/agent-context/skills/metaobjects-codegen/references/typescript-docs.md index 01e2b70ec..778eab520 100644 --- a/agent-context/skills/metaobjects-codegen/references/typescript-docs.md +++ b/agent-context/skills/metaobjects-codegen/references/typescript-docs.md @@ -6,6 +6,14 @@ Documentation is NOT a `meta gen` generator. The single door is the `meta docs` command, which emits three cross-linked **surfaces** under one output dir (default `./docs`): +## Contents +- Never wire `docsFile()` or `apiDocsFile()` into `generators` +- An owned docs generator goes in `docs: { generators: [...] }`, never in `generators` +- There is no `mermaidErDiagram()` generator +- Before calling any generated code, read `./docs/api/AGENT-API.md` +- Ejecting routes hands over the adapter too +- Wire a generator only for output you consume, and narrow it with its `filter` + - **model surface** (`./docs/.md`, `./docs/